2nd hand U.2 SSD usage problem

[randomname]

I am having trouble with a U.2 drive with a PCI-E adapter card, hoping someone can help me out.

Drive : Netlist N1962 U.2 15.36TB PCIe 4.0 (NS1962UF115T) https://netlist.com/wp-content/uploads/2021/03/Netlist-NVMe-SSD-N1962-Product-Brief.pdf
PCIe Adapter Card : Glotrends PU21 PU21 Dual U.2 to PCIe 4.0 X8 Adapter without PCIe Bifurcation
Motherboard : Asus ProArt X670E
CPU AMD 7950X

The problem I'm having is that the drive is recognised both in BIOS and Windows, and if I do small file copies (10s of MB) it works fine. But if I move a big file (e.g. a couple of GBs) or if I do a benchmark using CrystalDiskMark, the drive stops responding and sometimes disappears from Windows. After reboot, however, it works, only to disappear again if I do the same thing.

Once the copying started, the drive decided "Nope" and shut itself off (or similar). There is a blue power I'm guessing) LED on the drive that stays on even after it "disappears" in Windows though.

As you can see the adapter board has two U.2 slots. I've tried both, I've also tried the two PCIE slots on my motherboard, same problem in all of these. They all recognised the drive though, so it's likely not a problem with the bifurcation.

I know Wendell from level1techs has done a video

about these adapters (including M.2 ones, I think he recommended this M.2 M-key PCIe Gen4 with ReDriver to SlimSAS 4i (SFF-8654) Adapter) so I am not completely ruling out the possibility that the adapter I have is not "good enough", however, I am not experienced enough to tell if this is indeed a problem with the adapter card, or if it's something else (e.g. the drive itself). This is the first time I used a U.2 drive or any sort in any configuration.

For what it's worth, the drive is a 2nd drive from eBay, "pulled from unused server", the seller insists it was tested before shipment. I have no reason to question him for now, but, nevertheless, if there's something else I can test, that would also be great.

OR, would this combination work if it is indeed a problem with the adapter card? (yes I will need two U.2 connections) PCIe x8 Gen4 with ReDriver to SlimSAS 8i (SFF-8654) Add-in-Card + SlimSAS 8i (SFF-8654) Cable to Two U.2 Connectors - 50 CM

I am choosing these components purely because "Wendell says the M.2 adapter with redriver from this brand works so maybe the PCIE version with redrivers would also work"...

At this moment I only have 1 U.2 drive so I am unable to use another drive to see if it works. And anyway, my second incoming drive will be an Intel 905P, so that's PCIe 3.0, which is more forgiving, so I don't know if that would actually help me troubleshoot.

I do have a 12cm fan blowing directly at the drive, so while cooling might not be the best compared to say, being inside a server, there's still *some* cooling, not none.

Any help is appreciated. Thank you very much in advance.

 

[Tech Junky]

These things can be a real PITA sometimes. I went through 2 micron drives and different PCB mount options and ended up sending them back when they wouldn't work in under a week each.

My working solution for the past ~9 months now is using OCULINK M2 / cable and a Kioxia drive. The initial system though was an Intel and it spit out occasional errors that wanted a disk check to clear them but, I switched to AMD and those went away completely.

So, here's what I would do if you need dual drives....
Get an oculink card w/ 2 ports which will provide you access to a total of 4 drives with 2 split cables / card ~$50 / cables ~$50/ea

I'm using my drive though with a M2/cable but, when looking into the U idea initially I was thinking if I made the switch to AMD I would occupy one slot with the card mentioned above but, ended up needing to add a GPU for Plex stuff. I could still move stuff around and get a card with a single port and split to 2*U down the road. With new AMD MOBO's coming soon with features that could free up some slots it would open the door to another rebuild to open more options.

Anyway.... Oculink for me was the only surefire solution for getting the bandwidth from the drive w/o complicating things with older tech i.e. SAS

While most of the adapters are passthrough and should yield whatever the bus provides for speed I didn't see the advantage of using legacy connectors / adapters vs using something current. Each OCU interface though can provide x4 to each drive and hit the current Gen 4 speeds. Whether they will hit Gen 5 speeds is something to be tested if someone has a G5 drive.

 

[randomname]

These things can be a real PITA sometimes. I went through 2 micron drives and different PCB mount options and ended up sending them back when they wouldn't work in under a week each.

My working solution for the past ~9 months now is using OCULINK M2 / cable and a Kioxia drive. The initial system though was an Intel and it spit out occasional errors that wanted a disk check to clear them but, I switched to AMD and those went away completely.

So, here's what I would do if you need dual drives....
Get an oculink card w/ 2 ports which will provide you access to a total of 4 drives with 2 split cables / card ~$50 / cables ~$50/ea

I'm using my drive though with a M2/cable but, when looking into the U idea initially I was thinking if I made the switch to AMD I would occupy one slot with the card mentioned above but, ended up needing to add a GPU for Plex stuff. I could still move stuff around and get a card with a single port and split to 2*U down the road. With new AMD MOBO's coming soon with features that could free up some slots it would open the door to another rebuild to open more options.

Anyway.... Oculink for me was the only surefire solution for getting the bandwidth from the drive w/o complicating things with older tech i.e. SAS

While most of the adapters are passthrough and should yield whatever the bus provides for speed I didn't see the advantage of using legacy connectors / adapters vs using something current. Each OCU interface though can provide x4 to each drive and hit the current Gen 4 speeds. Whether they will hit Gen 5 speeds is something to be tested if someone has a G5 drive.

Thanks very much for the reply.

Yes I deliberately avoided SAS drives because I did not want to bother with an active card (HBA) for the drive. That and the comparatively slower speeds. The reason I went for that card that I have is also because of the U.2 connector on-board. I did not want to bother with any problems (if any) with additional cables. I was hoping the signal traces are short enough for it not to be too much of a problem. I guess my most ideal card would be one that's similar to what I have but with redrivers. I also consciously wanted to avoid M.2 cards as I have some other M.2 drives that would actually occupy the M.2 slots.

Nevertheless, if you don't mind me asking, can you share which Oculink M.2 card you're using? And is your Kioxia drive PCIe3 or 4 (or 5)? And, when you say "legacy connectors/adapters vs using something current" are you saying the PCIE slot is not current? Just to understand better.

Thank you!

 

[Tech Junky]

I think we're of the same mindset on some of this. I wanted to go the PCB route as well since I thought it would be the best idea but, after 2 drives / 2 PCB cards and 0 results I changed my mind and drive brand.

https://www.amazon.com/gp/product/B09HTPWLXM - M2
https://www.amazon.com/gp/product/B09BFL485F - cable
KCD8XRUG15T3 - drive / Gen 4

are you saying the PCIE slot is not current? Just to understand better.

Just didn't want to use the $1000+ drive with legacy SAS protocols if they could slow things down.


I'm a network guy but got bored a few years back and dove into collapsing several devices into a single "PC" for simplicity. I rolled in 6-7 devices / functions into a single box and have been upgrading/rebuilding it ever since. I've been through at least 5 cases / mobo / etc. over the years with different ideas and the latest was to convert the ton (weight) of spinners into a SSD setup. Initially it lead me down the path of 4*M2 before stumbling upon U drives instead. The appeal of the U drives was the cost/capacity and getting beyond 8TB per drive. I had been running Raid 10 and M2's topping out at 8TB / G3 speeds vs a single U 16TB at G4 was more appealing for not only the speed but not having to mess with it in a couple of years. Dropping the raid part of it too was appealing to keep the costs down on having to get multiple drives. R10's appeal really was the faster bandwidth @ 400MB/s+ while still being relatively cheap. Since the PC is the router there's no bottlenecks to deal with other than ISP speed at this point. Considering the Kioxia does 6.5GB/s there's the client side that can't hit those speeds w/o upgrading the network considerably but, laptop's don't offer much in terms of high end network speed.

 

[randomname]

I think we're of the same mindset on some of this. I wanted to go the PCB route as well since I thought it would be the best idea but, after 2 drives / 2 PCB cards and 0 results I changed my mind and drive brand.

https://www.amazon.com/gp/product/B09HTPWLXM - M2
https://www.amazon.com/gp/product/B09BFL485F - cable
KCD8XRUG15T3 - drive / Gen 4


Just didn't want to use the $1000+ drive with legacy SAS protocols if they could slow things down.


I'm a network guy but got bored a few years back and dove into collapsing several devices into a single "PC" for simplicity. I rolled in 6-7 devices / functions into a single box and have been upgrading/rebuilding it ever since. I've been through at least 5 cases / mobo / etc. over the years with different ideas and the latest was to convert the ton (weight) of spinners into a SSD setup. Initially it lead me down the path of 4*M2 before stumbling upon U drives instead. The appeal of the U drives was the cost/capacity and getting beyond 8TB per drive. I had been running Raid 10 and M2's topping out at 8TB / G3 speeds vs a single U 16TB at G4 was more appealing for not only the speed but not having to mess with it in a couple of years. Dropping the raid part of it too was appealing to keep the costs down on having to get multiple drives. R10's appeal really was the faster bandwidth @ 400MB/s+ while still being relatively cheap. Since the PC is the router there's no bottlenecks to deal with other than ISP speed at this point. Considering the Kioxia does 6.5GB/s there's the client side that can't hit those speeds w/o upgrading the network considerably but, laptop's don't offer much in terms of high end network speed.

I can't help noticing your M.2 adapter claims PCIE3 support only. Do you actually get PCIE4?

 

[Chriggel]

I do have a 12cm fan blowing directly at the drive, so while cooling might not be the best compared to say, being inside a server, there's still *some* cooling, not none.

Are you sure that this isn't a temperature issue? These could be symptoms of the drive overheating.

U.2 drives use a lot of power, even when idle, so their base line temperature is already fairly high. Depending on how the drive is engineered, the top cover doesn't act as a heatsink. This is at least the case for the Kioxia CM6 I was testing, but it's probably true for other manufacturers as well. I recently discovered this when doing tests about the viability of U.2 drives in desktop workstations.

I think these drives benefit massively from the air flow in 19" rack servers with hot air pulled away from them, potentially through air vents in the top cover, like the Kioxias have. I still need to verify how much exactly they benefit from that.
If you mount them to an adapter card, the bottom side gets partially cut off from fresh air, the PCB cut outs are there for a reason but potentially not enough. Hot spots around the SSD when hot air can't escape is also a problem.

During my tests, best case scenario was about 35°C over ambient. Worst case, 50°C+. And that's while idle. This was already near or even beyond the specs, I didn't even bother doing load tests. Wouldn't have been possible.

 

[Tech Junky]

@Chriggel is onto the heat issue as well with these drives. I never took readings on the micron drives but someone I was talking to picked up a couple and reported high temps around 70c but mounted fans directly to them and got a bit more reasonable temps.

The CD8 idles at 40c though which is inline with the M2 temps I see. I actually just have the drive laying in the case rather than mounted somewhere.

 

[randomname]

Thanks for the replies @Tech Junky @Chriggel.

I did try to eliminate that problem by basically putting a 12cm fan spinning at 2000rpm blowing at the SSD directly. So it should not be a problem.

For what it's worth, the drive in HWiNFO64 reports 40-41C regardless what it's doing until it stops working in Windows, so I'm taking that number with a pinch of salt...

 

[jei]

@Chriggel is onto the heat issue as well with these drives. I never took readings on the micron drives but someone I was talking to picked up a couple and reported high temps around 70c but mounted fans directly to them and got a bit more reasonable temps.

The CD8 idles at 40c though which is inline with the M2 temps I see. I actually just have the drive laying in the case rather than mounted somewhere.

I have Micron 7450 Pro 15TB idling 45C in 32C ambient with some basic airflow. CD8 and 7450 have identical power specs so I don't know where all this data about vastly different results come from. The drives have a different kind of heatsink so with sample size of 1 maybe it's just an issue with the airflow/case configuration.

 

[Tech Junky]

taking that number with a pinch of salt...

For me with Micron it seemed like the drive would "drop out" on Intel and a reboot fixed it but then it got to the point where it just wouldn't mount to the OS for use. Both drives did this within a week or in the case 4 hours for one of them. A clue was when trying to access the share from Windows is it would report an I/O error indicating a dismount on the server side.

sample size of 1 maybe it's just an issue with the airflow/case configuration.

2 drives from different sellers and 2 different PCB boards

switching brands and to a cable/adapter setup resolved most of it but, still the occasional error popping up which pushed the switch to AMD I had been pondering prior to the switch to flash. Spending time on troubleshooting the issues with Micron wasn't something I felt like doing with tying up the cash and being under the gun for return periods. I did spend more time on the second one though but couldn't come up with a solution to get it to take / hold data. Something on both drives zapped the partition table from them but they were still electrically visible from the OS.

CD8 and 7450 have identical power specs so I don't know where all this data about vastly different results come from.

I can't say because the other person reported the temps from their new build. I didn't bother monitoring the temps on the drives I had briefly before they became useless. Had they lasted longer I would have gathered more stats on them. All I have is 2 drives that became useless within their return period and got sent back.

 

[jei]

2 drives from different sellers and 2 different PCB boards

switching brands and to a cable/adapter setup resolved most of it but, still the occasional error popping up which pushed the switch to AMD I had been pondering prior to the switch to flash. Spending time on troubleshooting the issues with Micron wasn't something I felt like doing with tying up the cash and being under the gun for return periods. I did spend more time on the second one though but couldn't come up with a solution to get it to take / hold data. Something on both drives zapped the partition table from them but they were still electrically visible from the OS.

Ah, I meant the temperature statistics. But while we're at it, maybe 12 threads about problems with 1 Micron configuration is enough.

 

[Tech Junky]

Ah, I meant the temperature statistics. But while we're at it, maybe 12 threads about problems with 1 Micron configuration is enough.

Or maybe you could just ignore it as I only mentioned it as a comparison to the OP's experience with their brand acting in the dame manner.

 

[randomname]

Just an update, tried a Live CD Linux, same problems...

 

[Tech Junky]

Well, I would try a different adapter method or wait for the Intel drive to show up and test with that. If the problem follows with a different drive it might point to the adapter being the issue. If you swap adapters and it still persists then I would be looking at the slots. If you switch to a cable make sure it's 50cm or less as the signal tends to drop out and the drive won't come online.

 

[Chriggel]

Yeah, if it's not a temperature problem and also not a software problem, it's most likely a hardware issue, maybe physical connectivity or signal integrity. The usual suspect would be the ASMedia PCIe switch on the adapter card, so I'd try any connection method without it next.

This adapter looks sketchy as hell. The description of the adapter claims it uses an ASM1182e. It further claims that it does PCIe 4.0 x8 to x4x4 bifurcation. ASMedia says their chip is a 1x PCIe 2.0 x1 to 2x PCIe 2.0 x1 switch.

 

[Tech Junky]

This adapter looks sketchy as hell.

The description on these things is mind boggling as they're cheap and don't translate very well. It's all salt until it's installed and tested in my book. Sure, reviews by users help mitigate some frustration when they find bad products but, I've ordered stuff that comes with an insert asking for 5 star reviews for $20 gift cards to bypass the paid review flags.

I would go by the specs on the ASM side though and disregard anything else. With some things I search by the controller model vs just looking for something generic. A big category for fluff results are TB cards with old controllers or enclosures as well.

 

[nexox]

I'm pretty sure the adapter linked in the initial post doesn't have any ASMedia chips, because it clearly says it requires bifurcation support on the motherboard, I think these sort of listings either add that to get search hits or from copy paste errors.

 

[randomname]

The discussion regarding the ASM1182 chip made go and have a close look at the (only) chip on that board.

Google says the ASM1182 looks like this, a square (more or less) chip


There's nothing on that board that is vaguely a square.



There's nothing on the back. The chip I circled is not anything from ASM as far as I can see, but this PCIe 3.0 Clock Buffer

PI6C20400B

1:4 Clock Driver for Intel PCIe® 3.0 Chipsets

www.diodes.com

https://www.diodes.com/assets/Datasheets/PI6C20400B.pdf

Maybe this explains why I'm having problems with PCIe 4.0?

 

[jei]

If you do not have bifurcation enabled for that slot, it would explain a lot. Even the janky manufacturer says this card needs bifurcation.

 

[nexox]

Maybe this explains why I'm having problems with PCIe 4.0?

Yeah that seems not great for 4.0, it appears that all their 4.0 and later generation buffer chips come in different/more modern packages, which suggests the SSOP chip you have may not physically be capable of supporting the higher frequencies due to fun effects like inductance in the pins.