There's two that I know of - BPN-NVME5-LA26-S12 (I bought it here), and BPN-SAS3-LA26A-N12. I'm not sure what the difference is. I found the manual for the latter, and it seems to have the same connector layout. It might just be gen4 SlimSAS vs gen5 MCIO connectors.
Whats the deal with U.2 enterprise SSDs selling so cheap?
- Thread starter MC68000
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Probably this and higher quality pcb/traces for the newer pcie version signals
Yeah exactly, difference in connectors and PCIe Gen. The Gen4 backplane won't run Gen5 reliably...
I would love to find an Intel/Supermicro/[other trusted brand or otherwise known good] Gen3 M.2 to U.2 adapter cable for even €20 + shipping. Or, say, €30 for one that was likely to work with Gen4. Any links/tips about such cables? (EU sellers?)
The ones I've managed to find are no-name and ~€30 for the M.2 card (with either SFF-8643 or SFF-8611 connector) + an additional ~€30-€50 for the cable (incl shipping from two sellers). I've also seen some 20 cm long direct adapters from M.2 to U.2 for something like €65. Feels like very slim pickings! Maybe consumer retailers and ebay.de aren't the right places to look?
@homeserver78
It's going to run ~$50 for a M2/cable setup which gives you flexibility of placement / airflow or you can get 1/2/4 PCB based options that just mount to the slot anywhere from $20-$75.
If you're using Intel and it's not HEDT then it won't support bifurcation and M2 will be the cheaper option unless you have an x4 slot to put a PCB adapter into.
Those are the options.
It's going to run ~$50 for a M2/cable setup which gives you flexibility of placement / airflow or you can get 1/2/4 PCB based options that just mount to the slot anywhere from $20-$75.
If you're using Intel and it's not HEDT then it won't support bifurcation and M2 will be the cheaper option unless you have an x4 slot to put a PCB adapter into.
Those are the options.
Nitpick: It won't support bifurcation down to x4/x4/x4/x4. Bifurcation is on (nearly?) all systems with PCIe, just not always in the most flexible configurations. Recent modest Intel CPUs sometimes support x8/x4/x4, which can also be interesting, particularly when presented physically as two x8 slots.
Yeah, that's pretty much what I've concluded as well. That's why I was intrigued by the mention of $14 cable adapters from Intel!
For context:
I'm running a home NAS+hypervisor; a bit of a Frankenstein's monster based on a Ryzen 4650G (for its ECC capability) with one Intel P4610 (bought used, Oracle FW) in a PCIe x4 PCB adapter in the single x16 slot on the motherboard (for VMs, torrents, ...), one Crucial P3 in the single M.2 slot (for "cold" storage of movies and such), and one WD60EFZX HDD for first-line backups. I'd like to find a way to free up the x8x4x4-capable PCIe slot for more enterprise drives in the future (but there's no hurry). The only 4 U.2 slot PCB option I've found is the DeLock 90092 but it's north of €250, and as you say, a cabled solution would be much preferred for flexibility and cooling.
I wonder if DeLock 90111 works. They claim PCIe v5.0 x16 -> 4x SFF-8654 (SlimSAS?) 4i. Does SlimSAS even support PCIe Gen 5? I just hate that so many manufacturers shamelessly lie about the capabilities of their products; it makes things very difficult and wastes a lot of time. I certainly don't want to give any money to a company that lies to my face.
I'd also like to be able to get an enterprise NVMe drive for my desktop, where the only available option is an M.2->U.2 cable adapter. But until there's a sane solution I guess I'll just live with my old SATA drives, hehe.
@homeserver78
I would either go all in on the PCB or get a dumb quad M2 adapter and split the functions between m2 drives and some adapters. Then you maintain the speed and diversify for larger U drive capacity and only tie up a single slot. Even if it means a mobo upgrade to get x4*4 instead of that odd x8/x4*2. I went with an ASRock board because all others did odd lane splits when adding a second card.
I would either go all in on the PCB or get a dumb quad M2 adapter and split the functions between m2 drives and some adapters. Then you maintain the speed and diversify for larger U drive capacity and only tie up a single slot. Even if it means a mobo upgrade to get x4*4 instead of that odd x8/x4*2. I went with an ASRock board because all others did odd lane splits when adding a second card.
It would not make sense to buy DELOCK 4x4x4x4 bifurcation adapter for 250€ when you can get Supermicro NVMe switch (no need for bifurcation) for 110€.
Supermicro AOC-SLG3-4E2P ab € 106,77 (2024) | Preisvergleich Geizhals EU
✔ Preisvergleich für Supermicro AOC-SLG3-4E2P ✔ Produktinfo ⇒ Anbindung: PCIe 3.0 x8 • Schnittstellen: 4x OCuLink/SFF-8611 • Intern: 4x OCuLink/SFF-8611 (PCIe 3.0 x4) … ✔ Storage Controller ✔ Testberichte ✔ Günstig kaufen
geizhals.eu
If you really want 4x4x4x4, this one is 230€ cheaper:
This would be the ideal solution, if it worked. I would certainly be prepared to give something like the Asus Hyper M.2 X16 Gen 4 plus a known good Gen 4 M.2->U.2 adapter kit (from a single manufacturer) a try. Hooking up three adapters in series, all from different manufacturers and two of which only works for Gen 3 individually and if you're lucky - not so much... Or have you actually tried this and confirmed it works? If so, what parts did you use, exactly?
The DeLock 90111 is ~€140, but yeah, I would much rather spend that money on a Supermicro solution. Then again, with retimers and switches comes increased power draw. ATM my NAS draws <25 W at idle, so even +5 W is noticeable. Perhaps I should try to find a AOC-SLG3-4E4R redriver board? Should have a higher chance of success than the passive solutions at least?It would not make sense to buy DELOCK 4x4x4x4 bifurcation adapter for 250€ when you can get Supermicro NVMe switch (no need for bifurcation) for 110€.
Supermicro AOC-SLG3-4E2P ab € 106,77 (2024) | Preisvergleich Geizhals EU
✔ Preisvergleich für Supermicro AOC-SLG3-4E2P ✔ Produktinfo ⇒ Anbindung: PCIe 3.0 x8 • Schnittstellen: 4x OCuLink/SFF-8611 • Intern: 4x OCuLink/SFF-8611 (PCIe 3.0 x4) … ✔ Storage Controller ✔ Testberichte ✔ Günstig kaufen
If you really want 4x4x4x4, this one is 230€ cheaper:
Hah, that Ceacent PCIe x16 -> 4x SFF-8643 board really is €230 cheaper!
Any idea if it actually works? And if so, using which cables? About cables, there's at least four different connectors in use:
- SFF-8643 "MiniSAS"
- SFF-8654 "SlimSAS" (4i and 8i variants)
- SFF-8611 "OCuLink" (4i and 8i variants)
- SFF-TA-1016 "MCIO" (8i - is there a 4i variant as well?)
Yeah, that's the time for this stuff
I found it best with PM983 8Tb drives - tons of these on ebay new and used got from 550 down to 250 in past 2 years.
For chassis - sas SYS-1028U or if lucky SYS-1029U, upgrade to NVMe backplane, some cabling, risers or even AOCs - 10xNVMe X10/X11ultra server is ready to run.
2U SYS-2029U or SYS-2028U are damn rare and still expensive, but I hope there's a way to build almost the same on regular X11DPU in 2U SAS SYS-2029U SYS-2029UZ-TN20R25M | 2U | SuperServers | Products | Super Micro Computer, Inc.
just need that passive NVMe BPN-NVMe3-216N-S4 and some proper risers and AOCs
I found it best with PM983 8Tb drives - tons of these on ebay new and used got from 550 down to 250 in past 2 years.
For chassis - sas SYS-1028U or if lucky SYS-1029U, upgrade to NVMe backplane, some cabling, risers or even AOCs - 10xNVMe X10/X11ultra server is ready to run.
2U SYS-2029U or SYS-2028U are damn rare and still expensive, but I hope there's a way to build almost the same on regular X11DPU in 2U SAS SYS-2029U SYS-2029UZ-TN20R25M | 2U | SuperServers | Products | Super Micro Computer, Inc.
just need that passive NVMe BPN-NVMe3-216N-S4 and some proper risers and AOCs
Or rather the Retimer Supermicro AOC-SLG3-4E4T ab € 147,62 (2024) | Preisvergleich Geizhals EU
Electronics will use power sure, if that's the metric then try passive. But U.2 enterprise drives are power hungry so maybe say goodbye to that 25W in anycase.
Ceacent works as good if not better than any random passive card. Personally I would trust Ceacent more than random Amazon sellers which are mostly/all chinese noname imports too anyway.Hah, that Ceacent PCIe x16 -> 4x SFF-8643 board really is €230 cheaper!
About cables, there's at least four different connectors in use:
I understand the last one, MCIO, has a chance of supporting PCIe Gen 5 speeds. What about the others, are they approximately the same or are some better than the others?
- SFF-8643 "MiniSAS"
- SFF-8654 "SlimSAS" (4i and 8i variants)
- SFF-8611 "OCuLink" (4i and 8i variants)
- SFF-TA-1016 "MCIO" (8i - is there a 4i variant as well?)
For cables, depends what you are trying to do. Direct connection would be for example https://www.aliexpress.com/item/32975750347.html
The connector type does not matter very much IMHO. Use what is available / cheap. I wouldn't worry about Gen5 speeds if you're planning to use U.2 drives. Anything above Gen3 combined with passive connection and you're in for a pain. Even for Gen3 they do not make redriver/retimer cards just for fun, if the signal is not good, there will be problems.
For PCIe4 there's c-Payne's Gen4 collection SlimSAS gen4 - PCIe Host Adapters
I've personally used their redriver and it gives you 4 NVMe drives at x4 from a PCIe x16 slot, with 0.75m cables. Heck I've even used it on an ASUS Q87 motherboard and it actually works
I also picked up a refurb Broadcom switch - the P411W-32P - cheaply at ~ $400. That's one of the few things that can give you surprise removal and hot plug (only tested with a Supermicro X11 DPH-T). NVMe hot-plug is a hot mess and the switch seems better than others at it. If you use it, be sure to read the long forum post over at Level1Techs about it
Also tried to use HPE's 826689-B21 8x SFF drive cage - seemed like a good idea because these things go for $100 at times, have 4x SlimSAS 8i connectors - what's not to like. Except I've never gotten it to work despite powering it correctly and connecting to a variety of AICs, PCIe switches and redrivers. I don't know what magic crap HPE is doing with it's NVMe riser, but that cage doesnt work with normal SlimSAS 8i connections
As for ICYDock - there's some concept products they are harping on about like the CP104 which is a 8x U.2 cage in a 2 x 5.25" bay and uses MCIO (i.e claims to be ready for PCIe5 but no U.3 support) these if they become reality will retail for upwards of $500. ICY Dock also has other useless 9.5mm height drive cages (a lot of NVMe drives are now 15mm because of heatsink fins) and the average price is usually $400 ~ $500 for 4x bays of U.2 NVMe - that would mean $900 for just 8x bays
All in all, it seems buying a refurb Dell or Supermicro server with 16x or 24x NVMe bays is actually a better deal
I've personally used their redriver and it gives you 4 NVMe drives at x4 from a PCIe x16 slot, with 0.75m cables. Heck I've even used it on an ASUS Q87 motherboard and it actually works
I also picked up a refurb Broadcom switch - the P411W-32P - cheaply at ~ $400. That's one of the few things that can give you surprise removal and hot plug (only tested with a Supermicro X11 DPH-T). NVMe hot-plug is a hot mess and the switch seems better than others at it. If you use it, be sure to read the long forum post over at Level1Techs about it
Also tried to use HPE's 826689-B21 8x SFF drive cage - seemed like a good idea because these things go for $100 at times, have 4x SlimSAS 8i connectors - what's not to like. Except I've never gotten it to work despite powering it correctly and connecting to a variety of AICs, PCIe switches and redrivers. I don't know what magic crap HPE is doing with it's NVMe riser, but that cage doesnt work with normal SlimSAS 8i connections
As for ICYDock - there's some concept products they are harping on about like the CP104 which is a 8x U.2 cage in a 2 x 5.25" bay and uses MCIO (i.e claims to be ready for PCIe5 but no U.3 support) these if they become reality will retail for upwards of $500. ICY Dock also has other useless 9.5mm height drive cages (a lot of NVMe drives are now 15mm because of heatsink fins) and the average price is usually $400 ~ $500 for 4x bays of U.2 NVMe - that would mean $900 for just 8x bays
All in all, it seems buying a refurb Dell or Supermicro server with 16x or 24x NVMe bays is actually a better deal
Hot plugging anything that's directly connected to CPU is a bad idea yeah.. but 400€ seems a lot for HBA, but I digress It's not like I haven't paid stupid money for gear.
Like these IcyDock cages (didn't know about HP etc at the time)
geizhals.eu
It's not like I haven't paid stupid money for gear.Like these IcyDock cages (didn't know about HP etc at the time)
Icy Dock ToughArmor MB699VP-B V3 ab € 324,24 (2024) | Preisvergleich Geizhals EU
✔ Preisvergleich für Icy Dock ToughArmor MB699VP-B V3 ✔ Produktinfo ⇒ Typ: Backplane • Anschlüsse intern: 4x 2.5" U.3/U.2/SFF-8639 (PCIe 4.0 x4) • Anschlüsse extern: 4x OCuLin… ✔ Wechselrahmen ✔ Testberichte ✔ Günstig kaufen
geizhals.eu
for 4 disks, thats a bad deal. You'd be better off putting velcro somewhere in your case and sticking those disks in.
(else you can look at aliexpress - bit better deals there - but crappier quality)
(else you can look at aliexpress - bit better deals there - but crappier quality)
It would've been interesting to try their passive SlimSAS adapters together with short (~30 cm?) cables. Too bad all the straight versions are sold out, and that I have no idea where to get short SlimSAS 8i => 2x SFF-8639 cables.For PCIe4 there's c-Payne's Gen4 collection SlimSAS gen4 - PCIe Host Adapters
I've personally used their redriver and it gives you 4 NVMe drives at x4 from a PCIe x16 slot, with 0.75m cables. Heck I've even used it on an ASUS Q87 motherboard and it actually works
IcyDock tech & customer support is ... pathetic. They told me they have a version of firmware for the P411W-32P that is customized for IcyDock and not available generally. And that's the only way to use the MB699VP-B V3 with the NVMe switch.
Got the firmware from them, flashed it ... and ended up with a bricked NVMe switch. Keep in mind, I've flashed a couple of firmwares on this adapter before, so it's not like I don't know how etc.
Adapter is on RMA back to Broadcom and hopefully it gets reflashed properly but ...
- IcyDock is handing out questionable firmware
- They did a vanishing act after I told them the adapter got bricked
... means I personally will NOT be buying IcyDock again, nor will I recommend them to others
Wow. Broadcom is not famous for stability on it's own and to add IcyDock firmware to it. I can't quickly think of a situation where one company offers firmware for another's product like this. Hope it gets fixed.
IcyDock support told me they have no "active ICs" between the MB699VP-B V3 cage and PCIe and everything that can work directly should work thru the cage. Clearly this is not the case.
IcyDock support told me they have no "active ICs" between the MB699VP-B V3 cage and PCIe and everything that can work directly should work thru the cage. Clearly this is not the case.