Intel 900P 280GB U.2 drives $229

JediAcolyte

Active Member
May 29, 2020
131
41
28
US
Au contraire! ... Did you expect to find postings like "I've got this mobo and the bios has this option and it works."? Like Jell-o, it's hard to screw up bifurcation. It's dead simple.

I believe you've "thrown the baby (ie, bifurcation) out with the bathwater (the AOC-SLG3-2E4R)". Something on that redriver card is not playing nice with the X9DRI-F.

Consider canceling the 2E4 (@$150-200?) and getting instead either:
1. PCIe x8 to 2x SFF-8643 adapter (w/1 cable) [Link] for $45.
or 2. PCIe x16 to 4x SFF-8643 adapter [Link] for $35.

I understand, and appreciate, your hesitancy to buy from unknown/distant sellers. I am quite risk-averse myself. But, after seeing that others (on STH) had purchased from each of the above sellers, I also did, with no regrets. (In both cases, delivery was <17 days after order.)
=====
I'm not cheap -- I'm capital-efficient.
Just to clarify, and ensure I understand correctly, this will work when your motherboard supports PCIe bifurcation?
 

mattlach

Active Member
Aug 1, 2014
183
26
28
Au contraire! ... Did you expect to find postings like "I've got this mobo and the bios has this option and it works."? Like Jell-o, it's hard to screw up bifurcation. It's dead simple.

I believe you've "thrown the baby (ie, bifurcation) out with the bathwater (the AOC-SLG3-2E4R)". Something on that redriver card is not playing nice with the X9DRI-F.

Consider canceling the 2E4 (@$150-200?) and getting instead either:
1. PCIe x8 to 2x SFF-8643 adapter (w/1 cable) [Link] for $45.
or 2. PCIe x16 to 4x SFF-8643 adapter [Link] for $35.

I understand, and appreciate, your hesitancy to buy from unknown/distant sellers. I am quite risk-averse myself. But, after seeing that others (on STH) had purchased from each of the above sellers, I also did, with no regrets. (In both cases, delivery was <17 days after order.)
=====
I'm not cheap -- I'm capital-efficient.
I appreciate the recommendation. The thing has already shipped though, so it may be too late for me.
 

UhClem

Active Member
Jun 26, 2012
122
45
28
NH, USA
I appreciate the recommendation. The thing has already shipped though, so it may be too late for me.
Some sellers, you can contact them before receiving, offer to cover some of shipping, and get their OK to refuse the delivery. lv note for carrier and sign it ...
 

mattlach

Active Member
Aug 1, 2014
183
26
28
I have to admit, upgrading the old pool always makes me a little giddy:

1626734568284.png

I'm a little bit surprised how long the replace resilver takes for log drives though, since tey essentially only ever store a second worth of write data, I expected this to be quick! I guess it has to read through the entire pool just to be sure, for some reason.
 

mattlach

Active Member
Aug 1, 2014
183
26
28
Out of curiosity, how was the wear on these drives?
The ones I got were new old stock, sealed in retail boxes, so no wear at all. When I read smart values on first startup they each had 1 powerup, 0 power on hours and 0 writes.

I can't speak to the ones others have received though.
 

Gohst_oc

New Member
Jul 26, 2017
2
1
3
30
I appreciate the recommendation. The thing has already shipped though, so it may be too late for me.
I'm using the same motherboard without any problems running bifurcation on a AOC-SLG3-4E4T-O.
All 4 NVME drives are working.

-> It's known that only proper redriver or simple passive solutions are working. So in fact any cheap 4x m.2 to PCIE 16x card or the mentioned Supermicro one (there's a 2 port version too).

I would not use plx chip cards, they add power consumption (~10W) and some latency which is bad when using an optane ssd.

You need bios 3.4 (the latest) since in any other bios bifurcation settings are broken (even if one can set them in bios).
Then any port seems to bifurcate nicely (even the PCIE 8x slots).

Edit: the S2I cable is "nonsense" it only works on Supermicro X10 boards. It should enable communication to IPMI to monitor proper card/disk health.
 
  • Like
Reactions: T_Minus

UhClem

Active Member
Jun 26, 2012
122
45
28
NH, USA
I'm using the same motherboard without any problems running bifurcation on a AOC-SLG3-4E4T-O.
All 4 NVME drives are working.
...
I would not use plx chip cards, they add power consumption (~10W) and some latency which is bad when using an optane ssd.
I would not use retimer chip cards, they add power consumption (~13W) and some latency ... ;)

Unless those redriver/retimer cards add real functionality (like hot-swap/plug), you are much better off with the "simple" (& cheaper) cards.

If you do choose to go with a PCIe_switch-based card, definitely opt for a configuration that over-subscribes the PCIe slot (i.e., #downstream [target] lanes > #upstream [host] lanes). For little added cost, you get the benefit of more effective use of bandwidth, plus the extra connectivity. Also, unless someone can demonstrate (competently) that there is measurable (and not insignificant) added latency, ignore their babble.:)
 

mattlach

Active Member
Aug 1, 2014
183
26
28
I would not use plx chip cards, they add power consumption (~10W) and some latency which is bad when using an optane ssd.
I would not use retimer chip cards, they add power consumption (~13W) and some latency ... ;)
I agree with both of you, and going without a PLX chip was the original plan until I determined what I thought was the bifurcation not working on my board. (It turns out the Supermicro AOC-SLG3-2E4R needs motherboard firmware support via i2c in order to work, I thoguht they were just dumb bifrucation adapters, so when it didn't work, I assumed it was bifurcation that was broken)

All of that said, going without a PLX chip is certainly better than using one both for power consumption and latency purposes, but I doubt the latency issue is particularly significant. The PLX specifications all suggest "less than 150ns of latency" which is not a lot.

Even with the added latency the Optane drives are MUCH better SLOG devices than the S3700 SATA SSD's I was using before them :p

I have ordered another dumb bifurcation splitter adapter to replace the PLX version I have, but as an interim solution, it really isn't bad.
 
Last edited:

Gohst_oc

New Member
Jul 26, 2017
2
1
3
30
PCIe retimers seem to have (way) lower latency than plx chips (PCI SIG specces them at less than 64ns).

And they do make PCIE signals more reliable (which is their purpose if I did understand the source below).

.

So in an enterprise solution, they fit well.
PCI SIG means they will have a bright future.

But hinestly, best is a passive "splitter" and short cables (or directly m.2 modules on a PCB) when it comes to performance/latency.

Then one shall consider maybe a retimer over a plx, if enough lanes (and bifurcation) are available.
 

UhClem

Active Member
Jun 26, 2012
122
45
28
NH, USA
Look, you wasted your money on that retimer card. And, in an attempt to deny that, you wasted your time reading that PCISIG stuff--(1)you don't really understand it (2)you (and I) are NOT its intended audience. That stuff is for Super-Master Plumbers. You can not even imagine (let alone comprehend) that level of PCIe wrangling.

I won't waste any more time discussing this.
 

mattlach

Active Member
Aug 1, 2014
183
26
28
I'm using the same motherboard without any problems running bifurcation on a AOC-SLG3-4E4T-O.
All 4 NVME drives are working.

-> It's known that only proper redriver or simple passive solutions are working. So in fact any cheap 4x m.2 to PCIE 16x card or the mentioned Supermicro one (there's a 2 port version too).

I would not use plx chip cards, they add power consumption (~10W) and some latency which is bad when using an optane ssd.

You need bios 3.4 (the latest) since in any other bios bifurcation settings are broken (even if one can set them in bios).
Then any port seems to bifurcate nicely (even the PCIE 8x slots).

Edit: the S2I cable is "nonsense" it only works on Supermicro X10 boards. It should enable communication to IPMI to monitor proper card/disk health.
I agree with both of you, and going without a PLX chip was the original plan until I determined what I thought was the bifurcation not working on my board. (It turns out the Supermicro AOC-SLG3-2E4R needs motherboard firmware support via i2c in order to work, I thoguht they were just dumb bifrucation adapters, so when it didn't work, I assumed it was bifurcation that was broken)

All of that said, going without a PLX chip is certainly better than using one both for power consumption and latency purposes, but I doubt the latency issue is particularly significant. The PLX specifications all suggest "less than 150ns of latency" which is not a lot.

Even with the added latency the Optane drives are MUCH better SLOG devices than the S3700 SATA SSD's I was using before them :p

I have ordered another dumb bifurcation splitter adapter to replace the PLX version I have, but as an interim solution, it really isn't bad.

Alright, I ordered one of those Chinese boards. It made me a little nervous. I usually tend to avoid "designed in china" electronic products because of their well documented history of espionage, but it doesn't appear to have enough components on it to truly be a cause for concern. :p

I almost reported back here that it still didn't work. I spent a good deal of time troubleshooting, and I could only get one of the two drives to show using the Chinese non-PLX splitter. On a whim I tried a setting that shouldn't have worked, but it did.

It turns out, at least on the latest BIOS revision on the X9DRI-F they have reversed the slot numbers of the frist two 8x slots in the BIOS where you configure bifurcation.


The layout (from left to right) is as follows:
CPU 1 Slot 1: 8x
CPU 1 slot 2: 16x
CPU 1 Slot 3: 8x
CPU 2 Slot 4: 16x
CPU 2 Slot 5: 8x
CPU 2 Slot 6: 16x

For whatever reason in the bifurcation configuration screen, the two 8x slots on CPU 1 (Slots 1 and 3) are configured together as one 16x slot that cannot be configured as 16x (because they are two separate slots) so your options are 8x8x, 4x4x8x, 8x4x4x or 4x4x4x4x.

At first I inserted the card in Slot 1, and configured it so the BIOS tells you there are two 4x on Slot 1, but I only had the first U.2 drive show up.

I spent way too long troubleshooting without success. Before giving up, I on a whim reversed the setting, so that it said Slot 1 was 8x, and Slot 3 was 4x4x, and to my surprise, when I did this, it worked.

It looks like in this BIOS screen, and only in this BIOS screen they messed up and reversed Slot 1 and Slot 3.

Now that I've figured that out, it seems to work perfectly. Now I'll either sell the Supermicro PLX card, or keep it if I ever want to use it in a desktop that lacks bifurcation support.
 

mattlach

Active Member
Aug 1, 2014
183
26
28
Personally, I will prefer the adapter cards with PLX controllers without requiring bifurcation like this post.
It's a mixed bag.

They are more likely to work regardless of the motherboard, but on the flipside they introduce more latency and they use a surprising amount of power. The Supermicro PLX card I replaced got pretty damn hot!

If you can make a non-PLX card work, it is the better choice. If you can't, that's when you have to use PLX.
 
  • Like
Reactions: NateS and newabc