Supermicro X9DR3-LN4F+ and/or X9SRL-F with an Optane P4801 100GB M.2

[skorpioskorpio]

Seems like the place to ask this, so I am looking to use an Optane P4801 100 GB M.2 on a Supermicro X9DR3-LN4F+ and also on another box with an X9SRL-F, in both cases I would be using a PCIe (V3) x4 slot with just a run of the mill PCI to single stick M.2 NVMe adapter, specifically an ECM26 from Silverstone, which is low profile, specifically X4 and will hold the 110mm length of the P4801. Probably unsurprisingly to this crowd, it's for a ZIL drive on a ZFS NAS/SAN/Container Server, although perhaps less obviously I'm planning on IllumOS, specifically Omni-Tribblix, which is Tribblix with the OmniOSs LX Zones moved over to it, but I don't see how that really matters much, so long as the Optane just "looks" like an NVMe drive which seems to be the case from what I've read.

My understanding is that NVMe on a board of this age and well specifically the X9 boards, can have issues, but those issues tend to revolve around, trying to boot from the NVMe, which I don't care about, multi-slot cards that require bifurcation, again not what I am doing, or PCIe versions which don't agree, I plan specifically to use the X4 V3 slot so everything is matchy matchy, and honestly anything I can think of to do PCIe-wise in these boxes requires an X8 slot like HBAs or 10G NICS. Both boards have x4 slots, the DR3 has 1 and the SRL has 2, well 3 but one is a V2 slot. Also thinking I may do 2 of those Silverstone adapters in the SRL, one with an Optane and one with a Samsung 950-970 or maybe a FireCuda, not sure yet, as an L2ARC, but I'd think the Optane is probably more of a wildcard than NAND which is what it is, so my assumption would be if the P4801 works, the Samsung (or whatever NAND it ends up being) should work.

Does this all sound accurate? I already bought one of the Silverstone adapters, well because they're cheap, but the other stuff isn't quite so cheap, it's not expensive in the grand scheme of things, but it's not bag-o-fast-food cheap, so I figured I'd ask before spending on filler for the shelf of abandoned projects.

 

[Spearfoot]

Seems like the place to ask this, so I am looking to use an Optane P4801 100 GB M.2 on a Supermicro X9DR3-LN4F+ and also on another box with an X9SRL-F, in both cases I would be using a PCIe (V3) x4 slot with just a run of the mill PCI to single stick M.2 NVMe adapter, specifically an ECM26 from Silverstone, which is low profile, specifically X4 and will hold the 110mm length of the P4801. Probably unsurprisingly to this crowd, it's for a ZIL drive on a ZFS NAS/SAN/Container Server, although perhaps less obviously I'm planning on IllumOS, specifically Omni-Tribblix, which is Tribblix with the OmniOSs LX Zones moved over to it, but I don't see how that really matters much, so long as the Optane just "looks" like an NVMe drive which seems to be the case from what I've read.

My understanding is that NVMe on a board of this age and well specifically the X9 boards, can have issues, but those issues tend to revolve around, trying to boot from the NVMe, which I don't care about, multi-slot cards that require bifurcation, again not what I am doing, or PCIe versions which don't agree, I plan specifically to use the X4 V3 slot so everything is matchy matchy, and honestly anything I can think of to do PCIe-wise in these boxes requires an X8 slot like HBAs or 10G NICS. Both boards have x4 slots, the DR3 has 1 and the SRL has 2, well 3 but one is a V2 slot. Also thinking I may do 2 of those Silverstone adapters in the SRL, one with an Optane and one with a Samsung 950-970 or maybe a FireCuda, not sure yet, as an L2ARC, but I'd think the Optane is probably more of a wildcard than NAND which is what it is, so my assumption would be if the P4801 works, the Samsung (or whatever NAND it ends up being) should work.

Does this all sound accurate? I already bought one of the Silverstone adapters, well because they're cheap, but the other stuff isn't quite so cheap, it's not expensive in the grand scheme of things, but it's not bag-o-fast-food cheap, so I figured I'd ask before spending on filler for the shelf of abandoned projects.

I've used NVMe single-stick M.2 adapters on an X9SRL-F and an X9DRi-LN4F+ (very similar to your X9DRS-LN4F+) with success. My hunch is that the Optane will work, too, but I haven't tried one of those myself.

 

[itronin]

@skorpioskorpio as you noted so long as you don't need boot support then no incantations (modded bios) required.

used cheap single stick m.2 adapters (startech) in X9SRL-F & X9DRD worked a charm with optane 800p, PM963, samsung consumer gumsticks.
used dual SM adapters on X9DRD with bifurcation same nvme as above.
Optane 900P AIC worked fine in X9SRL-F & X9DRD
Optane 900P u.2 worked in X9SRL-F & X9DRD with cheap bifurcation adapters (single drive in X9SRL no bifur support), dual drives in X9DRD.
Usage primarily in FN/TNC used as both storage but mostly SLOG with Optane though tested against Linux as well.

I can't imagine that P480xX would be any different, @T_Minus has posted comments on the P4801X as SLOG just not sure if it is on this generation of boards.

 

[skorpioskorpio]

I've used NVMe single-stick M.2 adapters on an X9SRL-F and an X9DRi-LN4F+ (very similar to your X9DRS-LN4F+) with success. My hunch is that the Optane will work, too, but I haven't tried one of those myself.

Yea, I haven't read anything to suggest otherwise. It's always good to ask though right? I think the DRi and DR3 might even use the same BIOS. Looks like the big difference between the 2 is that I have twice as many useless SAS/SATA ports (well in IllumOS anyway, the SCU ports are not supported on any of these boards that have them, which is a lot of the X9 boards).

@skorpioskorpio as you noted so long as you don't need boot support then no incantations (modded bios) required.

used cheap single stick m.2 adapters (startech) in X9SRL-F & X9DRD worked a charm with optane 800p, PM963, samsung consumer gumsticks.
used dual SM adapters on X9DRD with bifurcation same nvme as above.
Optane 900P AIC worked fine in X9SRL-F & X9DRD
Optane 900P u.2 worked in X9SRL-F & X9DRD with cheap bifurcation adapters (single drive in X9SRL no bifur support), dual drives in X9DRD.
Usage primarily in FN/TNC used as both storage but mostly SLOG with Optane though tested against Linux as well.

I can't imagine that P480xX would be any different, @T_Minus has posted comments on the P4801X as SLOG just not sure if it is on this generation of boards.

Both of the boards are using SATA SSD mirrors for boot, and honestly I'm just using the onboard SATA2 ports for that, I'm not sure what I'd gain with anything faster for an OS drive. I mean the booting on these boards (either) is comically slow, as it is, if the boot time of this box was represented as the timeline of life on earth, the total IllumOS share doesn't start until the dawn of man.

I know I'd read somewhere about bifurcated cards specifically on the Supermicro X9 Series, but as I remember it had some caveats, my boards, the X9D...s anyway are V1.0 boards and my guess would be so are the X9S...s meaning they won't support E5-2600V2 CPUs. But that also means that everything about these boards is just stupidly cheap right now. I got both boards, plus a spare of each, and 8 core 16 thread CPUs for all the sockets (including the spares) for under $500. Oh and they're both also maxed out on RAM using 16GB DIMMS, which was also cheap.

Yea, I looked at TrueNAS, a little too closed for my taste and seemed to ignore actual filesystems altogether, focusing really on pools. I also went down the ZoL path as well, but, out of the box anyway, the performance was pretty inconsistent. My old box was Solaris and even though it was really old hardware with a fraction of the specs of the new box, and barely holding it together at that, it mopped the floor with the ZoL box. I mean that box wasn't even using PCIe for the HBAs. I was seriously disappointed in ZoL, and FYI I was just using ProxMox as the test for that. and yea I'm certain it can be tuned better, way better I'm sure, but IllumOS, out of the box, significantly outperformed ZoL and marginally outperformed BSD in my limited tests. Additionally IllumOS (and Solaris) transferred network files more consistently than either of the others.

...and Oracle dropped support for both the HBAs in my old box (Marvell) and couldn't see all the HBAs in the new box (LSI) so Solar-ish seemed like the logical path. So far happy with the decision, Tribblix was not the first choice, Tried SmartOS which is strange, OmniOS which is a little too stripped down for my taste, and OpenIndiana which, well I was never able to get to install at all. With LX zones added into Tribblix it seemed to answer most of the needs, a type one hypervisor would have been a nice addition (which does exist in SmartOS and OmniOS in the form of a Bhyve port) but the more I looked into it that port it seemed pretty crippled, I mean the first couple VMs I'd planned to spin up needed USB pass-through which it can't do. One was Windows, and a couple were to be Linux, which I can probably do in Zones, we'll see. Given that a native hypervisor was never really part of x86 Solaris, I can't say I'm surprised it's meh.

I'll start mining for threads on the P4801X, which I guess is the key, I'd searched for P4801 and didn't get back much. I guess I missed the magic trailing "X". This is my intro to NVMe exercise (and this forum), I have m.2 SATA drives all over the place, a couple dozen of them across about half that in machines but no NVMe yet, and certainly no Optane. One thing I am looking forward to is eliminating all the eMMC flash and SD Cards on the plethora of ARM and ATOM SBCs I have running, many of which have puked. I never configured the old box for SAN so that's something else new in this project, and since Oracle essentially forced me into upgrading the old box just to get my data off (hence the X9SRL-F) I'll have a backup server and a dev environment this time around.