NVMe - make the most of your PCI-e Slots (How-to config Supermicro boards for AOC-SLG3-2E4T et al)

[J-san]

If somebody has an AOC-SLG3-2E4R and a compatible board it would be interesting to see whether it is behaving identically to the 2E4T since it should share the same basic functionality (requiring bifurcation support) only with a different connector.

Confirmed that the AOC-SLG3-2E4R does NOT work on a X10DRi-T board - don't ask me how I know

Even with all the oprom, bifurcation tweaks it was not recognized.

Eg. No dice:

 

[Rand__]

Sorry to hear that and thanks for the feedback.

 

[T_Minus]

Confirmed that the AOC-SLG3-2E4R does NOT work on a X10DRi-T board - don't ask me how I know

Even with all the oprom, bifurcation tweaks it was not recognized.

Eg. No dice:

View attachment 10847

I don't see bifurcation settings on your screenshot ?

 

[sth]

had trouble on my x10sdv too with dual NVMe drives.

 

[J-san]

I don't see bifurcation settings on your screenshot ?

Hi T_Minus,

I played around with various settings in PCI SLOT 1 (x8), SLOT 6 (x16), and Slot 4 (x16)..

Testing with Slot 1 (x8) set to 4x4 didn't work (setting in screenshot just on Auto here):



And likewise the Cpu2 - Port 2 - Slot 6 (x16) didn't work either set to 4x4x4x4 (showing set to auto again here):




The screenshot in the previous post (also shown below) shows the detail of the split port after bifurcation was enabled on that last try. (cpu2 - port 3D = slot 4; 4th bifurcated port D)




The HBA was in that slot with NVMe plugged into 1st port of the HBA card.. I didn't catch the importance of the *limited* support for this "R" version of the card (AOC-SLG3-2E4R) before I ordered it:

https://www.servethehome.com/4-solutions-tested-add-2-5-sff-nvme-current-system/

After a few days of testing, we found a the card that costs $100 more, the Supermicro AOC-SLG3-2E4 had a PLX chip and much wider card compatibility.

I do have a AOC-SLG3-2E4 (non R) HBA card on order, so I hope to get things working with that card instead.

If anyone in the Vancouver BC area has a use for the "R" version of the card send me a PM and I can provide it for a good price

 

[J-san]

I received the AOC-SLG3-2E4 (non R) HBA card and now it works with my X10-DRi-T motherboard.

I did have some strangeness that I noticed with the system hanging on with boot code (92?) when trying to enter the BIOS only AFTER changing the Slot's OPROM to EFI.



It refused to enter the BIOS with OPROM -> EFI and Bifircation enabled, but strangely enough would boot into ESXi normally if I didn't try to enter the BIOS.

I was able to get back into the BIOS by disconnecting the Optane drive cable from the AOC-SLG3-2E4 HBA and then it would allow me to get back into the BIOS.

I've been testing with the slot's OPROM set to Legacy (not EFI) and it seems to be working fine.

 

[J-san]

It's not real without screenshots

SSDPE21K375GA01 [375GB Optane DC P4800X Series 2.5" U.2 P]

 

[GIAN]

I'd like to report that the Supermicro AOC-SLG3-4E4R card fitted in slot 6 of a Supermicro X11SPi-TF motherboard doesn't work in my setup.

Intel DC P4510 (1TB) NVMe drives connected via the BPN-SAS3-826EL1-N4 backplane of my Supermicro SC829HE1C4-R1K02LPB chassis to ports NVME0, NVME1 and NVME2 of the Supermicro AOC-SLG3-4E4R card with Supermicro CBL-SAST-0590 cables are recognised correctly and work fine.

From the moment I connect any of my 4 Intel DC P4510 (1TB) drives to port NVME3 of the Supermicro AOC-SLG3-4E4R card, the server remains stuck during the boot process at the screen DXE-IPMI Initialization.

Together with Supermicro Europe support, we tried everything but concluded that this setup is probably not compatible. Supermicro now suggests to try the (apparently more compatible) Supermicro AOC-SLG3-4E4T card.

Does anyone have experience with the Supermicro AOC-SLG3-4E4T card and X11 motherboards?

 

[Rand__]

I used them on X10 without issues, but have not run them in X11, sorry

 

[GIAN]

I used them on X10 without issues, but have not run them in X11, sorry

Thanks for the info, makes me optimistic for my X11 board. It does have x4x4x4x4 bifurcation. I ordered the card and will report back (after my holidays )

 

[GIAN]

Thanks for the info, makes me optimistic for my X11 board. It does have x4x4x4x4 bifurcation. I ordered the card and will report back (after my holidays )

The Supermicro AOC-SLG3-4E4T card works fine with my Supermicro X11SPi-TF motherboard . Problem solved!

 

[Rand__]

Glad to hear that

 

[lbjm]

Has any tried the the supermicro nvme cards in non supermicro motherboards? Namely the AOC-SLG3-2E4T or the Supermicro AOC-SLG3-2E4. I want to add Nvme ports to a gigabtye c621 board.

 

[csp-guy]

AOC-SLG3-4E4T sounds really interesting.

Is it possible to using it as mixed controller, for example using 2 ports as sata by this cable: Supermicro CBL-SAST-0933 50cm OCuLink SFF-8611 (x4) to 4 SATA Cable

And the other 2 ports as direct connect to U.2 drive: Supermicro CBL-SAST-1011 75cm OCuLink SFF-8611 (x4) to PCIE SFF-8639 U.2 with Power Cable

My Motherboard supports pci-e bifurication.

 

[Rand__]

Has any tried the the supermicro nvme cards in non supermicro motherboards?

I ran a SLG-2E2T in an ASUS board with no issues (after hacking the bios to enable bifurcation on the slot I needed).

Otoh, same card failed on an SM x11ssl-f despite it supporting bifurcation, so you won't know untill you try or find someone with same board/bios version

 

[zir_blazer]

AOC-SLG3-4E4T sounds really interesting.

Is it possible to using it as mixed controller, for example using 2 ports as sata by this cable: Supermicro CBL-SAST-0933 50cm OCuLink SFF-8611 (x4) to 4 SATA Cable

And the other 2 ports as direct connect to U.2 drive: Supermicro CBL-SAST-1011 75cm OCuLink SFF-8611 (x4) to PCIE SFF-8639 U.2 with Power Cable

My Motherboard supports pci-e bifurication.

No, cause you need a OCuLink Port that comes from a configurable controller that can selectively output either PCIe and SATA on the same pins, which that card doesn't has, as it seems to be based around a standard PCIe Switch (It has a heatsink covering it).
Also, if your Motherboard supports PCIe Bifurcation, then you may be able to find a cheaper, simpler card that is just an adapter. Cards with PCIe Switches are those that also work on Motherboard without PCIe Bifurcation support.

 

[hydra1981]

Hi, I installed last night the AOC-SLG3-2E4 card but I can't seem to be able to make VMWare ESXi 7 see it. The board is the X10-SRi-F and the card is currently installed in Slot 3 (if we assume the 16x slot is slot 1). It is an 8x slot physically and electrically. I put the slot in EFI mode and started ESXi in UEFI but that didn't make any difference. The 2 SAS connections are wired to an Icy Dock NVMe 3.5 inch mobile rack. Only one NVMe card is installed, a Samsung EVO. Any hints? I assumed that the 2E4 doesn't need any special bifurcation treatment, am I wrong? Thank you

 

[sth]

I spent quite a bit of time trouble shooting this (and other) supermicro boards in X10 & X11 motherboards. Ultimately Supermicro service concluded as per their product page, "This product does not work with 3rd party servers. It can only work with validated Supermicro server systems and Supermicro motherboards. Please refer to the validated platforms list.

The supported devices only includes, X10DRU-i+, X10DRW-i, X10DRW-iT, X11SPW-TF, X11SSZ-F, X11SSZ-QF, X11SSZ-TLN4F

The info is on their product page at AOC-SLG3-2E4 | Add-on Cards | Accessories | Products - Super Micro Computer, Inc.

ESXi 7 likely throws even more complication into the matter so maybe worth checking with another OS.

 

[hydra1981]

That would be a major bummer if the card didn't work however there is this guy who seems to have done it without issues

I've asked him too of course and I am waiting to hear back

 

[sth]

Do you know what motherboard he was using in that video? On a supported board the card was as simple plug and play as you would expect. I could in some configurations, some of the time, get one port to recognize the attached disk.It seemed linked to a warm reboot, not cold. I never got both to appear and it was certainly never reliable.

EDIT: I see its a x10sdv now, yeah, sorry, I couldn't make it work. I have a number of these boards on a shelf so if you find a solve I'm a keen to hear about it.