ESXi crash with AOC-SLG3-2E4 and Intel DC P3600 Passthrough

[K D]

So SuperMicro AOC-SLG3-2E4 (NVME AIC) does not work, just making sure I'm following the parts from original post to now?

Have you tried a Funtin adapter instead to see?

AOC-SLG3-2E4 does not work with X11SPL-F board. Tried connecting the SSDs directly with a cable as well as using a backplane. The drives are detected but seem to be in a read only mode. I'm able to read partitions, copy files from the drive to another drive but unable to write to the drive or modify the drive.

When I connected the AOC-SLG3-2E4 to X10DRH-iT, all these issues went away. They behaved normally. I was able to use partedUtil and clear a partition, use esxcli to add the drives to a vsan.

AOC-SLG3-2E4T is a new card with 2 oculink connectors. Found it by painfully browsing through each X11 SuperServer SKU's parts. Supermicro labels this as compatible with all X11 motherboards. I have one coming in next week. Will test that and reply back with the results.

And BTW., the X10DRH-iT is the one I got from you

 

[Rand__]

I hope you have better luck with your AOC-SLG3-2E4T than I.
It is indeed officially "compatible" with all x11 boards but requires bifurcation support on the mainboard side which is sparsely documented.
I am trying to run it on (one of several) E3 X11 boards, no luck yet (only one drive working in win, none in esx). Engaged SM support and they are currently looking to reproduce.
Have not tried on a E5 based X11 board (have none).

 

[K D]

I hope you have better luck with your AOC-SLG3-2E4T than I.
It is indeed officially "compatible" with all x11 boards but requires bifurcation support on the mainboard side which is sparsely documented.
I am trying to run it on (one of several) E3 X11 boards, no luck yet (only one drive working in win, none in esx). Engaged SM support and they are currently looking to reproduce.
Have not tried on a E5 based X11 board (have none).

 

[Rand__]

Lol

 

[K D]

Just picked up the new card and cables from FedEx. Will install it and test tomorrow. Hope this works out.

 

[K D]

Installed the AOC-SLG3-2E4T card and connected to the backplane. Installed a 750 and a P3600. Booted off a Windows recovery disk and was able to edit both drive partitions. Previously was unable to do this. Cleaned out all the drives in the system.


Will do a fresh install of esxi and try to install VSAN.

Fingers crossed....

 

[K D]

SUCCESS !!!!

 

[Rand__]

Nice
SM support is still working on getting mine to run on X11/E3 boards.
Did you configure bifurcation settings?
If you did not then probably the X11/e5 boards support Auto-bifurcation (in certain slots) - that's what SM indicated but it didnt work on the X11/E3 board

 

[K D]

Yes. The board supports auto bifurcation in 4 slots per the manual. Which X11/E3 board ate you using?

 

[nitrobass24]

Yes. The board supports auto bifurcation in 4 slots per the manual. Which X11/E3 board ate you using?

Remind me what board are you using?


Sent from my iPhone using Tapatalk

 

[K D]

X11SPL-F

 

[Rand__]

Yes. The board supports auto bifurcation in 4 slots per the manual. Which X11/E3 board ate you using?

X11SSZ-TLN4f, X11SSH-CLN4F or X11SSX-F are my test candidates. SM is working on the first one, but only got it running on a X11SSH-F as of now.
New Bios, explicit bifuration...

My hunch is that those cards have been tested with the X11/E5 boards where they will work pretty much out of the box.
Then they said X11 is compatible failing to realize that there is an older X11 generation (e3) out there which does not have (manual or auto) bifurcation bios support per default.
If we are lucky they fix this on X11/E3 (at least on request), else they'll just reduce compatibility statement.
But I think since they got it running we have a fair chance to get it.

That would be great since even the 4 port card is pretty cheap and SM OcuLink cables are way cheaper than NVMe cables in general.

 

[K D]

I have the X11SSH-CTF. But that's the main home server ". If I can find a window to reboot it, I will check if the bios has an auto bifurcation option.

 

[Rand__]

Pingback to a thread of mine re the -T card in x10 e5/x11 E3 boards

Auto-Bifurcation does not seem to be the primary approach (if it is there at all and not a support myth)