Intel VROC with Bifurcation Card, only detects one drive. But Windows sees all four.

[stvmyr]

Hi There!

We've just build new Machines based on Intels 3rd gen Intel Scalable.
We got a Delock 90077 PCIe Bifurcation Card that splits a x16 PCIe into 4 times x4.

After forcing the IIO configuration to "x4x4x4x4" and enabling VMD on the corresponding Port, all drives show up. But in I wired configuration.
One drive show up in the VROC menu. Three Drives Show up as "normal" PCIe Devices, but there are not VROC menu.
The Windows device manager shows 3 Intel Drives, and one NVMe RAID drive.

The Components we are used:

Supermicro X12SPi-TF
Intel Xeon Gold 5317
8x 64GB Samsung DDR4 ECC 2933MHz
4x Intel DC P4510 4TB NVMe SSD
VROCPREMOD License Key

The BIOS is up to date.
Anyone have a clue?

Thanks
Steven

 

[Negative Entropy]

Sounds like only one of the 4 "ports" is in VMD mode. When not in VMD mode the NVMe drives just show up as regular NVMe drives. I have a different X12 system and all of the NVMe connectors I am using are on mainboard (not using retimer cards). So my experience may not apply, but I'll try:

In my BIOS I set VMDs to "Auto" under NVMe Mode Switch and that makes all 8 ports that I have drives connected to VMD ports.

Looks like your BIOS may have 2 additional places you need to enable VMDs for each PCIe slot, sounds like you may have done this, but worth noting:
If you are using Slot 6 you need to enable it for IOU 0 and Slot 6;
If you are using the other x16 slot, Slot 4, you need to enable it for IOU 1 and Slot 4.

Also it's possible each VMD port needs to be enabled separately, but I have not played with the manual settings on mine, just Auto.

On a fresh OS install, with VMD set to off, the drives showed up in Windows server as regular NVMe drives. Enabling the VMDs in BIOS caused the drives to disappear (in device manager with "!") until the VROC drivers are installed in Windows. Note these drivers are not meant to be uninstalled ever, just upgraded.

I'll also warn you I have had some significant issues with the v7.8 and 8 VROC drivers that I am trying to work through with Supermicro. I only have the 1 VROC capable system so it may be some obscure hardware issue even though it's a fully supported configuration. I encourage you to test for corruption using the cofi open source tool. GitHub - yandexx/cofi: cofi — corruption finder. Designed to find data corruption on storage as the data is being written

I have attached a Windows executable I used (source of compiled binary: Veeam). Not sure if the forum takes zipped .exe files.

Example usage to test D drive:

C:\temp\cofi.exe -t 4 1M 10M D:\testfolder\testfile.dat

Please note this test does a lot of reads/writes on the tested drive. In my case with a similar system, CPU limited speed of about 250MB/s on a similar drive (per tested drive - seemed to scale to 4 drives in parallel). If you can run that for an hour you'll know your system does not have the same issue mine does. It runs until you escape it (ctrl+z) or it finds corruption.