NVMe boot with SuperMicro X9DA7 & X9DRi-F

[awedio]

Parts used:
SuperMicro X9DA7
Samsung 960 EVO NVMe - 500GB
Angelbird Wings PX1
2x Kingston USB flash drives

Step 1:
Create Win10 UEFI USB.
I used the 1st part of these instructions: Windows 10 Tip: Create Windows 10 Setup Media the Right Way - Thurrott.com
I added 2 folders to the USB drive:
-Folder #1*: for the Samsung NVMe driver from here: Recommended AHCI/RAID and NVMe Drivers
* I installed the NVMe driver right before starting the Win10 install
-Folder #2**: Intel RST driver: Download Intel® RSTe AHCI & SCU Software RAID driver for Windows*
** this fixes the yellow bang in Device manager

This is what the contents of the USB stick look like after including the 2 folders:

Step 2:
Create a DOS bootable USB stick (I used Rufus - Create bootable USB drives the easy way)
**Grab the x9DA7 BIOS from Supermicro
**I used this guide to modify the BIOS
**I changed the Supermicro BIOS file extension from X9DA724.702 to .ROM (and back again when I was done with the mod)

Step 3:
Update the BIOS using the "modded" version (follow the SuperMicro instructions to upgrade)
Power down system after successful "modded" BIOS upgrade

Contents of modded BIOS USB stick (looks exactly the same at the stock BIOS):

Step 4:
Power system back on & change a few BIOS settings, F4 to save, exit & reboot

Step 5:
Power system on...F11 to invoke boot menu...Install Windows 10 (via UEFI)
After Win 10 reboot, go back into BIOS & set system to boot from "Windows Boot Manager" (aka NVMe M.2)
The NVMe drive will NOT show up as a boot drive.
"Hard Disk : Windows Boot Manager" is the NVMe drive.

 

[awedio]

Kudos to the all the peeps at Fernando's Win-RAID Forum (Storage Drivers - BIOS Modding)

Ha....Supermicro tech support said there was no way to do this

 

[awedio]

I ordered 1x PCI Express Gen 3 Carrier Board for 4 M.2 SSD modules - Amfeltec

Need to find a few $$ for some 960 EVOs

 

[mafrieger]

Many thanks for all these work and details.

Just modded Bios v3.2 of X9DRi-LN4F+
=> now 1TB 960Pro not only works but also boots perfectly W10 within kryoM.2 PCIe 3.0 x4

 

[mafrieger]

maybe also interesting (to save some time by preventing need for new install):
before making boot possible on Supermicro X9,
I migrated a sata ssd from MBR to GPT and than moved system partition to new Samsung (PCIe NVME)
(all done with MiniTool Partition Wizard pro v10.2)

 

[mafrieger]

only downtime: in Benchmark Seq. Read is not perfectly fast - it's nearly the same as write (around 2,000 MB/sec, which matches advertised expectations).

Are there any ideas on this?
spec: 3GHz 10c ivy, W10, latest Samsung driver

 

[mafrieger]

hm just noticed that 2 GByte/s would be typical for PCIe 2.0 4x
Faster speeds would only be possible when using 3.0 (which Ivybridge, the mainboard and of course 960 support)
so have to check if this is the problem. Any ideas how to do this best in W10?

 

[awedio]

@mafrieger Feels good to boot from NVMe, SM support said it couldn't be done !!
I have not touched my install since completing the mod.

 

[Jon Can]

I have an X10 motherboard with the same problem. I tried few NVME drives including the 960 PRO and EVO with no luck.
On the other hand the 950 PRO with the latest firmware update (2B0QBXX7) worked fine. This NVME is backward compatible with Legacy bios.

 

[Dave's Mayhem]

Many thanks for all these work and details.

Just modded Bios v3.2 of X9DRi-LN4F+
=> now 1TB 960Pro not only works but also boots perfectly W10 within kryoM.2 PCIe 3.0 x4

I am attempting to get this same board to recognize a 960 evo 250GB. I would love to have a copy of your modded BIOS, or any hints. I have already modded a BIOS, following the instructions of the previous links, but the mod doesn't appear to change anything. Still can't install.

 

[mafrieger]

As described I didn't make a complete new install but migrated the system from a sata ssd. But imho a fresh install should be not impossible.

some other lessons learned:
There was a problem when trying to update windows to latest 1709. After every reboot within the update process: the system hangs.
Solution: switch system of before it restarts and wait some seconds - so you do not have some reboots but do some cold starts during update process.

 

[Krobar]

Big thanks to mafrieger for the help. I managed to fresh install and configure an X9DRI-LN4F+. Couple of things not mentioned earlier:
1 - You cant UEFI boot from CD/DVD you must use a USB Bootdisk
2 - Didn't need any extra drivers at all, you can resolve the driver requirement for sata after the install (On Supermicro site as SCU driver)

I also tried to use an Asus Hyper M2 X16 quad card, only the first M2 slot works. I tried to enable the slot bifurcation with the hidden bios settings which did turn the PCIE speed setting for the slot from 1 setting to 4 but does not seem to configure the Root Ports correctly. PCIE Bifurcation can likely be added but only by a skilled bios engineer. Of course I am assuming that the Hyper M2 X16 card does not have any special requirements as I'm not sure how the VROC / Standard mode setting in Asus bios is applied.

 

[Eds89]

I also tried to use an Asus Hyper M2 X16 quad card, only the first M2 slot works. I tried to enable the slot bifurcation with the hidden bios settings which did turn the PCIE speed setting for the slot from 1 setting to 4 but does not seem to configure the Root Ports correctly. PCIE Bifurcation can likely be added but only by a skilled bios engineer. Of course I am assuming that the Hyper M2 X16 card does not have any special requirements as I'm not sure how the VROC / Standard mode setting in Asus bios is applied.

Interested in this;

I am looking to mod my X9SRL-F BIOS to support NVMe and use this very adaptor (want to use a 960 pro for ESXi/FreeNAS VM datastore, and a PM953 for a FreeNAS SLOG).
Keen to understand why the other three ports aren't available. Did you ever find out anything further on this?

SuperMicro have confirmed by board supports bifurcation, so guess it must be to do with the ASUS chips onboard and VROC?

EDIT: have put in a query to Asus to see what they say. Will report back.

Cheers
Eds

 

[Eds89]

Response from ASUS on bifurcation support for their Hyper card:

Hello James,

Thank you for the update, but unfortunately this card does not support PCIe Port Bifurcation.

 

[staph]

Theorectically, once you've installed the option ROM and gotten NVMe working with Windows 10, it should then also work with any modern Linux distro you install on that system, right?

 

[awedio]

***Update:
Using the same instructions, I "modded" the BIOS for a SM X9DRi-F and I can boot from NVMe (o/s is Server 2016).

 

[staph]

For my X9drl-i3/f I was also able to use the process above to flash the additional drivers into the default BIOS and then re-install the new BIOS into the board.

As I think others have mentioned, after this process the NVMe drive isn’t listed in the BIOS but I can install onto it and boot from it fine now.


Sent from my iPhone using Tapatalk

 

[awedio]

As I think others have mentioned, after this process the NVMe drive isn’t listed in the BIOS but I can install onto it and boot from it fine now.

This is normal behavior.
After the o/s is installed, go into the BIOS & you will see a hard disk (aka NVMe) labeled "Windows Boot Manager"

 

[staph]

This is normal behavior.
After the o/s is installed, go into the BIOS & you will see a hard disk (aka NVMe) labeled "Windows Boot Manager"

I forgot to mention, I installed Ubuntu 18.04 onto the drive. I was able to avoid having to use Windows on the server. But I still had to use Windows on a separate box in order to splice the new nvme rom into the original BIOS.


Sent from my iPhone using Tapatalk

 

[jaysa]

Response from ASUS on bifurcation support for their Hyper card:
Thank you for the update, but unfortunately this card does not support PCIe Port Bifurcation.

?? I can't think what Asus mean by that.

I've put an Asus Hyper M2 X16 quad card in the single x16 slot of an Asrock Z370 board using bifurcation of x16 to x8x4x4 set in the BIOS .

This makes a single x16 slot into 3 separately functioning sets of lanes at x8, x4 and x4, so the quad card works perfectly with 3 SSDs onboard - and would see 4 SSDs if the ASrock BIOS had a bifurcation option of x16 to x4x4x4x4 - which supported Asus boards do. Gigabyte also has Z370 boards that do this.

So the Asus quad card definitely works with bifurcation - I can't really see how it could work otherwise ...

Modding a BIOS to introduce bifurcation sounds tricky.