m.2 PCIe adapter in Supermicro x9 motherboards?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
J

jaysa

New Member
May 25, 2018
9
4
3
Yes. Presumably to carry an M.2 NVMe drive or adapter to U.2 SSD ? And perhaps to NVMe boot?

On my X9DRi-LN4F+ it doesn't work out of the box - BIOS mods needed. My next project ...

AFAIK, all single m.2 pci-e cards are equivalent, barring poor manufacture (I have 4 different ones from a previous project), so long as you only want one drive.

Beware of using the various cards that carry up to 4 drives, which require bifurcation support in the BIOS, which would be difficult to add. Though any 4 drive adapter will work as a 1 drive adapter without bifurcation.

There are also more expensive m.2 boards for multiple drives with PLX switchers that do not need bifurcation support, but I have no experience of those ...

Threads suggest some X9 boards can be made to work, others not. Anyone?
 
F

frogtech

Well-Known Member
Jan 4, 2016
1,482
272
83
36
@Savant and I have both used the Hyper M.2 x16 card from ASUS with the exact same results, it looks like only the first and fourth drive show up, when all 4 slots are populated. BIOS supports x4x4x4x4 bifurcation supposedly, but for some strange reason it just doesn't show all 4 drives. Very odd. I would like to try the ASRock implementation but it doesn't seem to be that available anymore, at least not for the price of the ASUS. Seems like a really missed opportunity on the part of ASUS or maybe even Supermicro?

I tested mine in the X9SRW-F.
 
C

cactus

Moderator
Jan 25, 2011
830
75
28
CA
frogtech said:
@Savant and I have both used the Hyper M.2 x16 card from ASUS with the exact same results, it looks like only the first and fourth drive show up, when all 4 slots are populated. BIOS supports x4x4x4x4 bifurcation supposedly, but for some strange reason it just doesn't show all 4 drives. Very odd. I would like to try the ASRock implementation but it doesn't seem to be that available anymore, at least not for the price of the ASUS. Seems like a really missed opportunity on the part of ASUS or maybe even Supermicro?

I tested mine in the X9SRW-F.
Click to expand...
With a modded BIOS for the x4x4x4x4?
 
K

Krobar

Member
Aug 25, 2012
57
12
8
I tried this with an X9DRI-LN4F+ too. Only tried the first two M2 slots but only 1 M2 drive was detected. The X9DRI-LN bios actually has the Bifurcation options hidden in the bios and I tried enabling them but they did not help. Looking at the comments above maybe I should have tried slot 4 on the Asus card.
 
realtomatoes

realtomatoes

Active Member
Oct 3, 2016
252
33
28
44
anyone managed to get bifurcation work on the x9?
i just got an x9srif and after the bios update i see an option to do it for slot 4 and slot 6.
neat feature if it would work.
 
M

metag

Member
Apr 26, 2016
64
16
8
realtomatoes said:
anyone managed to get bifurcation work on the x9?
i just got an x9srif and after the bios update i see an option to do it for slot 4 and slot 6.
neat feature if it would work.
Click to expand...
Yes, I have it working on my x9dri-ln4f+. Just make sure you set the correct slot up because the sequence number of the bifurcation settings (IOU# as shown in frogtech's screenshot above) is not the same as the PCI slot #, at least on the x9dri IIRC.

I loaded my Asus Hyper M2 x4 card with 4 HP 1TB ex920. They were then setup as a simple stripe disk in Windows Server 2016 storage space. Here are the CrystalDiskMark result:

-----------------------------------------------------------------------
CrystalDiskMark 6.0.1 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : Crystal Dew World
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 5018.222 MB/s
Sequential Write (Q= 32,T= 1) : 3461.086 MB/s
Random Read 4KiB (Q= 8,T= 8) : 1055.939 MB/s [ 257797.6 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 800.416 MB/s [ 195414.1 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 190.762 MB/s [ 46572.8 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 151.539 MB/s [ 36996.8 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 57.158 MB/s [ 13954.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 110.048 MB/s [ 26867.2 IOPS]
Test : 1024 MiB [H: 0.9% (35.6/3807.9 GiB)] (x5) [Interval=5 sec]
Date : 2018/11/03 9:48:58
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 14393] (x64)

I noticed the Random R/W 4K Q32T1 numbers are much lower than the numbers published here for the same disk. The number is similar when tested on single disk so it is not storage space issue. My server configuration is dual Xeon E5-2630 with 128GB memory, which I think should be more than enough. So I wonder whether that is a limitation of bifurcation in general or just the ASUS card. For $59, I probably should not expect more.
 
Last edited:
  • Like
Reactions: bogi, Marsh and realtomatoes
realtomatoes

realtomatoes

Active Member
Oct 3, 2016
252
33
28
44
metag said:
Yes, I have it working on my x9dri-ln4f+. Just make sure you set the correct slot up because the sequence number of the bifurcation settings (IOU# as shown in frogtech's screenshot above) is not the same as the PCI slot #, at least on the x9dri IIRC.

I loaded my Asus Hyper M2 x4 card with 4 HP 1TB ex920. They were then setup as a simple stripe disk in Windows Server 2016 storage space. Here are the CrystalDiskMark result:

-----------------------------------------------------------------------
CrystalDiskMark 6.0.1 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : Crystal Dew World
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 5018.222 MB/s
Sequential Write (Q= 32,T= 1) : 3461.086 MB/s
Random Read 4KiB (Q= 8,T= 8) : 1055.939 MB/s [ 257797.6 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 800.416 MB/s [ 195414.1 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 190.762 MB/s [ 46572.8 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 151.539 MB/s [ 36996.8 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 57.158 MB/s [ 13954.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 110.048 MB/s [ 26867.2 IOPS]
Test : 1024 MiB [H: 0.9% (35.6/3807.9 GiB)] (x5) [Interval=5 sec]
Date : 2018/11/03 9:48:58
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 14393] (x64)

I noticed the Random R/W 4K Q32T1 numbers are much lower than the numbers published here for the same disk. The number is similar when tested on single disk so it is not storage space issue. My server configuration is dual Xeon E5-2630 with 128GB memory, which I think should be more than enough. So I wonder whether that is a limitation of bifurcation in general or just the ASUS card. For $59, I probably should not expect more.
Click to expand...
thanks for confirming.
time to get me some more nvme.
 
K

Krobar

Member
Aug 25, 2012
57
12
8
metag said:
Yes, I have it working on my x9dri-ln4f+. Just make sure you set the correct slot up because the sequence number of the bifurcation settings (IOU# as shown in frogtech's screenshot above) is not the same as the PCI slot #, at least on the x9dri IIRC.

I loaded my Asus Hyper M2 x4 card with 4 HP 1TB ex920. They were then setup as a simple stripe disk in Windows Server 2016 storage space. Here are the CrystalDiskMark result:

-----------------------------------------------------------------------
CrystalDiskMark 6.0.1 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : Crystal Dew World
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 5018.222 MB/s
Sequential Write (Q= 32,T= 1) : 3461.086 MB/s
Random Read 4KiB (Q= 8,T= 8) : 1055.939 MB/s [ 257797.6 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 800.416 MB/s [ 195414.1 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 190.762 MB/s [ 46572.8 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 151.539 MB/s [ 36996.8 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 57.158 MB/s [ 13954.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 110.048 MB/s [ 26867.2 IOPS]
Test : 1024 MiB [H: 0.9% (35.6/3807.9 GiB)] (x5) [Interval=5 sec]
Date : 2018/11/03 9:48:58
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 14393] (x64)

I noticed the Random R/W 4K Q32T1 numbers are much lower than the numbers published here for the same disk. The number is similar when tested on single disk so it is not storage space issue. My server configuration is dual Xeon E5-2630 with 128GB memory, which I think should be more than enough. So I wonder whether that is a limitation of bifurcation in general or just the ASUS card. For $59, I probably should not expect more.
Click to expand...
Can you share which slot and IOU# setting you used?
 
M

metag

Member
Apr 26, 2016
64
16
8
Krobar said:
Can you share which slot and IOU# setting you used?
Click to expand...
Sorry, I did not get notified by your reply. My server is running right now. I will try to remember to take a screenshot next time I reboot it.

Meanwhile, I would think any one of the x16 slots should work. The key is to make sure the corresponding IOU setting is set to x4x4x4x4. I remember I set it wrong at first then I had to use some logic deduction to figure out which IOU# is corresponding to which PCIE slot. You should be able to do the same. Or you can just go with trial and error.
 
M

metag

Member
Apr 26, 2016
64
16
8
kron said:
Looking for this too. Having issues setting up this with only 2 drives on the asus hyper with the X9DRi-LN4F+.
Click to expand...
See my reply above. Also for 2 drives, which I started with, frogtech's hint earlier worked for me as well: install the two drives in M2 slot #1 and #4.
 
K

kron

New Member
Nov 12, 2018
10
0
1
Ok, this seems what I've done so far. IIO 2 IOU#2 & #3 are set to x4x4x4x4. The drives are in slot 1 and 4 of the asus card and it's in Slot 5. Still no drives on boot. I'm booting with UEFI enabled to an UNRAID usd drive.

Imgur: The magic of the Internet

Thanks for the help!!
 
M

metag

Member
Apr 26, 2016
64
16
8
kron said:
Ok, this seems what I've done so far. IIO 2 IOU#2 & #3 are set to x4x4x4x4. The drives are in slot 1 and 4 of the asus card and it's in Slot 5. Still no drives on boot. I'm booting with UEFI enabled to an UNRAID usd drive.

Imgur: The magic of the Internet

Thanks for the help!!
Click to expand...
Looks like IIO 2 IOU 2 is for SLOT 5 and you had it set. Things to consider:
1) Pretty sure you have. But just in case: do you have CPU 2 populated?
2) I think (not 100% sure) I used SLOT 3 (IIO 1, IOU2). Maybe you can try that one also?
3) I booted with legacy BIOS though
4) I only tried this with Windows Server 2016
 
K

kron

New Member
Nov 12, 2018
10
0
1
Yes, IIO2 IOU2 is for slot 5 but I had both set to switch the card to slot 4 and test.
-Yes, CPU 2 is present.
-Later today I'll try the other slots and move the LSI HBA cards around to test on all the x16 slots.
-According to some posts on the UNRAID forums UEFI boot is required for the card to display. I'll play with this more later and user a windows server install to see if I can get this running on there first.
 
K

kron

New Member
Nov 12, 2018
10
0
1
Bios Settings

Build Ver 3.3 - Latest and it looks correct.

CPU Configuration - Standard stuff here..

North Bridge Config - The card is on Slot 3 now IIO 1 IO3 @ x4x4x4x4. Feels like I've tried every combination of slot xMode here. I've read Gen3 is correct but haven't tried pulling that down to 1 or 2.

PCIe/PCI/PnP Configuration - Legacy Only and UEFI only
- Tried both options here with no change. Is there another setting I have wrong here?

Boot Options -



Unraid Details

Flash Drive - The server boot mode is set to UEFI on the flash drive.


PCI Devices and IOMMU Groups - No nvme devices here as expected.


I just don't know where to go, hahaha.
 
Last edited:
M

metag

Member
Apr 26, 2016
64
16
8
kron said:
North Bridge Config - The card is on Slot 3 now IIO 1 IO3 @ x4x4x4x4. Feels like I've tried every combination of slot xMode here. I've read Gen3 is correct but haven't tried pulling that down to 1 or 2.
Click to expand...
Maybe your screenshot was not taken at the right time. It was set for SLOT 1 (IIO1 IOU3), not 3 (IIO1 IOU2) ?
 
Last edited:
You must log in or register to reply here.
Top Bottom