Supermicro H11SSL-NC Rev 2 ATX AMD EPYC Motherboard

  • Thread starter Patrick Kennedy
  • Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
T

TXAG26

Active Member
Aug 2, 2016
416
134
43
In the review, it was mentioned that:
Supermicro will release a H12 generation motherboard after this review goes live that supports PCIe Gen4.
Click to expand...
Anything you can share on the timing of the upcoming H12 motherboard release?
 
P

Psycho_Robotico

Active Member
Nov 23, 2014
117
42
28
Sorry for reviving this thread, but is anyone out there using the board with an Epyc Rome (7xx2) and has SR-IOV working successfully?
Although the virtual functions of my Mellanox NIC do show up, they are stuck in the same IOMMU group. My older Epyc Naples doesn't have this issue. Any ideas?
 
P

Psycho_Robotico

Active Member
Nov 23, 2014
117
42
28
No, not anymore.

Code: 
Architecture:                   x86_64
CPU op-mode(s):                         32-bit, 64-bit
Byte Order:                                   Little Endian
CPU(s):                                          64
On-line CPU(s) list:                    0-63
Thread(s) per core:                     2
Core(s) per socket:                     32
Socket(s):                                      1
NUMA node(s):                           1
Vendor ID:                                   AuthenticAMD
CPU family:                                  23
Model:                                           49
Model name:                               AMD EPYC 7452 32-Core Processor
Stepping:                                      0
CPU MHz:                                    1498.813
CPU max MHz:                           2350,0000
CPU min MHz:                           1500,0000
BogoMIPS:                                  4699.62
Virtualization:                            AMD-V
L1d Cache:                                   32K
L1i Cache:                                    32K
L2 Cache:                                     512K
L3 Cache:                                     16384K
NUMA node0 CPU(s):              0-63
Flags:                                            fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca
 
Z

zir_blazer

Active Member
Dec 5, 2016
365
132
43
Psycho_Robotico said:
Sorry for reviving this thread, but is anyone out there using the board with an Epyc Rome (7xx2) and has SR-IOV working successfully?
Although the virtual functions of my Mellanox NIC do show up, they are stuck in the same IOMMU group. My older Epyc Naples doesn't have this issue. Any ideas?
Click to expand...
Did you enable in the Firmware PCIe ACS and AER? PCIe ACS is required for proper granularity of IOMMU Groups INCLUDING SR-IOV Virtual Functions.
 
P

Psycho_Robotico

Active Member
Nov 23, 2014
117
42
28
zir_blazer said:
Did you enable in the Firmware PCIe ACS and AER? PCIe ACS is required for proper granularity of IOMMU Groups INCLUDING SR-IOV Virtual Functions.
Click to expand...
IOMMU, SR-IOV and ACS are enabled in BIOS. I'm not quite sure if there is an AER option, but anything that looks remotely like it could be related to this is set to enabled. To make sure I even tried different combinations and the "Auto" setting.

Regarding the Mellanox NIC, it's set like this:

Code: 
mlxconfig -d /dev/mst/mt4099_pciconf0 q

Device #1:
----------

Device type:    ConnectX3       
Device:         /dev/mst/mt4099_pciconf0

Configurations:                              Next Boot
         SRIOV_EN                            True(1)         
         NUM_OF_VFS                          24             
         LINK_TYPE_P1                        ETH(2)         
         LINK_TYPE_P2                        ETH(2)         
         LOG_BAR_SIZE                        3               
         BOOT_PKEY_P1                        0               
         BOOT_PKEY_P2                        0               
         BOOT_OPTION_ROM_EN_P1               False(0)       
         BOOT_VLAN_EN_P1                     False(0)       
         BOOT_RETRY_CNT_P1                   0               
         LEGACY_BOOT_PROTOCOL_P1             None(0)         
         BOOT_VLAN_P1                        1               
         BOOT_OPTION_ROM_EN_P2               False(0)       
         BOOT_VLAN_EN_P2                     False(0)       
         BOOT_RETRY_CNT_P2                   0               
         LEGACY_BOOT_PROTOCOL_P2             None(0)         
         BOOT_VLAN_P2                        1               
         IP_VER_P1                           IPv4(0)         
         IP_VER_P2                           IPv4(0)         
         CQ_TIMESTAMP                        True(1)
Also, "options mlx4_core port_type_array=2,2 num_vfs=8,8,8 log_num_mgm_entry_size=-1"
is set in "/etc/modprobe.d/mlx4_core.conf".

All of this works flawlessly when sticking my older Epyc 7401 into the machine, or even in a Ryzen box.
 
T

TXAG26

Active Member
Aug 2, 2016
416
134
43
A new bios version was released today by Supermicro. Check the SM motherboard page for the link. It’s v2.1. It may address the issue you are experiencing.
 
  • Like
Reactions: Psycho_Robotico
P

Psycho_Robotico

Active Member
Nov 23, 2014
117
42
28
Thank you! I just checked their site a few days ago. Without your hint I surely wouldn't have checked again anytime soon. :)
The changelog appears promising...
 
V

vangoose

Active Member
May 21, 2019
326
106
43
Canada
Psycho_Robotico said:
Sorry for reviving this thread, but is anyone out there using the board with an Epyc Rome (7xx2) and has SR-IOV working successfully?
Although the virtual functions of my Mellanox NIC do show up, they are stuck in the same IOMMU group. My older Epyc Naples doesn't have this issue. Any ideas?
Click to expand...
Yes I have SR-IOV working. You need to turn on IOMMU and ASC. Don't leave them in Auto.

CX-3 doesn't support SR-IOV on ESXi 6.7.
 
You must log in or register to reply here.
Top Bottom