Supermicro H12SSW-NT - FreeNAS 11.3-U1 NVMe issues

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

chrisk2305

Member
Jan 25, 2017
38
2
8
40
Hi Guys,

I just completed my new freenas build with a Supermicro H12SSW-NT Mainboard and an eight Core Epyc 2 CPU. I have connected 4 U.2 NVMe SSDs to the board via two slimsas x8 cable.

The disks simply won't appear. Neither in GUI nor Shell.

I spent a good amount of time checking the BIOS and the cabling and finally decided to try the new TrueNAS 12 nightly. And voila - all 4 appeared.

Do you have any idea what FreeNAS 11.3 may be missing? AMD NVMe driver? I just don't get it.

Thanks in advance!

Chris
 

KarelG

Member
Jan 29, 2020
48
13
8
Nobody here with an Epyc 2 System and NVMe drives? Any help is much appreciated!
FreeNAS is based on FreeBSD, please give a try to some Linux live CD/USB and boot that. Do you see your NVMe drives in dmesg? Also do you see drives in ipmi/web management console? If so, then there is a hope just FreeBSD is missing driver(s) for your NVMe drives... In which case, better is to ask on FreeBSD mailing list -- but after you try to boot FreeBSD 13-current and neither see drives. If you would see drives in 13-current, then ask on FreeNAS mailing list.

Have fun!
 

chrisk2305

Member
Jan 25, 2017
38
2
8
40
Well I tried the TrueNAS 12 nightly (FreeBSD 12, right?) and I can see all 4 drives, so it is definitely a driver issue.
 

chilipepperz

Active Member
Mar 17, 2016
212
64
28
54
FreeBSD is so far behind Linux in driver support. Remember the c3000 drivers that took like years.

Hey why use a H12 EPYC system for FreeNAS? Do you have a 48+ drive system? Can you post the build?
 

chrisk2305

Member
Jan 25, 2017
38
2
8
40
Why not :) I wanted to have a highly integrated Mainboard with PCIe 4.0 support to be future proof. I am using this (although unsupported by Supermicro) in a 2U BAC4 WIO Chassis with Hybrid backplane (4x NVMe + 8x SATA/SAS) with 64GB RAM, 4x10TB Seagate Exos SAS Disks, 4x4TB WD Red, 2x256GB 860 EVO SSDs (in the rear of the chassis), 2x Samsung U.2 1,96TB SSDs (iSCSI for VMWare), 2x512GB M.2 SK Hynix SSDs in U.2. chassis (as cache for the 4x10TB SAS disks) - at least in theory because of my NVMe issue :-(
 
  • Like
Reactions: Ahmann Robert

KarelG

Member
Jan 29, 2020
48
13
8
Chris, nice report, but, I would *strongly* suggest to attach dmesg from FreeBSD boot where you have seen the drives. This way, FreeNAS staff knows what driver issue it is. It's way more convenient (for devs) to see this than to look and search for info about the board and platform on the internet, specially if you have been able to see drives in whatever FreeBSD you tried.
I suggest this, since this way (supply all you know and is relevant) you may have higher chance to get reply.
 

chrisk2305

Member
Jan 25, 2017
38
2
8
40
@KarelG: Yes, you are right. The issue I am facing with posting the dmesg output from TrueNAS nightly is that my NICs don't work in that version so I cannot copy and paste the output. A bit of a dilemma.
 

KarelG

Member
Jan 29, 2020
48
13
8
@KarelG: Yes, you are right. The issue I am facing with posting the dmesg output from TrueNAS nightly is that my NICs don't work in that version so I cannot copy and paste the output. A bit of a dilemma.
Save dmesg to usb flash. Or create newfs on drive accessible from older FreeNAS and save it there, or ... well, there is a lot of ways to do. In the worst case, show dmesg on the screen/terminal and then shoot with your mobilephone and post there picture. -- this way, scroll dmesg to part showing drives of course (and driver). Etc.
 
  • Like
Reactions: chrisk2305

KarelG

Member
Jan 29, 2020
48
13
8
I attached both dmesg outputs to the Jira Ticket.
Thanks, for me it is always interesting to see dmesg of such machines. Clearly nvd driver is either missing or pcie traversal buggy on 11.3. Fortunately for you, there is already man assigned to your task so good that you have provided this info...
 

joet

New Member
May 27, 2018
15
2
3
Sorry to drag this off-topic but @chrisk2305 could you please share the part-numbers of the Slim-SAS 8i -> U.2 drive cables you used? I am trying to find some for my H12SSL-NT.
 

joet

New Member
May 27, 2018
15
2
3
I believe the cable should be the cbl-sast-0953
Many thanks - I also reached out to the Supermicro online-shop's support and after a few hours figuring it out, they also came back with that part number for the H12SSL-NT.