- Jan 2, 2021
So far I have my fair share of experience with those boards overall and in-depth with the H12SSL-NT. Some notes are imho in order:
- Those boards are fast. From my (our) tests they are among the fastest you can run EPYC on.
- There were no compatibility issues under Windows or RHEL/CentOS/AlmaLinux. Fedora runs into weird issues starting from installation. Debian, Ubuntu installed fine too (apart from the current Ubuntu-installer issues with software RAID on boot drives).
- I didnt encounter any stability issues with various hardware. Broadcom, Mellanox and friends all ran out of the box.
- The boards are standard compliant. Even though the compatibility list is short, there were no issues with different items not listed. Most notably M.2 NVME drives from various manufacturers and memory from Kingston (more about that later).
- M.2 - it seems as if any drive compliant up to NVME 1.3c will work just fine (no wonder since its mostly the CPU, yet nonetheless they work).
- Controller - Broadcom and Supermicro have a story together. So its no wonder I didnt see any issues with Broadcom RAID or HBA adapters.
- Memory - Kingston offers a "product finder". I found it reliable for these boards. Different Kingston memory according their product finder just works.
- H12SSL-C, H12SSL-CT: Both boards are almost identical and offer a Broadcom 3008 controller onboard. The T in the CT just declares that the Ethernet ports are 10G ports. Thats it, no magic.
- H12SSL-i, H12-SSL-NT: Both boards offer slimline SAS ports (SFF-8654), one (1) in case of the "-i", two (2) in case of the "-NT". The slimline SAS ports can be used as SATA-ports (but 8i) or NVME ports (also 8i). The different usage can be set by BIOS or jumper. Each Slimline port is connect with x8 PCIe lanes to the CPU. In case of the "-i" there are additional SATA ports available compared to the "-NT".
- Usage of the Slimline SAS ports require Supermicro specific cables (not expensive but sometimes hard to come by). For the Slimline SAS ports the options are direct connect for SATA/SAS/NVME (breakout cables) or backplane connect (due to the PCIe 4.0 8i nature of the port you need "uncommon" breakout cables for most backplanes too). If looking for cables, the most important part are the xi declaration and to make sure the endpoints xi sum up to 8i. Example: SATA/SAS breakout cables (usually without i since each is one lane) is Slimline SAS (SFF-8654) to 8x SATA/SAS. In case of NVME direct connect its a Slimline SAS (SFF-8654) 8i breakout cable to 2x 4i U.2/U.3 (SFF-8639).
- Windows Server 2019 works fine.
- RHEL 8.4/CentOS/AlmaLinux works fine. There are the usual log entries, which are easy to come by:
- error - watchdog: if you dont use the watchdog, just blacklist the sp5100_tco kernel module.
- error - mcelog: mcelog doesnt work on AMD CPUs, just stop and disable and use rasdaemon instead.
- If you use Gnome (you shouldnt on a server ), by then pulseaudio (part fo default graphical install) runs haywire as well as Bluetooth.
- Fedora - installation requires manual intervention.
- Ubuntu - works.
- Debian - works.
- SATA cables for the slimline SAS ports are angled towards the CPU. Be cautious since you can easily break the cables this way and they are always bent during operations.
- Use heatsinks for the M.2 drives. Even with great case air flow, high end M.2 drives will get close to critical temperatures under heavy sustained load. I suggest "be quiet! MC1 Pro, M.2 SSD-heatsink" - they are a tight fit, loosen both drives, insert and gently press down both together. Under heavy M.2 load they went up to 85°C (=critical) without heatsink. With heatsink temperatures dropped to 66°C max (10+ hours sustained load). Meaning without heatsink your M.2 drives may bail out under heavy load. With heatsink your M.2 drives never even come close to critical temperature, even under the heaviest sustained load.
- The IPMI password is under the first M.2 slot and on the transparent throwaway CPU cover (You can easily miss the password if you dont know). Alternatively you can reset it to ADMIN/ADMIN (jumper) or change it from command line with ipmi tools.