TL;DR: What's the best Epyc motherboard to use with a Supermicro BPN-SAS3-826EL1 backplane (12 sata drives) and why?
With the release of Truenas scale, I plan to build a new server for it, and use my existing Truenas core one (an old Atom based SUPERMICRO MBD-A1SAi-2750F-O) for a backup. This is for 'home' use (homelab).
I bought a used Supermicro 2U case with 12 hot swap bays (826BE1C-R900LPB). It has a SAS3 backplane (BPN-SAS3-826EL1) with 4 mini SAS ports.
I've settled on an Epyc Rome/Milan CPU, but I'm having trouble settling on the motherboard due to my inexperience with server builds in general and SAS in particular - this is my first time building in a computer with a hot-swap backplane.
My primary requirements for the board are to be able to "feed" the case SAS backplane, and have 10Gb+ networking capability, while providing plenty of expandability (e.g. graphics card for plex transcoding, addition cards for Nvme drives). I'd prefer for the functionality of the motherboard overlap my wants/needs without buying adapter cards and extra cables (but I'm not averse to getting them if that's a better solution - the boards have plenty of PCI slots).
The first thing that hung me up: how many mini SAS HD connectors should I connect to the backplane for optimal performance?
Each mini SAS seems good for 4 drives, and the mobos or HBA cards seem to mostly do 2 mini SAS (8 drives). Since there are 12 drive bays I thought I needed 3 mini SAS to support all 12.
But after more research, I think the backplane has an "expander" that allows the bandwidth to be shared among all the drives? So maybe 2 mini SAS is adequate? I plan on using at least 8 hard drives (so no more than 4 SATA SSDs).
Additional points of confusion for me:
ROMED8-2T: has 2x mini SAS HD, but says they are SATA3 not SAS - does this matter for backplane compatibility? Or can I just get 2 SAS HD connectors and connect the motherboard to the backplane and have it 'work'?
MBDH12SSLNT: has 2 SLIM SAS, each of which can do 8 SATA on their own. But like the above, can it be work with the SAS backplane (rather than an 8 SATA splitting cable) with the appropriate connector? Does that connector even exist? (Looks like the SATA splitter is what's included in the retail box; I didn't see a slimSAS to mini SAS on the supermicro site, maybe I didn't know what I was looking for...)
MBDH12SSLC and MBDH12SSLCT: explicity have an LSI SAS 3008 controller. If the above(connecting mini HD SATA ports) works, what is the advantage of having the 3008 controller on the board? (since I'll be running zfs via Truenas, I think any raid features would be useless)
Boot drives for Truenas: I figure I would mirror, but I'm not sure which of the options is best for my 'homelab' like environment. This choice could influence the motherboard choice.
* 2 SATA SSDs in hot swap bays (reducing hot swap 'usable' storage to 10 - probably not an issue)
* 2 NvMe SSDs on the motherboard (don't think truenas makes use of the nvme performance, so this could have been used for high performance storage)(that said, there is no shortage of PCI slots to get add-on nvme pci cards)
* 2 SATA-DOM (MBDH12SSL C, CT, and i)(SATA-DOM drives seem rather expensive, and other 6 SATA ports on the motherboards would likely be unused?)
* 2 SATA SSDs in internal storage (make some custom brackets for the case to hold them, and use any free SATA ports for them)
Networking: ROMED8-2T has intel X550, Supermicro has Broadcom. I've read that Intel is 'better' (esp. for compatibility) but if the Broadcom works in Truenas scale do I care? Do people have experience with one versus the other in compatibility and power consumption?
Or should I ignore the 10G Base T and get some kind of SFP NIC?
While I have rewired the house for cat6 to support 10GbaseT, I only have 1Gb switches atm. So the switch compatibility is not a consideration at this point (presumably I can buy something that will work).
Does anyone know if the motherboard NICs consume any appreciable power if they are not connected? Having the 10G base T available on the mobo seems like a nice 'future proofing' feature, but not if it is gobbling up power doing nothing.
Some other considerations:
Any use of oculink ports in a Homelab? (from the ROMED8-2T board) I read (I think in these forums) that they don't work for SAS and are essentially for U.2 enterprise SSD storage.
I love USB-C in my desktop computers, but does it have any use in the server rack? (from the ROMED8-2T board)
Given that the case doesn't seem to have any free SATA power cables or drive mounting points, are sata ports on the mobo good for anything?
What can practically (i.e. with cables that don't cost a ton) be done with SlimSAS ports (MBDH12SSLNT and MBDH12SSLi)?
Thanks in advance for any advice you folks can provide!
Also I want to thank the folks who have already written about these motherboards, especially @lihp who posted Supermicro H12SSL-i, H12SSL-C, H12SSL-NT, H12SSL-CT boards - notes, experiences and @jpmomo for his comments about these 2 boards.
With the release of Truenas scale, I plan to build a new server for it, and use my existing Truenas core one (an old Atom based SUPERMICRO MBD-A1SAi-2750F-O) for a backup. This is for 'home' use (homelab).
I bought a used Supermicro 2U case with 12 hot swap bays (826BE1C-R900LPB). It has a SAS3 backplane (BPN-SAS3-826EL1) with 4 mini SAS ports.
I've settled on an Epyc Rome/Milan CPU, but I'm having trouble settling on the motherboard due to my inexperience with server builds in general and SAS in particular - this is my first time building in a computer with a hot-swap backplane.
My primary requirements for the board are to be able to "feed" the case SAS backplane, and have 10Gb+ networking capability, while providing plenty of expandability (e.g. graphics card for plex transcoding, addition cards for Nvme drives). I'd prefer for the functionality of the motherboard overlap my wants/needs without buying adapter cards and extra cables (but I'm not averse to getting them if that's a better solution - the boards have plenty of PCI slots).
The first thing that hung me up: how many mini SAS HD connectors should I connect to the backplane for optimal performance?
Each mini SAS seems good for 4 drives, and the mobos or HBA cards seem to mostly do 2 mini SAS (8 drives). Since there are 12 drive bays I thought I needed 3 mini SAS to support all 12.
But after more research, I think the backplane has an "expander" that allows the bandwidth to be shared among all the drives? So maybe 2 mini SAS is adequate? I plan on using at least 8 hard drives (so no more than 4 SATA SSDs).
Additional points of confusion for me:
ROMED8-2T: has 2x mini SAS HD, but says they are SATA3 not SAS - does this matter for backplane compatibility? Or can I just get 2 SAS HD connectors and connect the motherboard to the backplane and have it 'work'?
MBDH12SSLNT: has 2 SLIM SAS, each of which can do 8 SATA on their own. But like the above, can it be work with the SAS backplane (rather than an 8 SATA splitting cable) with the appropriate connector? Does that connector even exist? (Looks like the SATA splitter is what's included in the retail box; I didn't see a slimSAS to mini SAS on the supermicro site, maybe I didn't know what I was looking for...)
MBDH12SSLC and MBDH12SSLCT: explicity have an LSI SAS 3008 controller. If the above(connecting mini HD SATA ports) works, what is the advantage of having the 3008 controller on the board? (since I'll be running zfs via Truenas, I think any raid features would be useless)
Boot drives for Truenas: I figure I would mirror, but I'm not sure which of the options is best for my 'homelab' like environment. This choice could influence the motherboard choice.
* 2 SATA SSDs in hot swap bays (reducing hot swap 'usable' storage to 10 - probably not an issue)
* 2 NvMe SSDs on the motherboard (don't think truenas makes use of the nvme performance, so this could have been used for high performance storage)(that said, there is no shortage of PCI slots to get add-on nvme pci cards)
* 2 SATA-DOM (MBDH12SSL C, CT, and i)(SATA-DOM drives seem rather expensive, and other 6 SATA ports on the motherboards would likely be unused?)
* 2 SATA SSDs in internal storage (make some custom brackets for the case to hold them, and use any free SATA ports for them)
Networking: ROMED8-2T has intel X550, Supermicro has Broadcom. I've read that Intel is 'better' (esp. for compatibility) but if the Broadcom works in Truenas scale do I care? Do people have experience with one versus the other in compatibility and power consumption?
Or should I ignore the 10G Base T and get some kind of SFP NIC?
While I have rewired the house for cat6 to support 10GbaseT, I only have 1Gb switches atm. So the switch compatibility is not a consideration at this point (presumably I can buy something that will work).
Does anyone know if the motherboard NICs consume any appreciable power if they are not connected? Having the 10G base T available on the mobo seems like a nice 'future proofing' feature, but not if it is gobbling up power doing nothing.
Some other considerations:
Any use of oculink ports in a Homelab? (from the ROMED8-2T board) I read (I think in these forums) that they don't work for SAS and are essentially for U.2 enterprise SSD storage.
I love USB-C in my desktop computers, but does it have any use in the server rack? (from the ROMED8-2T board)
Given that the case doesn't seem to have any free SATA power cables or drive mounting points, are sata ports on the mobo good for anything?
What can practically (i.e. with cables that don't cost a ton) be done with SlimSAS ports (MBDH12SSLNT and MBDH12SSLi)?
Thanks in advance for any advice you folks can provide!
Also I want to thank the folks who have already written about these motherboards, especially @lihp who posted Supermicro H12SSL-i, H12SSL-C, H12SSL-NT, H12SSL-CT boards - notes, experiences and @jpmomo for his comments about these 2 boards.