Picking a board/cpu for a SMB file server

[stresslvl0]

Hi all, I am putting together a build for a file server that can handle some light VM applications. 10Gb NICs are ideal, and IPMI is required.

Originally I was looking at the X12STH-F paired with something like a E-2378G but really prefer something with more throughput than the 2x1Gb ports & more SATA ports so I can have at least 8x HDDs for storage and 2x OS drives. I saw the X12SPM-TF which has 10 SATA and 10Gb NICs but that is quite the step up in cost (+325 from my vendor)

I see the A2SDI-H-TF looks like a decent option since I don't quite need the power of that Xeon CPU, and it is cheaper than the original combo, but I'm not sure if there is anything newer available or a better option I'm not thinking of. Really would love some perspective on this! Cheers!

Edit: this will be running FreeNAS Scale and using ZFS for the storage pools

 

[slidermike]

You can still use that mobo/cpu combo you wanted if you are willing to consume 2 of the pcie slots for an HBA controller and 10gb nic.
Going used on the pcie cards will save you the premium of new.

 

[stresslvl0]

I'd be willing to use the slots but on the other hand the A2SDi is ~$300 less than the original board/cpu combo (X12STH-F/E-2378G) and already has the 12 SATA ports and 10Gb NICs. I guess I'm having a hard time deciding on if the A2SDi is too old / slow compared to other options, vs the cost savings

Not trying to go used on any parts for this particular build

 

[Oarman]

The SuperMicro boards with integrated heat sinks are essentially designed for 1U systems, often with fan shrouds and/or industrial (loud) fans. Cooling them effectively outside those conditions can take trickery. If you are using a bigger case and want quiet operation, you're better off with the board with socketed CPU.

The 2378G is fairly high up in the Xeon-E stack. Just looking at NewEgg (I don't know your market) you could get a lesser Rocket-Lake Xeon for $300 less if you didn't need the CPU power. While the Atom C has a bunch of cores, they're older and slower; they're primarily useful if you can use the built in accelerators. Similarly you should consider if you a) want to use the iGPU b) if the board can even use it (I think you can with the C256 chip set, but verify.) No sense in buying a -G Xeon if you don't use the -G part. Unfortunately Intel doesn't let you drop an i3 in as a socket warmer any more.

 

[Tech Junky]

too old / slow compared to other options

well, a file server doesn't need much to begin with.

Now, the only thing that comes into play is the underlying transport and looking forward in storage moves in the future. Spinners will always be more economical and not require the best / newest tech to live inside a box only doing storage.

Things get more complicated when you go flash though and even moreso with NVME and dealing with lanes needed per drive for max performance requiring getting creative or paying up to do so. Though you can consolidate things a bit by going flash with the U.x drives where capacity can exceed 30TB/drive thus not needing to run tons of drives in raid for parity.

I was running R10 w/ 5 spinners for many years w/o issue and switched to flash instead to boost speed and keep capacity and ditch the raid costs. Going with a decent U drive should yield decent speeds / life span. I went with a Kioxia after testing some others that didn't work for more than a week but, the Kioxia hits 6.5GB/s on a single drive and at 15.36TB amply holds the prior configuration's data.

I guess what I'm saying here is there's no need to go with the enterprise CPU/MOBO aka spending $$$$ when you could put it towards the storage instead.

 

[TLN]

You can still use that mobo/cpu combo you wanted if you are willing to consume 2 of the pcie slots for an HBA controller and 10gb nic.
Going used on the pcie cards will save you the premium of new.

I keep mentioning: EPC612D4U-2T8R.

mATX board with Xeon with integrated both: HBA/SAS and 10G NIC. Small form factor, plenty of slots for spare devices.

 

[Wasmachineman_NL]

The SuperMicro boards with integrated heat sinks are essentially designed for 1U systems, often with fan shrouds and/or industrial (loud) fans. Cooling them effectively outside those conditions can take trickery. If you are using a bigger case and want quiet operation, you're better off with the board with socketed CPU.

The 2378G is fairly high up in the Xeon-E stack. Just looking at NewEgg (I don't know your market) you could get a lesser Rocket-Lake Xeon for $300 less if you didn't need the CPU power. While the Atom C has a bunch of cores, they're older and slower; they're primarily useful if you can use the built in accelerators. Similarly you should consider if you a) want to use the iGPU b) if the board can even use it (I think you can with the C256 chip set, but verify.) No sense in buying a -G Xeon if you don't use the -G part. Unfortunately Intel doesn't let you drop an i3 in as a socket warmer any more.

please don't get RKL in 2024, it's absolute shit and gets curbstomped by Alder/Raptor Lake in basically anything. Especially power use.

 

[DaveLTX]

And W680 boards can mount Core i CPUs so you can throw on a 12100 or 12400 to keep it cheap?
With a board like IMB-X1314. No IPMI but it has Vpro so that's something

 

[chinesestunna]

I'd be willing to use the slots but on the other hand the A2SDi is ~$300 less than the original board/cpu combo (X12STH-F/E-2378G) and already has the 12 SATA ports and 10Gb NICs. I guess I'm having a hard time deciding on if the A2SDi is too old / slow compared to other options, vs the cost savings

Not trying to go used on any parts for this particular build

I recommend against onboard SATA if you're running software raid in Linux. SAS controllers are cheap and much more robust when it comes to throughpit

 

[Wasmachineman_NL]

And W680 boards can mount Core i CPUs so you can throw on a 12100 or 12400 to keep it cheap?
With a board like IMB-X1314. No IPMI but it has Vpro so that's something

W680 boards IIRC cannot run Core CPUs, you need a Xeon.

 

[Tech Junky]

onboard SATA if you're running software raid in Linux.

I ran my disks for at least 5 years this way and they're still in great shape. No need for a HBA / Raid card though I did attempt one at one point and couldn't get it working and sent it back to CN.

Cards really only come into play when you need more density. It's just another layer of frustration if you need to find the cause of a failure.

 

[DaveLTX]

W680 boards IIRC cannot run Core CPUs, you need a Xeon.

You can, W680 came out before Xeon E2400.
You're thinking of C262

In fact it was the ONLY way to run ECC on a intel platform and was somewhat recognized as a workstation/server platform as intel never updated Xeon E until very recently.

 

[stresslvl0]

I recommend against onboard SATA if you're running software raid in Linux. SAS controllers are cheap and much more robust when it comes to throughpit

I should've mentioned up front, but this will almost certainly be running FreeNAS Scale and using ZFS for the storage pools. Does this change your opinion at all?

The SuperMicro boards with integrated heat sinks are essentially designed for 1U systems, often with fan shrouds and/or industrial (loud) fans. Cooling them effectively outside those conditions can take trickery. If you are using a bigger case and want quiet operation, you're better off with the board with socketed CPU.

Hmm, I wasn't thinking too much about that. I figured cooling wouldn't be a big deal for such a low powered chip.

I'm going to be building inside a Supermicro 743TQ-903B-SQ case.

I keep mentioning: EPC612D4U-2T8R.

mATX board with Xeon with integrated both: HBA/SAS and 10G NIC. Small form factor, plenty of slots for spare devices.

I'll definitely check to see if that is available and how pricing compares!

well, a file server doesn't need much to begin with.

Now, the only thing that comes into play is the underlying transport and looking forward in storage moves in the future. Spinners will always be more economical and not require the best / newest tech to live inside a box only doing storage.

Things get more complicated when you go flash though and even moreso with NVME and dealing with lanes needed per drive for max performance requiring getting creative or paying up to do so. Though you can consolidate things a bit by going flash with the U.x drives where capacity can exceed 30TB/drive thus not needing to run tons of drives in raid for parity. [...]

I'm going with WD CMR drives for this box. I don't foresee going with flash/ssd for this build, ever. I did some googling into Kioxia that you mentioned and I'm seeing something like $1500 for 12TB SSDs? Not sure if that's what you're referring to. I'd want online redundancy so that doesn't negate my need for RAIDZ/mirrors I believe.

 

[mattventura]

If you want to do primarily HDD storage, you can still get a pretty sizeable performance benefit from read and/or write caching on SSDs.

Onboard SATA is generally fine if it's a more enterprise-y kind of board. Also, if it's a more enterprise-grade board, it should come with SGPIO support so you can manage the 743's backplane.

I also happen to be running a lower-powered system in a 745 (743 but with redundant PSUs). Whole system (D-1521, 6 SAS/SATA HDDs, 6 SSDs, one expander) generally sits at ~150w wall power with my baseline CPU usage of about 30%.

 

[BlueFox]

W680 boards IIRC cannot run Core CPUs, you need a Xeon.

Most definitely can. I'm making this post using a W680 motherboard, 12900K, and ECC memory. Until a few months ago, LGA1700 Xeons didn't even exist.

 

[chinesestunna]

I should've mentioned up front, but this will almost certainly be running FreeNAS Scale and using ZFS for the storage pools. Does this change your opinion at all

As others have pointed out, onboard SATA can be fine - I'm not suggesting that one must go SAS, but my experience has been to go that route.
My first server was running 8 port onboard SATA with Linux mdadm raid and had disks drop or not detect once in a while. When I moved array to SAS controller this issue was gone and raid sync (check and rebuild) speed increased about 20% and this was with 100MBps 2TB HDDs back in 2010s.