Future FreeNAS platform considerations (Intel Xeon E vs Xeon v6)

ReturnedSword

Active Member
Jun 15, 2018
400
119
43
Santa Monica, CA
I'm planning to build a FreeNAS in the near future (possibly around November/December). I would very much like to use an AMD based platform, but there seem to be some issues with Zen support even 18 months after it was released. Specifically I would like to use ECC RAM, and don't prefer to implement a non-official version of it which some motherboards seem to support. That leaves me with an Intel platform.

My current list have the following core components:
  1. Supermicro X11SSL-CF (C232)
  2. Intel Xeon E3-1275 V6
  3. 2 x 16GB DDR4 2400MHz
I'm planning to use the FreeNAS mostly for backup and media storage, VM images for my homelab, and maybe a few containers (e.g. Plex). I'd have to see how constrained I am on throughput, and if I am, use a 10 GbE NIC later. The FreeNAS will use a 6 drive RAID-Z2 zdev0/zpool, which I'll later add a second identical zdev1.

I've been following the Xeon E launch, and it appears much better than the E3-120x v6, at least on paper. Both platforms are constrained by 16 PCIe lanes, which isn't a huge issue for me since the maximum number PCIe slots I'd be using is perhaps 2x PCIe x4.

This motherboard looks nice: Supermicro X11SCA-F (C246). The main annoyance is that it is a workstation motherboard with IPMI, and it doesn't have SAS built-in, but that can be rectified by a HBA card.

Is the Intel C246 chipset a workstation/desktop chipset that had been re-purposed for entry level servers?

Would I be better served to wait out the soft-launch of the Xeon E (for the 8-core variant and more server oriented motherboards)?
 

Stephan

Active Member
Apr 21, 2017
374
220
43
Germany
As a rule of thumb, from the date you can actually buy Intel workstation/server class hardware like C23x/C246 I would personally wait 12 months until I'd consider buying it and implementing Linux on it. It takes this long for drivers to be written or adapted, people to buy the hardware for Linux, find bugs in hardware and/or software, let Linux driver writers correct that, and then some more months to shake out bugs in those rewritten parts.

With FreeNAS (FreeBSD-based) it could be 18 to 24 months, at which point the hardware is close to EOL already. Or it could be never, if compatibility of FreeNAS for a certain platform is not progressing into a stable plateau, which means you'd have to either be very picky for hardware (Intel only, with select components) or choose an option different from FreeNAS.

For your purposed I'd get more RAM (64 MB UDIMM ECC, max on C236) and just go with the 1275v6. Also go with a SAS expansion card rather than onboard, because more options, mainboards easier to find and cheaper, and better replaceable in case it fails. Make sure to cool it well, i.e. put the card into the direct air flow of a fan.

Aspeed 2x00 IPMI will draw 3-5 watts more power than without, in your case can be ignored with all those disks in there. You will have to experiment with different PWM fans a little to get good cooling with not a whole lot of noise. Most mainboards lack fine control of fans, might necessitate a dedicated fan control solution with sensors.
 

K D

Well-Known Member
Dec 24, 2016
1,439
320
83
30041
The C232 chipset in the X11SSL-CF doesnt support Intel HD graphics. So with E3-1275 v6 go with a C236 chipset (I'm using an X11SSH-CTF with this CPU). You will be able to use the intel graphics in your VMs, especially for plex transcoding.
 

ReturnedSword

Active Member
Jun 15, 2018
400
119
43
Santa Monica, CA
One thing that is quite nice about the Coffee Lake S Xeon E is you can use the quad core i3-8300, or the upcoming octacore Xeon E with a newer platform. That's my main draw tbh.

Stephan, you make a good point about EOL. At work we are used to getting to play with the newest stuff, so I am a bit spoiled there. I am aware many homelab setups use older platforms, even going back to the x5xx Xeon. I would prefer newer hardware, and am planning to buy new components (exception for HBAs and NICs).

I was planning on 32GB RAM initially with the 6 x 8TB vdev0, and adding another 32GB RAM when I add a second 6 x 8TB vdev1. I am not familiar with FreeNAS (this will be my first jab at it). Does adding more ARC really improve performance by a large amount? Previously, I think FreeNAS had a 1TB:1GB rule of thumb, but with newer versions I am not sure if this still applies.

K D, thanks for your recommendation. Looks like the X11SSH-CTF is much more expensive, though it would get me dual 10 GbE straight away and save me having to buy an X550 converged NIC later. For transcoding, would it be helpful to drop in a GPU? I have a spare GeForce GTX 770 that might work nicely for that.

I am a bit ashamed to say that I will be using a Fractal Design R5 for this build. I only have 2U left in my rack. Later, I would love to buy one of those top load 4U cases, though.
 

Krobar

Member
Aug 25, 2012
54
10
8
I have an X11SCA-F. With the soon to be release bios update Linux support is excellent with one sore exception which has been identified as a kernel bug. The PCI quirk applied for the Intel 300 series chipset breaks PCI passthrough on the X11SCA-F. For now it is easiest to use a Pre May 2018 kernel which wont contain the patch, I have a patch to test this weekend which should sort the problem but it wont likely main line for a month or two (Assuming it is accepted)

I'll try to write up my thought in a week or 2 on this board once the patch is approved and the new bios released but any questions please ask.
 

ReturnedSword

Active Member
Jun 15, 2018
400
119
43
Santa Monica, CA
@Krobar Yes that would be great i f you can report back your experiences with the X11SCA-F! May I ask how much you purchased your motherboard for? I've seen some listings, but none of the major US sellers. The board also seems to be going for around $300 USD, which seems a bit high for a server-lite motherboard without an included SAS controller.
 

HeffElf

New Member
Sep 29, 2018
2
0
1
@Krobar I'm also really interested. I currently have a freenas install, my first one, on an old gaming mobo which has a failing CPU (lots of L3 cache errors) and I want to redo my server with better hardware. I'm primarily using my server to run Plex in a jail. I have 6 4tb we red disks and a SSD boot disk, so the 8 SATA connectors on the x11sca-f are appealing to me.
I also wanted to run a e-2146g CPU as it has a passmark score of over 16000 so it looks like it future proofs me if I want to transcode 4k stuff.
I can't really find any info about what support there is for the c246 chipset except for this thread.
Thanks in advance for any help, I'm a total noob when it comes to this stuff.
 

ReturnedSword

Active Member
Jun 15, 2018
400
119
43
Santa Monica, CA
@HeffElf I still haven't seen the X11SCA-F (or Xeon E CPU/platforms for that matter) available for sale anywhere here in the US. There are a handful of UK/EU e-tailers that have the X11SCA-F in stock, but I don't need it so urgently that I would pay the shipping and import duty costs.

Considering the channel availability issues for Intel 14nm CPUs at the moment, it's probably best to wait and see how the upcoming 8 core Xeon E performs (some sparse reports saying it will be released in Q4, but I'm not holding my breath). The longer I wait, I'm hoping there will be a proper ITX C246 motherboard released as well for a new pfSense I've been wanting to build.
 

HeffElf

New Member
Sep 29, 2018
2
0
1
@HeffElf I still haven't seen the X11SCA-F (or Xeon E CPU/platforms for that matter) available for sale anywhere here in the US. There are a handful of UK/EU e-tailers that have the X11SCA-F in stock, but I don't need it so urgently that I would pay the shipping and import duty costs.

Considering the channel availability issues for Intel 14nm CPUs at the moment, it's probably best to wait and see how the upcoming 8 core Xeon E performs (some sparse reports saying it will be released in Q4, but I'm not holding my breath). The longer I wait, I'm hoping there will be a proper ITX C246 motherboard released as well for a new pfSense I've been wanting to build.
@hoankiem, I just ordered the motherboard from Newegg and it's already been shipped. So not sure about availability issues there. I also ordered a CPU, but initially the seller said they had it in stock then when. I completed the order they said it was back ordered, so we'll see how that goes...
 

ReturnedSword

Active Member
Jun 15, 2018
400
119
43
Santa Monica, CA
Nice, I'm not quite ready to buy mine yet since plans got pushed back a bit due to a fast approaching work trip. By the time I get back, it'll be nearly Cyber Monday already so hopefully I can nab some good deals for other components to complete the build.