Looking for FreeNas hardware advice

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
M

Matt Lund

Member
Oct 20, 2016
48
8
8
48
My intent is to run a FreeNAS 11 on a dedicated server, no virtualization.

So a dedicated server... So where I'm stuck is whether to get a used Supermicro X9 generation mobo and CPUs and RAM (heck, I'd buy the whole thing already built with the exceptions of the drives/adapter cards/additional NICs) versus whether to buy a brand new X11 motherboard with an E3 v6 processor and new or used DDR4 RAM. If I go the X11 route I'd still probably get a used 3u or 4u Supermicro case - I think.

I think I'll do the cost differences tonight but my impression as of now is that the used X9 route will give me more RAM (cheap DDR3 and the MB can take far more RAM than I'd ever install). The CPUs will have more cores but lower clockrates. IE - A dual proc E5-2xxx v1 will provide 16 cores. The additional cores will probably never be utilized by my FreeNAS box and the lower clockrate probably won't matter for my uses.

The X11 with single E3 v6 approach gives me a much newer part set, faster RAM, and faster single-threaded (higher clock, higher instructions per clock). But I'll be capped at 64GB RAM.

I'm leaning a bit towards the X9 with dual dual E5-2xxx v1 and DDR3 approach. It's nice that you can buy a system like this pre-built used and it's gonna "just work" versus that slight chance of getting something defective with new parts. I'm pretty comfortable troubleshooting hardware but this would be my only X11 system so I won't have spare CPUs or RAM or such to troubleshoot any problems that do come up.

My use cases will be:
- General NAS uses (many terabytes of media and such)
- Heavy torrenting which sometimes entails heavy read access when re-verifying data, etc.
- Periodic use in my software dev like SQL Server storage, etc.
- Maybe as a datastore for vsphere - though I may keep my vsphere box using local storage

I might play with volume encryption and dedup though I suspect I won't use either long-term.

Very few users but when I need to move terabytes of data I want it to go very fast. When torrent clients need to re-check multiple terabytes of data I want it to move fast. If I do use it as a SQL Server datastore I want that to be quick.

Anyone else go through the used X9 versus new X11 decision process for a dedicated FreeNas box? Any regrets or additional perspectives you gained after you made your choice?
 
S

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
I'd go with the X9 Plenty of compute for a NAS box and the cheaper RAM will let you give more to ZFS and ZFS likes all the RAM you will give it. The only advantage to going new would be saving some power but you can buy a lot of power for the price difference.
 
  • Like
Reactions: Matt Lund
T

ttabbal

Active Member
Mar 10, 2016
747
207
43
47
I agree, unless power is expensive for you, X9. Lots of RAM. If you need speed to a large amount of data, use lots of mirror pairs.

Keep in mind, dedupe is forever. At least last time I looked it was. Disabling dedupe does not remove the dedupe tables. So the higher RAM use etc stays the same. You have to create a new dataset and copy the data over, and then destroy the old one to clear that out. I've seen very few people get good use out of it.
 
  • Like
Reactions: Matt Lund
C

cactus

Moderator
Jan 25, 2011
830
75
28
CA
My remote FreeNas box is a single E5-2650v2(same speed and cores as the 2670) in a GA-7PESH2 with 64GB of ram, 8x dying 3TB Seagates. 4x 8TB Reds, 4x 500GB Raptors, F40 SSD, T420 nic. Single local 10GbE user for video editing and backup. It pulls a ton of power for what it does and it could have just been an E3, which my personal NAS is. I built thinking Corral was going to enable VM and containers all in a single nice interface. That didn't work out, but going back to Ubuntu would mean getting rid of the GUI which is nice when I have to walk someone through fixing something. I had CPU/mobo/ram/drives already and upgraded from a single G34 6128 HE that pulled less power. Dedup is enabled on the 8x3TB RAIDZ2 and it is a total waste, so it is not something I would use as a deciding factor. Dedup works really well for some datasets and worthless for others (like video, audio, pictures, compressed backups).

Be realistic with what it is going to be used for the majority of the time. If it is sitting there being a NAS 90% of the time, build for that and not the 10% of edge cases. E5v1 is cheap CPU/memory, but boards are not amazingly reasonable, idle power is not great, and boards with 10GbE come with a premium. Getting something with more modern 10GbE built in would be nice, which is not standard for the E3 line yet. That 10GbE is going to be using the same PCIe you need for HBAs on the E3 line. If I were building today, I would look for an X10SDV-* board with the onboard SAS controller, SFP+ 10GbE, and put 128GB of ram in it and forget about it for many years.
 
Last edited:
  • Like
Reactions: Matt Lund and whitey
M

Michael Hall

Member
Oct 9, 2015
40
10
8
Like cactus said, it depends on what you're doing...

My NAS (running Ubuntu, rather than FreeNAS) is a Pentium G3240, with 8GB of RAM and 5x2TB WD Reds, on a Supermicro X10SL7-F. It's basically just holding my movies, music, photos, etc., and is bored 99% of the time. My couple of docker containers run from a local SSD in the host.

Before that, I was running FreeNAS on an FX-4150, again with 8GB of RAM, but only 3x1.5TB WD Reds. That machine's now gone to a friend as an AIO, and he's perfectly happy with it.

On the other hand, at work, our 50TB or so of mixed NFS, iSCSI and FC runs on a NetApp FAS8020, which has a single E5-2520 with 24GB of system RAM and 8GB of NVRAM per controller. Our several hundred VMs live on a Pure FlashArray//m10, but what we know of its specs is covered by an NDA.
 
  • Like
Reactions: Matt Lund
M

Matt Lund

Member
Oct 20, 2016
48
8
8
48
So, I didn't mention power in OP and you guys did mention that if it is a factor I should consider that as criteria. I do live in an expensive power area (I think $0.40 per KWH). Hmmmmm... Now I'm wondering if I should be considering x9 versus x11 (E3) versus X10 (xeon d). If we compare x11 with e3 v6 + 10gbe adapter versus a xeon d with builtin 10gbe what kind of idle power differences are there? I'll try to read up more on this...
 
P

poutnik

Member
Apr 3, 2013
119
14
18
Xeon D-15xx is really one to consider, especially if your energy is so expensive. In the past week I have tried a comparison of Xeon D-1541 with Xeon E5-2660v1 machine, and the Xeon D under standard load (FreeNAS with ZFS disks, something like 6 VMs running) draws less energy than Xeon E5 idling... And yet, when I transferred the FreeNAS boot disk and all the data pool disks (3x3TB+3x2TB) to the E5-2660v1, the VM machines seemed to be slower...
 
T

ttabbal

Active Member
Mar 10, 2016
747
207
43
47
At $0.40/KWh, I would consider power a significant cost that should be considered. If you intend for this server to primarily be a filer, you don't need a lot of CPU, so why power it? RAM is useful, and there's only so much you can do about drives. Even Xeon-D makes sense at that level.

My high tier is $0.15/KWh and I'm installing a solar array.. At $0.40/KWh I think I'd move. :D
 
M

Matt Lund

Member
Oct 20, 2016
48
8
8
48
As an aside, we just moved from $0.11 *TO* the $0.40 plus! I'll start reading more about the Xeon D, thanks for pointing me there guys! After we sell our (now) out of state house and buy (here) I'll very likely do solar.
 
W

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Where do you live @Matt Lund? Hell I'd probably move or take solar more seriously if that were the case as well seeings how I run 3x E5 series servers 24/7/365. Right now I believe my stack costs roughly $45-50 a month at my KwH rate of $0.12.
 
M

Matt Lund

Member
Oct 20, 2016
48
8
8
48
We just moved from Utah to San Diego. Lots of reasons, electricity cost had to take a far lower priority :). The other bummer is that I had a proper equipment closet in my Utah home where noise and heat were no issue. I'm now in a house with nowhere to put a rack which is driving me crazy. Garage is possible but involves cable challengers and the internet isn't terminated there, whine whine whine :)
 
  • Like
Reactions: cactus
M

Matt Lund

Member
Oct 20, 2016
48
8
8
48
poutnik said:
Xeon D-15xx is really one to consider, especially if your energy is so expensive. In the past week I have tried a comparison of Xeon D-1541 with Xeon E5-2660v1 machine, and the Xeon D under standard load (FreeNAS with ZFS disks, something like 6 VMs running) draws less energy than Xeon E5 idling... And yet, when I transferred the FreeNAS boot disk and all the data pool disks (3x3TB+3x2TB) to the E5-2660v1, the VM machines seemed to be slower...
Click to expand...
Did you record the wattage numbers per chance? I'm curious if we're talking about a 10% difference or something much more substantial.
 
P

poutnik

Member
Apr 3, 2013
119
14
18
@Matt Lund no I didn't write the numbers down.

When I look at the remote management of the current configuration for my Intel Avamar machine with Intel S2600GZ board, 2CPUs (E5-2660v1), 64GB RAM in 8sticks, Dell 310 HBA, 4HDDs and 1SSD, one 750W PSU connected to 220V mains (europe), it says 96W minimum power draw, 124W average and 242W maximum.

From memory, the Xeon D-1541 build (MiniITX board) with 64GB RAM in 2 sticks, LSI HBA, 8HDDs and 2SSDs was idling somewhere around 60-70W and definitelly didn't hit anything over 100W anytime... When I do some system maintenance on it, I'll try to plug my Wattmetter there...
 
E

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
Xeon-D will idle and perform under load at significantly reduced power consumption compare to just about anything except the e3 v4/v5/v6 and e5 v4 which can both be very similar if designed well.
 
_alex

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
A single socket 2011 with 16xx CPU still can give plenty of (cheap) DDR3 RAM, high clocks and enough cores for ZFS. I saw a lot of people build FreeNAS around the 16xx single socket CPU.

I have a system with E5 1620 v2 / 64Gb in 8Gb sticks that has a very reasonable power draw, not really running Free NAS but Proxmox with a ZFS Pool that serves good speeds. This is a 1u JP intel box i got from kalleyomalley for a decent price, could be expanded with a JBOD, too. Not sure about the exact numbers atm, but saw this idle under 60W. Could do a more accurate measurement/summarize the actual specs the box has if you're interested.

Alex
 
You must log in or register to reply here.
Top Bottom