Upgrading NAS to Xeon, thoughts?

[Solkku]

Hi guys and greetings from Finland!

I'm a long time reader, first time poster. I wanted to run this by you, as you guys probably have better understanding about these things than me. I currently have NAS4Free running with 3 jails (Apache, Plex & ownCloud) on my HP Microserver N54L (1 x 4GB ECC ) and wanted to upgrade to a more powerful NAS. The Microserver is currently populated with 3 x 3TB WD Red drives in RaidZ1 (for my use) and 1 x 2TB WD Red (my friend's 2nd backup to ownCloud).

Autumn is approaching fast and soon it's time for me to play modded Minecraft again. I don't want to host a server on my desktop PC again this winter, so I wanted my NAS to take care of it. I don't think the CPU in my current NAS will cope with modded Minecraft, even if I feed it more RAM, hence why I turn to you guys.

I've researched & researched about this matter and initially I was attracted to ESXi, but after reading horror stories about ZFS pools failing in the VM environment, not so much anymore (prove me wrong! ).

I really want my Plex media on my 3x3TB pool to stay safe, since I share the library with my family and they rely on it.

Here's the new parts I've been looking at:
CPU: Intel Xeon E3-1231v3 (256€)
Motherboard: Asus P9D-M C224 (178€)
Case: Fractal Design Node 804 (91€)
Memory: 2 x Kingston 8GB ECC DDR3L, KVR16LE11/8KF (á 62€, 124€ total)
PSU: Cooler Master 450W GM Series modular (49€)
Custom 1: 3 x 120mm Arctic F12 case fans (á 3,50€, 10,50€ total)
Custom 2: Thermalright True Spirit 90 CPU cooler (~25€)
Custom 3: SanDisk Cruzer Fit 8GB USB 2.0 stick (6€)

Total: ~740€ + ~40€ shipping

I didn't opt for any hard drives as I was thinking about moving the four current ones from my old NAS to the new one.

Then the crucial questions/points:
- Is this overkill for what I'm trying to achieve?
- Is running ESXi a viable/reliable option for a home server with NAS4Free (or even FreeNAS) if you do it right? Or should I just go bare-metal on the NAS OS with phpVirtualBox for the "random VM experiments"?
- Do I benefit from the HT in E3-1231v3 or should I save ~20€ and go for the E3-1225v3/E3-1226v3?
- Intel Avaton C2750 SOC boards are tempting, but expensive (~450€ shipped to Finland at the cheapest). The combination of the above CPU+motherboard is ~435€ and with support for VT-d in ESXi if needeed.
- I plan on selling the HP Microserver N54L once I have the new NAS up and running (to help fund the new build). Money doesn't grow on trees for me, unfortunately.
- If I only want one NAS in my apartment, which route should I go software-wise? I can source a ~100-200GB SSD for a ESXi datastore for VMs if needed.
- Is there some other OS that can do what I want to achieve, but more elegantly?
- I still have space on desktop hard drives to empty out the Microserver NAS if needed.
- Supermicro boards with IPMI would be nice, but cost 2x more.

I know 16GB might be pushing it in ESXi (plus a modded minecraft server running), but I'm not yet set on ESXi. This is a merely a starting point for me, since the CPU & motherboard supports 32GB ECC RAM. I'll be upgrading once I get the funds to do so.

I apologize for the long post and really appreciate if anyone of you have some input.

 

[cheezehead]

Overkill: Depends on the workloads, if using MKV's for the video contain with Plex the Nix VC-1 decoder is single threaded so you'll need the single-threaded performance.
VM: well it depends, some think it's sacrilege to run your NAS as a VM with using ZFS... Personally I'd use a cheaper motherboard that supports VT-d and a separate controller for the disks....blindly pass it through the VM. Easy change before hardware is purchased and keeps performance smooth. After running a similar setup, I moved to just using Jails within FreeNAS works-around the VT-d issue and everything is clean.
HT: tough call
C2750 route: works good as long as your not thinking of doing MKV's for Plex....VC-1 is too much for it if running Plex on Linux/Unix...if running a Windows VM technically it should work


The ESXi question is a really about what do you want to ultimately do with the setup. I use to run ESXi in my home lab but converted to running FreeNAS with jails (lack of VT-d on my setup). Apache/Plex/OwnCloud all run without any issues. The only issue I have has been an existing windows server vm (IIS & mySQL) which I moved to running on vmware workstation on my desktop for now.

I know some of the guys over at the FreeNAS forum were looking into getting a virtual box jail so it could handle windows VM's....extremely light duty mind you but if you needed an RDP VM for some reason.

Depending on how soon you are looking at buying, possibly some of the D-1520 boards might be an option. They should be shipping later this month but I have no idea how the availability of those boards would be in Europe.

 

[Patrick]

Hi guys and greetings from Finland!

I'm a long time reader, first time poster. I wanted to run this by you, as you guys probably have better understanding about these things than me. I currently have NAS4Free running with 3 jails (Apache, Plex & ownCloud) on my HP Microserver N54L (1 x 4GB ECC ) and wanted to upgrade to a more powerful NAS. The Microserver is currently populated with 3 x 3TB WD Red drives in RaidZ1 (for my use) and 1 x 2TB WD Red (my friend's 2nd backup to ownCloud).

Autumn is approaching fast and soon it's time for me to play modded Minecraft again. I don't want to host a server on my desktop PC again this winter, so I wanted my NAS to take care of it. I don't think the CPU in my current NAS will cope with modded Minecraft, even if I feed it more RAM, hence why I turn to you guys.

I've researched & researched about this matter and initially I was attracted to ESXi, but after reading horror stories about ZFS pools failing in the VM environment, not so much anymore (prove me wrong! ).

I really want my Plex media on my 3x3TB pool to stay safe, since I share the library with my family and they rely on it.

Here's the new parts I've been looking at:
CPU: Intel Xeon E3-1231v3 (256€)
Motherboard: Asus P9D-M C224 (178€)
Case: Fractal Design Node 804 (91€)
Memory: 2 x Kingston 8GB ECC DDR3L, KVR16LE11/8KF (á 62€, 124€ total)
PSU: Cooler Master 450W GM Series modular (49€)
Custom 1: 3 x 120mm Arctic F12 case fans (á 3,50€, 10,50€ total)
Custom 2: Thermalright True Spirit 90 CPU cooler (~25€)
Custom 3: SanDisk Cruzer Fit 8GB USB 2.0 stick (6€)

Total: ~740€ + ~40€ shipping

I didn't opt for any hard drives as I was thinking about moving the four current ones from my old NAS to the new one.

Then the crucial questions/points:
- Is this overkill for what I'm trying to achieve?
- Is running ESXi a viable/reliable option for a home server with NAS4Free (or even FreeNAS) if you do it right? Or should I just go bare-metal on the NAS OS with phpVirtualBox for the "random VM experiments"?
- Do I benefit from the HT in E3-1231v3 or should I save ~20€ and go for the E3-1225v3/E3-1226v3?
- Intel Avaton C2750 SOC boards are tempting, but expensive (~450€ shipped to Finland at the cheapest). The combination of the above CPU+motherboard is ~435€ and with support for VT-d in ESXi if needeed.
- I plan on selling the HP Microserver N54L once I have the new NAS up and running (to help fund the new build). Money doesn't grow on trees for me, unfortunately.
- If I only want one NAS in my apartment, which route should I go software-wise? I can source a ~100-200GB SSD for a ESXi datastore for VMs if needed.
- Is there some other OS that can do what I want to achieve, but more elegantly?
- I still have space on desktop hard drives to empty out the Microserver NAS if needed.
- Supermicro boards with IPMI would be nice, but cost 2x more.

I know 16GB might be pushing it in ESXi (plus a modded minecraft server running), but I'm not yet set on ESXi. This is a merely a starting point for me, since the CPU & motherboard supports 32GB ECC RAM. I'll be upgrading once I get the funds to do so.

I apologize for the long post and really appreciate if anyone of you have some input.

So here's a small thought. IDF is approaching.

 

[Solkku]

@Patrick If by "IDF" you are referring to the new Xeon-D SoC boards coming, they'll probably a bit out of my price range. I've seen the prices quoted from $800-$1200 on those boards, which is more than the whole server part list above.

@cheezehead You put forward a lot of good points. I've been using Plex on my N54L for a year now with some Full HD content viewed locally and remotely (though most/all of it h264/x264 encoded, not VC-1). Haven't had any hickups/stuttering with any of the movies or TV shows so far.

I really want this to be as future proof as possible (or at least as upgradeable as possible) for my budget at ~800€, so that I'm not looking for new server parts again next summer, hehe. Also, I don't see a problem with purchasing a compatible LSI SAS PCI-E card to pass through hard drives in ESXi, as long as it works. There are some cheap ones on eBay.

I think the motherboard I have listed above only has one SATA-controller, so I'd need an add-in SATA/SAS card anyway for the NAS storage drives and use an SSD (datastore for VMs in ESXi) with the onboard SATA controller, right?

The only thing I'm kinda worried about is the modded Minecraft server. Last winter when I was hosting a server on my desktop PC (i5-2500K + 16GB RAM) I noticed it can be quite the resource-hog. Though when I noticed this, I was playing on the machine the server ran on.

 

[Patrick]

If you are running Minecraft, you want high single threaded speeds, so you are probably looking at something like Skylake.

 

[Deslok]

If you are running Minecraft, you want high single threaded speeds, so you are probably looking at something like Skylake.

Agreed on that the hyper-v host i run it on(inside suse) has a 3770 and with enough people on it it can load up one core no problem. I typically run the servers in different vm instances so they're fully isolated and portable to new hardware with minimal efford(my old host was a W510 with a QM720) ESXI and Hyper-v as a platform with an isolated linux distro(pick your favorite mine happens to be suse) offer the same portability there.

 

[Solkku]

So the E3-1231V3 doesn't have good enough single-threaded performance for a Minecraft server?

Skylake means moving to DDR4, right? That means the part list becomes even more expensive because of current memory pricing.

 

[Deslok]

Actually that E3 does, an avoton board isn't as good on single thread, that e3 should run a minecraft server about as well as an i7-4770(a quick look on ark shows them being very similar, the e3 loses graphics and a 100mhz turbo but gains TSX-NI and ECC memory capabilities) use benchmarks for the 4770 to gauge it's performance where you can't find a benchmark of the e3

 

[RTM]

As you are relatively cost sensitive, have you considered buying hardware used?
You can find some pretty good deals on eBay on older hardware, if the seller uses the Global Shipping Program you will know in advance how much you have to pay with VAT and shipping.

 

[Patriot]

Actually that E3 does, an avoton board isn't as good on single thread, that e3 should run a minecraft server about as well as an i7-4770(a quick look on ark shows them being very similar, the e3 loses graphics and a 100mhz turbo but gains TSX-NI and ECC memory capabilities) use benchmarks for the 4770 to gauge it's performance where you can't find a benchmark of the e3

TSX is borked in Haswell...

 

[Deslok]

TSX is borked in Haswell...

good to know, so really all the e3 has is ECC memory left then?

 

[Patriot]

good to know, so really all the e3 has is ECC memory left then?

Pretty much... but if you get a broadwell chip you can get tsx...
Personally if you want TSX I would wait for skylake or go Xeon D.

 

[Solkku]

@RTM I thought about it, yes. The thing with old hardware is that it tends to use more power (especially CPUs) and thus becoming more expensive in the long haul (with an increased power bill every month).

@Patriot Thanks for pointing this out. I read an article about this that said it mostly affects "enterprise-related matters such as database handling, not really something that affects the wide majority of the mainstream consumer spectrum". Since I'm not going to be doing enterprise level database-handling, I should be ok, right?

 

[Patriot]

@RTM I thought about it, yes. The thing with old hardware is that it tends to use more power (especially CPUs) and thus becoming more expensive in the long haul (with an increased power bill every month).

@Patriot Thanks for pointing this out. I read an article about this that said it mostly affects "enterprise-related matters such as database handling, not really something that affects the wide majority of the mainstream consumer spectrum". Since I'm not going to be doing enterprise level database-handling, I should be ok, right?

Well... TSX helps in multi-threaded scenarios. But the software has to be coded for it, very little is due to the errata.

 

[Solkku]

Everything I plan to do with this build (so far) seems to be only single threaded... not counting on java becoming multi-threaded anytime soon anyway (for the Minecraft server).

 

[cheezehead]

@Patrick If by "IDF" you are referring to the new Xeon-D SoC boards coming, they'll probably a bit out of my price range. I've seen the prices quoted from $800-$1200 on those boards, which is more than the whole server part list above.

@cheezehead You put forward a lot of good points. I've been using Plex on my N54L for a year now with some Full HD content viewed locally and remotely (though most/all of it h264/x264 encoded, not VC-1). Haven't had any hickups/stuttering with any of the movies or TV shows so far.

I really want this to be as future proof as possible (or at least as upgradeable as possible) for my budget at ~800€, so that I'm not looking for new server parts again next summer, hehe. Also, I don't see a problem with purchasing a compatible LSI SAS PCI-E card to pass through hard drives in ESXi, as long as it works. There are some cheap ones on eBay.

I think the motherboard I have listed above only has one SATA-controller, so I'd need an add-in SATA/SAS card anyway for the NAS storage drives and use an SSD (datastore for VMs in ESXi) with the onboard SATA controller, right?

The only thing I'm kinda worried about is the modded Minecraft server. Last winter when I was hosting a server on my desktop PC (i5-2500K + 16GB RAM) I noticed it can be quite the resource-hog. Though when I noticed this, I was playing on the machine the server ran on.

There will be lower priced Xeon-D boards coming out soon.

The SAS card....check ebay, much cheaper. If your looking at raid controllers (which can be cheaper, lots out there) make sure you can run it in "IT Mode" or where it has the capability (or can be flashed to) function like it doesn't have raid (for the ZFS-based NAS distros)

 

[Solkku]

I've now managed to score the Node 804 case and an IBM ServeRAID M1015 (flashed to IT), used for cheap. I also found 4 x 8GB sticks of Samsung M393B1K70BH1-CH9Q1 DDR3-1333 ECC for sale at 110€, which is cheaper than 2 x 8GB DDR3-1600 sticks cost new, so couldn't resist.

Still contemplating on the motherboard though, there seems to be some very mixed reviews on Newegg about the Asus P9D-M. The Asus costs around 185€ here in Europe, but for around 220€ I can get a Supermicro X10SLM-F board.

Does anyone know which of these boards is "less likely to hickup" with the Samsung memory I bought? Of course neither had this memory in their respective memory compatibility list, but ofc that doesn't mean that it won't work...

 

[Deslok]

I might be inclined towards the Asus with it's onboard USB3 if you use usb attached storage for any form of backup, in regards to memory we have an ASUS P9A-I/C2550/SAS/4L and it hasn't been the least picky about memory.

 

[Solkku]

Thank you for your reply Deslok. I don't think I'll need any other USB 3.0 ports other than the Type A port on the board (for a thumb drive with OS). The Supermicro board has this, while the Asus' is USB 2.0 Type A.

I can always invest in one of these in case plans change, since both boards also have USB 3.0 headers onboard.

The IPMI on the Supermicro (missing out of the box on the Asus afaik) is really making me want to lean towards the Supermicro. But then again I've read that some people have had problems getting Kingston memory modules to work with it, that even were on the supported list. Choices choices... I suppose it's a crapshoot at this point.

 

[CreoleLakerFan]

Does anyone know which of these boards is "less likely to hickup" with the Samsung memory I bought? Of course neither had this memory in their respective memory compatibility list, but ofc that doesn't mean that it won't work...

Neither. Unfortunately you purchased Registered ECC DIMMs - those boards only work with unbuffered memory. At this point your options are to return or resell the memory, or go with an older Xeon platform.