Server Build (Xeon E3-1275v5)

[Eric Faden]

Hey All,

So I finally settled on the following.... (Goal is to server 2 to 3 people w/ transcoding, run a VM or two, and run Plex, CouchPotato, SickRage, etc)....

Fractal Designs R5
Seagate IronWolf 6TB x 3
Corsair HX850i
SuperMicro X11SAE-F
Crucial 16GBx2 DDR4 ECC 2133
MasterLiquid Pro 140
OCZ RD400 M.2 2280 512GB PCIe 3.0 x4 SSD
Xeon E3-1275v5

Any feedback? (Still not sure if I should go for the E5-2620v4 instead).

Any recommendations on where to start with OS? (ESXI, Xen, Ubuntu w/ Docker... etc(?

 

[BlueLineSwinger]

The PSU is ridiculously over-specced. Look for something in the 3-400 watt range.

Liquid cooling is uncalled for. Just use the CPU's included heatsink.

That mainboard is more of a workstation design, with features (e.g., audio, legacy PCI slots) that shouldn't be needed for a server. For a basic server I'd be looking to get something like the X11SSL-F or maybe the X11SSH-F instead. edit: Also consider the Xeon D-1500 (Broadwell-DE) boards. Potentially lots of low-power cores, but unfortunately not cheap.

Given a proper server mainboard that has its own GPU to drive the VGA port, you don't need to get a CPU with one integrated (hopefully you'll be going headless with IPMI regardless). Get a Xeon that ends in '0' instead of '5'. FWIW, you can get the 1230v5 for considerably cheaper than the 1270v5, and only lose 200 MHz.

Ditch the NVMe drive completely. If you need a SSD (say, for storing guest OSes) go with a SATA unit instead. It'll be far cheaper, and you'll not notice any difference in real-world performance.

The case is fine, but FWIW if you go with a mATX mainboard (such as one of the ones linked above), you can go smaller.

 

[Netwerkz101]

When you say you don't know if you should go for the E5-2620v4 instead, you do mean with an
alternate motherboard too, right?

I am trying out a similar build (X11SSH) with XenServer 7.
I previously ran XenServer 5.x - 6.x, but have to learn/maintain skills with ESXI.

I actually prefer XenServer in my home lab.

 

[Ivan Dimitrov]

Hi, I have just finished my home server rebuild with very similar configuration. I went for E3 1245 v5 for financial reasons. If you have the money go for the E5 platform event with slow CPU in the begging.
I would strongly recommend to go for the fastest NVMe drive you can afford. They are great for VM disk drives. Such drive can save you the need of configuring NFS or iSCSI servers.
I use ProxMox for hypervisor and can strongly recommend it. With the ZFS support the storage management is a breeze. I have my desktop and router running as VMs and the performance is OK.
The MB X11SAE is quite extensible. IOMMU group separation is sufficient, but for painless virtulization platform I will be saving money to jump to the big boys club with E5. Lets see where Intel will take the market in 2017.

 

[Eric Faden]

@BlueLineSwinger
Thanks for the info. I wasn't really sure if the NVMe/SSD would be worth it or not. I’ll look at those motherboards. I was actually hoping to get a mATX motherboard to get a smaller case, but was trying to get the M.2 Socket.

Re: Xeon D series… where do you even buy those? I can’t seem to find ANYONE that sells the things. They look nice though.

Re: PSU, I’ll look for a smaller one. It seemed like overkill and expensive. I was hoping to have a modular one.

Ideally I’d like to be able to support 8 or so SATA drives for capacity moving forward.

Re: SSDs. I was mostly planning to use the NVMe/SSD for the Main and Guest OSes.

@Netwerkz101
Lol… yep. With a different motherboard. I was debating between the E5-2620V4 and the E3-1275V5. Both with SuperMicro Motherboards with IPMI. I want IPMI since I’m planning to shove this thing in a corner in my house with no screen/keyboard/mouse etc.


@Ivan Dimitrov
Thanks for the info.

I was leaning towards XenServer or ProxMox…. There are just too many options. I used to run a lot of servers, but since my career change haven’t.

Thus far am considering XenServer, ProxMox, Ubuntu, FreeNAS….

 

[i386]

You can get xeon d mainboards from the same places where you get other server/workstation mainboards (amazon, newegg, ebay).

 

[zir_blazer]

If you're going for LGA 1151, I would pick the Xeon E3 1245v5 over the 1275v5. A 100 MHz bin bump are nowhere worth around 50 U$D, and there doesn't seem to be any other difference between those two models. ARK | Compare Intel® Products

You said you want to do transcoding. Do your applications use the Intel IGP QuickSync for that? If you want QuickSync, you can't go LGA 2011-3 (Alternatively, you will need a discrete Video Card to use the fixed encoding engines, or do it by CPU brute force). Nor I would pick any non-IGP Xeon E3, since these 20-30 U$D difference between non-IGP and IGP models are totally worth it (10-15% Processor cost to allow you to use pretty much half of the Processor die, which are rather useful. More so if you are going to use VMs, check out Intel GVT-g).
Just remember that Intel requires you to use a C236 Chipset in order to use Xeons E3-12x5v5 IGP, they don't work on C232, in case you want to switch to another Motherboard.

Finally, assumming you don't need QuickSync thus are free to go to LGA 2011-3, consider the Xeon E5 1620v4 (300 U$D) and a Supermicro X10SRA-F (320 U$D). They are in the same price range of your current LGA 1151 choices, and may be better. The E5 2620v4 with 8C/16T and 2-way support (That you probabily aren't going to use) looks rather good for just 130 U$D over the E5 1620v4, but I'm not sure how the lowish base Frequency could affect you. ARK | Compare Intel® Products
Just remember that the Supermicro X10SRA-F does NOT support Broadwell-E out of the box if you get one with an old BIOS, and may need a Haswell-E Processor to update the BIOS before getting a Broadwell-E to POST in it. Not sure if the BIOS chip is socketeable and you can workaround it with a Flash reprogrammer instead of having to purchase a 200-300 U$D Haswell-E Processor paperweight to get it working. Also, the X10SRA-F is Workstation oriented, there are other variants that doesn't have Audio, in case you want something pure Server.

 

[Eric Faden]

@zir_blazer

Interesting points. I know that Emby supports QuickSync, but Plex does not. I'm actually not opposed to adding a discrete graphics card for transcoding GPU ($150 or less for a GEForce)....

Good information on the Broadwell-E... I don't want to have to buy a processor just to upgrade the bios.... lol...

Which would be better though QuickSync w/ 1275 (or equivalent) or E5-2620 (or equivalent) with NVidia GeForce 1050 Ti for hardware transcoding....

I'm still thinking all of this is likely overkill and I can handle my needs with anything....

-Eric

 

[BlueLineSwinger]

Is it even possible to pass the Intel IGP through to a guest for QuickSync?

Also, has QuickSync gotten any better? Last I can recall, its quality was noticeably inferior to CPU-based transcoders.

Regardless, buying any IGP/GPU for Plex/etc. seems like overkill to me.

 

[zir_blazer]

QEMU can do IGD Passthrough since 2.6. I think that the iGVT-g solution also allowed for QuickSync usage, but it should have lower performance due to the mediated passthrough overhead. But yes, you should be able to get it working on a VM.
Not sure on quality. I recall AnandTech doing analysis of encoding quality during the SB/IB era, but that was three generations ago. Video Cards also had the same issues with their fixed video encoding engines, so if you wanted the highest possible quality you had to do it with CPU brute force.

Whatever the use is, spending 30 U$D more for a Xeon E3 with IGP is always useful if you're on a Motherboard that can use it (As C236 based ones do), since the IGP adds interesing potential utility. I would absolutely regret it more if I figured out later than the IGP would have been useful and I decided against it to save a few dollars.
Basically, for me is either C232 + Xeon E3 without IGP or C236 + Xeon E3 with IGP, regardless of IPMI or discrete Video Card.

 

[Eric Faden]

@BlueLineSwinger
I believe it is possible to passthrough.... as for quality... I don't know/

@zir_blazer
What about E5 without IGP vs E3 with IGP? .... If I go E3 I'm definitely getting the IGP....

My real dilemma right now is still debating between the E3 and the E5...

 

[Ivan Dimitrov]

@BlueLineSwinger
I believe it is possible to passthrough.... as for quality... I don't know/

@zir_blazer
What about E5 without IGP vs E3 with IGP? .... If I go E3 I'm definitely getting the IGP....

My real dilemma right now is still debating between the E3 and the E5...

GO with the E5. The platform will be good for years to come. If you have such doubts now, you will have regrets after a few months. Consider the RAM. Migrating between E3 and E5 is not that easy. E5 needs RDIMM and the E3 UDIMM memory. Most likely E5 will work with UDIMM as well but such combinations are rarely on the MB compatibility lists.
Next year Intel will change the socket and I guess the new MB will be quite expensive.