Home server update: LGA 2011-3 + Xeon E5 V3/V4

[tubs-ffm]

Hello,


is nowadays a LGA 20311-3 with Xeon E5 V3/V4 still a good bang for the bug to upgrade a home server? Or is it blast from the past?
Can I expect bigger changes in power efficiency or performance for newer platforms? Are there better ways to go today?

For close to a decade, I am running a E3-1230 system with 32 GB RAM on Supermicro X10SL7-F µATX board as home server. File server, backup server and a couple of VMs for all kind of required, helpful and play around staff. It was a good compromise in power consumption, size ("WAF") and performance. What I want to achieve is to go beyond the 32 GB RAM limit (64 GB to 128 GB) and I want more CPU cores by keeping power consumption increase low at the same time. ATX size is fine.

I haven't followed up on the market for years. So I completely lost track on it after Broadwell. I also feel that variety on CPU variants has increased.

Older dual CPU system like Xeon E5-2630 V3/V4 or a newer one CPU system?
Any recommendation for the direction?
I am fine with AMD as well, but I am more familiar with Intel naming.

 

[T_Minus]

If you only need 64-128GB and want to keep power consumption low and have minimal other requirements then have you looked into the Intel E-22xx line up or the AMD 5600 (299$)?

The E5 v3\v4 is still viable and among my favorite bang for buck for what you get... but if you don't need loads of PCIE etc... then there's other options I'd go with


The AMD 5600 will blow away the Intel E5-2630 v3 for processing power

 

[tubs-ffm]

If you only need 64-128GB and want to keep power consumption low and have minimal other requirements then have you looked into the Intel E-22xx line up or the AMD 5600 (299$)?

Thank' you a lot. These are the information that helps me for do a "research".
I am fine with AMD, but not familiar with the products. This helps.

For me the questions are:

• Good value for the performance.
• Good price and availability for used staff.

Enterprise staff nobody wants quite often is cheep and good. Staff that is promoted for "Gamers" is overpriced.

 

[T_Minus]

AMD Discounted the current line up as the new stuff is coming, but you will for sure pay more for UDIMM and current\last gen than the previous E5 v3\v4 but you also get something with much lower power and better performance if you're after that performance.

If you can pay a bit more in electric then it's really hard to pass up the value of the E5 v3\v4 lineup especially with affordable RAM right now too

 

[Stephan]

A 2011-3 is really great for a server because of its stability. Most bugs in chipset and CPU have been worked around in firmware and kernel by now. Also imho the preferable because cheap platform if you want out-of-band management with an Aspeed 2400, which AMD stuff for the most part will not have.

 

[T_Minus]

A 2011-3 is really great for a server because of its stability. Most bugs in chipset and CPU have been worked around in firmware and kernel by now. Also imho the preferable because cheap platform if you want out-of-band management with an Aspeed 2400, which AMD stuff for the most part will not have.

But you can get IPMI and ECC support if you want... AsRock Rack X470D4U

 

[Stephan]

Yes I know this model of board, that's why I wrote "for the most part". ;-)

Looking for a new 2-3-node solution myself, am contemplating 1-socket C621 3647, C246 1151v2, C612 2011-3, Xeon D. Best would be mini ITX, with Aspeed, with m.2 PCIe, with one x8 or x16 PCIe slot, with regular full-channel ECC RDIMM support.

While I trust that X470D4U is compatible with ECC UDIMMs, I have my doubts if ECC functionality is fully functional. Intel here is so much better compared to AMD AM4 consumer chips, and you can be certain with Intel that the EDAC driver will do its job and report trouble to user space properly. But that's just me, I like to have good ECC everywhere.

 

[tubs-ffm]

After having done some research, I see that for used staff there are plenty of offers around 2011-3 and boards from "server brands". I never put my hands on IT staff that is tagged with "gamer" and always used boards build for server and workstations. Looking back the last 15 years no stability issue at all with my current server and workstation and boards from Supermicro.

Build-in SPF+ would be nice, but otherwise I use my current PCIe card and only need to keep the required slot in mind.

Most of the time my homeserver will idle around. Taking broadwell as an example, can I expect only a small difference in power consumption between the different models, e. g. comparing E5-2680 V4 (14 cores, 2.4 GHz, 120 W TDP) to E-2650L V4 (14 cores, 1,7 GHz, 65 W TDP)?

 

[T_Minus]

Yes I know this model of board, that's why I wrote "for the most part". ;-)

Looking for a new 2-3-node solution myself, am contemplating 1-socket C621 3647, C246 1151v2, C612 2011-3, Xeon D. Best would be mini ITX, with Aspeed, with m.2 PCIe, with one x8 or x16 PCIe slot, with regular full-channel ECC RDIMM support.

While I trust that X470D4U is compatible with ECC UDIMMs, I have my doubts if ECC functionality is fully functional. Intel here is so much better compared to AMD AM4 consumer chips, and you can be certain with Intel that the EDAC driver will do its job and report trouble to user space properly. But that's just me, I like to have good ECC everywhere.

"For the most part" is irrelevant if you want ECC and want IPMI.. then you would buy the board that has that, so who cares if "for the most part" a CPU designed for desktop has no ECC support or IPMI, well of course not...get the board that does... that was never the point though... but there are options to get that... if it's what you want. What it does not mean is that you can't get it.

We're talking about an affordable home server and you're talking about making sure the bugs are entirely worked out, and guaranteeing ECC support... neither of which will be the end of world for a home server. The AMD will offer better performance and lower power but a bit more up front cost. It is a very viable option to entirely write it off... no.


I think the argument is a bit ridiculous as to why not to use the AMD.

To the person building I personally don't care either way, just that they realize both are viable options and entirely acceptable.

With all that said... my home server is an E5-2670 v3
Just because that's what I had from a couple years back when I went to E3-1270 V3 and now back to the E5 for more options

If I had a spare AMD sitting around I'd throw that in though too w\out concern of ECC or other random, very rare issues that all CPUs, motherboards, etc, face.

 

[tubs-ffm]

"For the most part" is irrelevant if you want ECC and want IPMI.. then you would buy the board that has that,

We're talking about an affordable home server and you're talking about making sure the bugs are entirely worked out, and guaranteeing ECC support... neither of which will be the end of world for a home server.

For my case, you described its purpose very well. But IPMI I need and ECC I want. ECC might not might not be needed, but is a good choice for a 24/7 device that also is hosting a file server and backup server. I do not want to miss the comfortable way to get remote access via IPMI when something is not working as expected and you need to move the server from its hidden place to a screen and keyboard.

The AMD will offer better performance and lower power but a bit more up front cost. It is a very viable option to entirely write it off... no.

As mentioned before, nothing against AMD. Unfortunately, I could not find one with some more cores. EPYC is quite expensive in comparison to E5. No need for high single core performance. A E5-2680 V4 or E5-2650L V4 would give me 14 cores.

What would be something comparable from AMD?

To the person building I personally don't care either way, just that they realize both are viable options and entirely acceptable.

Good approach. That's why I am asking in a forum, to share my idea and get other ideas.

 

[EasyRhino]

I mean... a AMD Ryzen 5950X has 16 cores and 32 threads, and that's still in their consumer line.

power consumption might actually be lower on a newer (better process) consumer board that doesn't have as many power hungry PCH's, IPMI controllers, etc.

I'm personally running a 2011-3 board. I like it, but I've recently become sensitive to electricity cost at home and it tempers my enthusiasm.

 

[tubs-ffm]

I mean... a AMD Ryzen 5950X has 16 cores and 32 threads, and that's still in their consumer line.

power consumption might actually be lower on a newer (better process) consumer board that doesn't have as many power hungry PCH's, IPMI controllers, etc.

Yes, a nice CPU. But what you put on the table is not Intel vs. AMD but old vs. new. The 5950X is from current line-up on ZEN 3 base and at much higher price. The E5 V3/V4 I started the discussion with is "blast from the past" but you get it used for only a little of money incl. board and RAM.

Energy cost also is a concern for me. And yes, this also is the point behind my original question: "Is it worth to go with an old platform?". How many years I have to run a modern platform to save the money on energy costs I have to invest in an up to date system?

 

[EasyRhino]

oh yeah sure, for electricity cost you need to decide, is the server idle more or under load more?

when idle, my Xeon board is kind of inefficient. The CPU idles at 25W and the board itself seems to have 20W of generic additional usage (I don't even have IPMI). Under load, the usage is probably same in watts, as a new CPU, but it's going to get a lot less work done.

But then the Xeon will have more intangible advantages like IPMI (probably), lots of PCIE lanes, more and cheaper memory support.

On my own homelab I'm actually conflicted. One one hand I have all kinds of hard drives and pcie adapters plugged in, but on the other hand the CPU is idle most of the time except for an occassional plex transcode. I could probably use an i3 no problem. I actually removed one CPU from my dual CPU board just to save on electricity costs.

 

[bayleyw]

I think the first thing you should decide on is whether you need more than 16 lanes of PCI-e or 256GB of RAM. If you do, Xeon V3/V4 is basically a no brainer, since your only other reasonable option would be Skylake-SP, a much more expensive platform with fairly marginal improvements over Broadwell (16 core Skylake is probably 20% faster than 16 core Broadwell, most of which comes from high clocks).

If you don't need either you can get a much faster system at a good price with either Ryzen (if you want to live on the edge), or Alder Lake/W680 (if you want an official server platform). You could get away with something like 4 x4 PCIe slots for a server but its tricky to find a board which will split the lanes that way (it may not even be possible), and for some peripherals you are forced to use the latest PCIe 4.0 parts to get the required bandwidth.

 

[sko]

I'm still running a Supermicro SYS-6029U-TR4T with 2x E5 2630L v3 and 128GB RAM as a home server. With currently 8 SAS disks, 4 SSDs and 2x NVMe (on AOC-SLG3-2M2) this system usually draws ~200W during the day while being reasonably quiet. Yes, it's audible, but not annoying even if the rack is directly behind me.
When running poudriere builds, power draw goes up to ~300W avg / ~350W peak with a little more noise, but I usually run those builds during the day when I'm at work - so I don't care.
(edit: I could run the system on a single PSU which would save ~15-20W, but then the single PSU will get hotter and slightly more audible)

So if you don't need every last percent in single thread performance, I still think a 2011-3 system makes a great home/soho server on a tight budget. Actual power draw completely depends on the actual CPU(s) used and the workloads you put on the system anyways, and the power efficiency isn't increasing as rapidly with those newer generations as it did ~10-15 years ago. So the differences here are marginal compared to the price difference. I.e.: you'd need to run the newer, pricier platform for a very long time until it would have amortized the price difference to the "less efficient" 2011-3 system.

If you don't need that much compute power, you might as well look at some of the Xeon-D15xx options (or the newer D21xx if on a bigger budget), which even have dual 10GBit on-chip.
The Atom C35xx/37xx might be also interesting if you want to go really low on power consumption, although they are still quite pricey and aren't available with 10GBit (at least not from the SoC AFAIK), so you might need to add a NIC which may equalize the lower power draw of the Atom vs Xeon-D.

 

[RolloZ170]

this system usually draws ~200W during the day while being reasonably quiet.

200/24h = 8 Watt idle ?

 

[Rand__]

200/24h = 8 Watt idle ?

Lol, sometimes you have a mean streak

 

[RolloZ170]

Lol, sometimes you have a mean streak

why ? 200W idle is to much thought.

 

[Rand__]

200W/h does not sound unreasonably high for the hardware in use

 

[RolloZ170]

200W/h does not sound unreasonably high for the hardware in use

but he wrote the following:

When running poudriere builds, power draw goes up to ~300W avg / ~350W peak with a little more noise