Yet another build / lab

[nask]

Hello everyone,

First topic, almost first post, glad to be a part of this forum. I was looking for a forum like this one for months.
Behind this not that much attracting title, there is a whole lab design questions.

I suppose you won't mind if I share some context, I'm a Network/Sys. Engineer and working on my own for some months. I always have stuff at home, I had a whole homelab that I sold when I move out, now I'm back to home, I'm planning on building a new lab. I should mention I'm living in EU (France).
As I'm working on my own, this lab is gonna be my daily playground which implies some projects I have to keep working, try, test build, etc. for me or anybody.
Working on:
vRealize / VSAN / NSX / Openstack / Hyper-V / Exchange etc. on a daily basis.

I already have :
Main PC, i7 4770S, 32 GB DDR3, SSD (850 500GB), 2x 2TB (RAID1) WD.
Dell R510, 4x hard cabled SATA drives, 2x X5650, 32 GB DDR3.
DL380G5 ... I won't mention details because it's way too old, I'm gonna sell it/give it for free.

Networking:
2x 2960G
2x 3750G
2x Aironet 1142N

I was thinking about :
Getting 2 or 3 servers so that I could have plenty of space / resources for all my current and future projects.
Thing is there are so many possibilities, kinda hard to really understand which one would be the best.
I definitely should get a rack because it's getting hard to keep desk clean, probably a 24U would be enough.

I estimated I would need at least 40 vCPU and 90 GB RAM for all my current projects and future projects.
Disk space is estimated to 2 - 4 TB max. I don't use my vRealize lab or some others VM on 24/7.

Configuration with 3 nodes:
Minimum:
Single CPU - 6 cores
30 GB RAM
200 GB SSD (cache)
2 TB HDD (capacity)

If I can have a low TDP / low consumption / quiet server, I could have everything running on 24/7.
Most important is define system / version / CPU, I was thinking about a v1/v2 system with some E5-2630L (v1/v2).
Regarding to money, I could spend up to 500/600 EUR per server, $568 - 682.

My first thoughts was on trying to get a quote from a Broker, UK seems to be perfect place in EU to get some interesting deals.
Then I found this forum, found some Xeon-D/ATOM C2758 Supermicro SoC CPU based server, then some others Supermicro Twin² and even Windmill 2 Node Chassis.
Most of the time it's cheaper to import from USA than buy in EU, and my biggest problem is there are so many options, I don't know on which platform I should go for.

What do you think about this whole lab, go ahead with any comments, questions, anything

 

[Sergio]

First of all, everything depends on the budget, and for that you can get only a Xeon-D mobo with 4-6 cores. If consumption/quiet is one of your concerns, then you should take a look to that platform. I really like the low power consumption (35-45w), the IPMI is a huge win, you can add up to 128G of RAM... that's what I like form the Xeon-D. The most important thing for me is how small is the box, if I want to have 3 nodes it shouldn't be that big, not to say noisy. A build with the Xeon-D 1528, 6cores and 64Gb ram and the Supermicro server case costs around 1300e in Europe.

At the end you have to give up on something. I think for a home lab, Xeon-D has a bunch of great features, unfortunately does come with a price, but you could buy 2 nodes (that's what I'm planning to do) and later build the last one. What websites are you looking at for the hardware?

 

[Paul Bommel]

Hello everyone,
Then I found this forum, found some Xeon-D/ATOM C2758 Supermicro SoC CPU based server, then some others Supermicro Twin² and even Windmill 2 Node Chassis.

If you've virtualisation in mind and you'll go the Atom line, I'd recommend C2750 over C2758, because of the turbo boost.

 

[nask]

First of all, everything depends on the budget, and for that you can get only a Xeon-D mobo with 4-6 cores. If consumption/quiet is one of your concerns, then you should take a look to that platform. I really like the low power consumption (35-45w), the IPMI is a huge win, you can add up to 128G of RAM... that's what I like form the Xeon-D. The most important thing for me is how small is the box, if I want to have 3 nodes it shouldn't be that big, not to say noisy. A build with the Xeon-D 1528, 6cores and 64Gb ram and the Supermicro server case costs around 1300e in Europe.

At the end you have to give up on something. I think for a home lab, Xeon-D has a bunch of great features, unfortunately does come with a price, but you could buy 2 nodes (that's what I'm planning to do) and later build the last one. What websites are you looking at for the hardware?

For Supermicro hardware ? In Europe, I have no idea. I'm still looking for a way to get decent prices.
If you're in EU and you plan to get some Xeon-D, we might have a better chance to group our orders.

If you've virtualisation in mind and you'll go the Atom line, I'd recommend C2750 over C2758, because of the turbo boost.

Atom line seems really cool but what about performances ?
I don't have to handle an important load but platform itself isn't dedicated to low consumption, just vRealize / NSX is an important load which could be already too important for an Atom based platform. I'll look for some performance threads / resources, if you have any, I'm in.

 

[Paul Bommel]

I'll look for some performance threads / resources, if you have any, I'm in.

Here you'll find some benchmarks from patrick about the Avoton-/Rangeley line.

As you may see, the E3 v3 xeons have better results.
Just to keep in mind Rangley is for telecommunication (router, firewall etc) and Avoton for light loads.
If the performance is to low, you should go with Xeons (socked or system on chip), but for a lab/low power setup they are great.

 

[nask]

Seems Xeon-D is definitely the best solution
Now it's time to build the best value/performance setup, for 2-3 nodes.

Edit: As soon as I have an precise idea of 2-3 nodes cost / performances etc. I definitely would like to compare it with E5-26XXL v1/v2/v3 setup.

 

[maze]

The Avoton boards are great.. but no VT-d which kinda sucks if you want to do a little more on it in regards to vmware.
I would for sure go for an Xeon-D board with the extra power, nice amount of ram and the 10g options available. Asrock rack and supermicro both have some nice models available.

Im saving up for an Xeon-D board (gotta love buying a house and blowing all my savings.. right? ).
+ Issues with customs here in dk is shitty.. we get a lot added if we go above ~$11 USD in value

 

[nask]

Got back on this project, having some issue with a retailer, I'm thinking about a new solution, Xeon-D might be too new / expensive, might give a try on a Xeon E5 v4 platform, still don't know which one would fit the best.
Supermicro Server ? Twin ? or Dell R430/630/730 ...