New ESXi server build - been awhile

[thecoffeeguy]

Hey Folks
I need to upgrade/refresh one of my ESXi servers. Current one is old, loud and noisy old SCSI drives.

Couple of things I am looking for:

Quiet and cool
Ability to support 128gb memory
(2) CPU's supported, may start with one for now.
4+ NIC's
2-4- 2.5 drives for internal storage

In the past, I have gone with SuperMicro and like their IPMI. Was thinking the same unless there issomethingn better different?

The case is the thing i have been thinking a lot about.
I dont need a GPU which should open up options for me.
I looked at some fractal designs and they have nice stuff.

Any combo deals that folks recommend?

Maybe something to just point me in the right direction.
Is memory still pricey these days? Been awhile since i've built/refreshed a rig.

Cheers and thank you.

TCG

 

[marcoi]

A price range will help narrow down the suggestions. IE budget of 1000.

Off the top - Many like systems based on e-5-2600 v2 CPU since they use ddr3 memory and prices for older memory aren't insane. DDR4 is top dollar right for example.

I think for 1k you could get a dual e5-2680 v2 system with 128GB ram. might need to shop around -ebay, CL, on FS threads here. etc..

 

[thecoffeeguy]

A price range will help narrow down the suggestions. IE budget of 1000.

Off the top - Many like systems based on e-5-2600 v2 CPU since they use ddr3 memory and prices for older memory aren't insane. DDR4 is top dollar right for example.

I think for 1k you could get a dual e5-2680 v2 system with 128GB ram. might need to shop around -ebay, CL, on FS threads here. etc..

yea, forgot to put my budget in. it is about $1k, depending.

I was looking for the same thing as far as memory because one of my current ESXi host (newer one) DDR memory is very expensive.

any suggestions on a case?
i should also mention i have a IKEA Lack rack setup that has my patch panel and 24 port switch mounted to it. My current ESXi hosts sit on top of it. I try to keep everything 'clean' best i can, hence why the case is osmething i have spent a lot of time on.

EDIT: One thing i was curious and looking for help on, when looking for mobo, what is the best way to filter? I a looking at supermicro boards on newegg and I just cant seem to narrow down the search. I cant seem to find a board that is NOT DDR4 288. Kinda driving me nuts..ahaha

Much appreciated!

 

[marcoi]

as for case, depends on if you want a server case with loud fans or more quiet pc case where you put in your own fans, etc. IF noise isnt an issue then a server case would probably work out best.

As for the parts, they would be used items as they are already a few generations old from a tech perspective. But work great for home lab use.
Alternatively you can also scope out some old dells r720 on ebay like this one
Dell R720 16-Core Server 2x E5-2690 2.9GHz 128GB-8 2.5in H710 iDRAC RPS | eBay

I have an dell r720 and love it.

 

[thecoffeeguy]

as for case, depends on if you want a server case with loud fans or more quiet pc case where you put in your own fans, etc. IF noise isnt an issue then a server case would probably work out best.

As for the parts, they would be used items as they are already a few generations old from a tech perspective. But work great for home lab use.
Alternatively you can also scope out some old dells r720 on ebay like this one
Dell R720 16-Core Server 2x E5-2690 2.9GHz 128GB-8 2.5in H710 iDRAC RPS | eBay

I have an dell r720 and love it.

Yea, as much asI would like a r720, the size and noise may get the locks changed on me.

What about mobos? I wa sthinking i could start there. ATX or micro ATX.

Any supermicro boards I should 'stear clear of'?

Thx

 

[marcoi]

Dual CPU you probably be in the ATX size unless you get some custom made motherboards that is meant to run in a specific case.

Many of us on the board ended up with a flavor of the intel boards s2600cp. Below is my build out. Which is fairly quiet expect on start up.
https://forums.servethehome.com/index.php?threads/intel-s2600cp-w-intel-p4000-case.9104/

Here is example of someone selling that board in a pc gaming case.
https://forums.servethehome.com/ind...on-e5-2670-128-ddr3-workstation-server.21849/

Just search around and when you get a list of boards/cpu/memory your interested in, post back to have it vetted out.

 

[thecoffeeguy]

Very coo. Will do that.
In the meantime, I came across an exisiting case I can use. It is a micro ATX board setup, but it is nice:

Fractal Design Define Mini Black Micro ATX Silent PC Computer Case w/ USB 3.0 support and 2 x 120mm Fractal Design Silent Fans - Newegg.com

Here are some boards I quickly looked at:

Computer Parts, PC Components, Laptop Computers, LED LCD TV, Digital Cameras and more - Newegg.com

My concern is the memory. Its all DDR4 which can be pricey.

This one looks promising:
SUPERMICRO MBD-X11SSH-F-O Micro ATX Server Motherboard LGA 1151 Intel C236 - Newegg.com

Can drop in a Xeon CPU, add a 4 Port NIC with some local drives...

 

[thecoffeeguy]

what about the above SuperMicro Mobo paired with this CPU:

Intel Xeon E3-1230 V6 Kaby Lake 3.5 GHz (3.9 GHz Turbo) LGA 1151 72W BX80677E31230V6 Server Processor - Newegg.com

would go in the case above, then i need to find memory.

thoughts?

 

[Rand__]

You are looking for dual CPU setup but list only current 1P boards

If you want 2P + Ram then you'll need older tech to stay in budget, if you want newer gen then you'll have to sacrifice something else...

 

[thecoffeeguy]

You are looking for dual CPU setup but list only current 1P boards

If you want 2P + Ram then you'll need older tech to stay in budget, if you want newer gen then you'll have to sacrifice something else...

yea, i should have clarified. 2 Proc would be nice, but not required. Looking to maximize what i can with my budget. I need to build a NAS box as well...heheh..so many toys!

 

[TLN]

I have ASUS tower, that comes with Z10PE mobo (Dual Xeon E5-v3/v4, DDR4) PSU and everything. Can be installed in the rack. Get CPU and memory and you're good to go. Comes with CPU Coolers and I'll add M.2 SSD to use as boot drive.

 

[thecoffeeguy]

One thing I was curious, the boards I a looking at all seem to be DDR 288.
64gb total, is 4 sticks and the sticks seem to be about 100-200 per stick?

that sound right? or am i way off

 

[Rand__]

288 pins you mean?
Else it would be DDR4-2133 or 2400, which can be around that price range for a 16GB stick.
Thats why the general recommendation was to go back one generation to use DDR3 memory if you need/want loads of it

 

[thecoffeeguy]

288 pins you mean?
Else it would be DDR4-2133 or 2400, which can be around that price range for a 16GB stick.
Thats why the general recommendation was to go back one generation to use DDR3 memory if you need/want loads of it

Thanks, yea. Wanted to confirm.
Now the task to find the motherboard with that memory.

Thank you

EDIT: One thing I wanted to ask is the proper way to match the memory to the mobo. I recall reading in some threads that not getting memory that has been officially tested by SuperMicro could be bad?
Just want to make sure before I start buying stuff.
Thx

 

[Rand__]

Usually there is a tested module list at the board page.
I have had mixed results with SM boards, sometimes even modules working fine in one board don't work in the other, but sometimes the boards take everything I throw at them without a hickup, so its a mixed bag in my experience.

 

[nk215]

You want an option to use 1 CPU on a dual CPU board + lots of cheap memory. The best option (if you can tolerate the size) is a Supermicro X9DR3-LN4F board. It has 12 memory slots per CPU so you can use the cheap 8gig stick and can get 96gb setup starting out with one CPU configuration. It also has 4 NICs that you wanted

I have this board and really like it. A top dual e5-269x DDR3 setup is significantly cheaper than a DDR4 setup with similar performance.

 

[thecoffeeguy]

You want an option to use 1 CPU on a dual CPU board + lots of cheap memory. The best option (if you can tolerate the size) is a Supermicro X9DR3-LN4F board. It has 12 memory slots per CPU so you can use the cheap 8gig stick and can get 96gb setup starting out with one CPU configuration. It also has 4 NICs that you wanted

I have this board and really like it. A top dual e5-269x DDR3 setup is significantly cheaper than a DDR4 setup with similar performance.

Thanks. I appreciate the feedback.
doing some research this weekend, I think that my options are to go with DDR4, 288 DIMMS. I just dont really see a lot of options. My setup would probably start wwith 64gb memory, but eventually get to 128gb.

i also started to think that I would prefer 3-4 local SSD drives attached to it, just for the local storage piece.


Think i am going to look for another case with more drive bays that i can mount drives in, just in case.

so now, the big question is, what mobo and CPU?

 

[thecoffeeguy]

Alright, herre is what I have come up with.
Feel free to tell me if its bad/good or needs rework:

case:
Fractal Design Define R6 USB-C Blackout Brushed Aluminum/Steel ATX Silent Modular Mid Tower Computer Case - Newegg.com

I like fractal and it has lots of bays inside for SSD drives, local storage for the server:

mobo:
SUPERMICRO MBD-X11SSH-F-O Micro ATX Server Motherboard LGA 1151 Intel C236 - Newegg.com

CPU:
Intel Xeon E3-1220 V6 Kaby Lake 3.0 GHz (3.5 GHz Turbo) LGA 1151 72W BX80677E31220V6 Server Processor - Newegg.com

Still matching memory.

yea? na?

Thanks folks!

 

[Rand__]

Couple of things I am looking for:
Quiet and cool
Ability to support 128gb memory
(2) CPU's supported, may start with one for now.
4+ NIC's
2-4- 2.5 drives for internal storage
TCG

Lol, funny how pricing reality takes everbody to the bottom, going from top original list to a X11SSH/E2-1220 (No worries, everybody has been there).

So, that is a cheap solid basis you picked but don't expect it to run loops around anything. I have 3 similar setups for my test cluster (just to get the lights on) and they are ok-ish.

Now it totally depends on what the box is going to be used for?
4 SSDs worth of local storage might be overkill for a 4 Core/4 Threads/64 GB Box... you will run into CPU issues after just a few VMs. Maybe better to at least upgrade to the 1230 which has 8 threads.

 

[thecoffeeguy]

Lol, funny how pricing reality takes everbody to the bottom, going from top original list to a X11SSH/E2-1220 (No worries, everybody has been there).

So, that is a cheap solid basis you picked but don't expect it to run loops around anything. I have 3 similar setups for my test cluster (just to get the lights on) and they are ok-ish.

Now it totally depends on what the box is going to be used for?
4 SSDs worth of local storage might be overkill for a 4 Core/4 Threads/64 GB Box... you will run into CPU issues after just a few VMs. Maybe better to at least upgrade to the 1230 which has 8 threads.

Yep. Reality brought me crashing back down.

would you recommend anything different, based off of your experience with your setup, besides the CPU? The idea for the local storage was just in case i need it (I have plans to build a FreeNAS box someday..heheh)

much appreciate your help

EDIT: I swapped out that CPU for the 1230...the extra 4 threads should be helpful.