Help me find: Bad ass VM server

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
I'm thinking it might be a good idea for me to buy a pretty serious VM server to put in our existing hosting service so I can play with it. This will be my money not company money. I'm not cheap, I want something pretty awesome but money is a factor. I'm looking to run either VMWare or Proxmox... haven't decided yet. But I want either a single or dual node server that can really run a ton of VM's... help me find something used that is great bang for the buck.... something with lots of CPU, RAM and a mix of mass storage and fast storage for different cases.....

I've been out of the hardware game for a while so most of the new stuff is beyond me, so looking for you guys who deal with it all the time to help me find a killer deal and tell me why it's a great buy!
 
P

Peanuthead

Active Member
Jun 12, 2015
839
177
43
44
What kind of iops are you thinking, storage space, cores needed, etc? Your requirements have too much ambiguity. What is bad to the bone for one is a file server for another
 
D

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
If you're looking for a one-off, the Dell Outlet is REALLY hard to beat. I buy from there for single case. IE:

Just bought an R630 (24 bay 1.8" case), Dual 2643v4's, 128G RAM, and 10x960GB SSD's 5yr onsite warranty for around $7k.

but what's a TON of VM's? I'm sure we can help spec out a server.
 
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
I don't really know what I want to be honest... I haven't measured stuff in IOPS in forever... Most of our DB servers I base it on 10k or 15k RPM (single drive) type stuff. I'd say, looking for RAID1 on the mass storage in the size of at least 20TB and maybe a RAID0 SSD or NVMe type thing. 256 to 512GB RAM and maybe 32 to 64 cores?
 
P

Peanuthead

Active Member
Jun 12, 2015
839
177
43
44
If you measuring IOPS from a single disk then perhaps you really don’t need that many cores. I have a dual E5 V4 system sitting here that I’ve never used because of what I have already handles more than enough what I need. I find myself running out of ram long before I’m out of Storage or processor. In fact I don’t think I’ve ever ran out of processor.
 
Last edited:
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
StammesOpfer said:
Also a little more specific on the budget side. $1k, $3k, $15k?
Click to expand...
I'd like to keep it between 5 and 8k.... and I don't need new stuff, I'm fine with stuff that's a couple gens old but still packs a good power to performance/price punch.

Also, IPMI equivalent is a must.

Used is OK, felt I should specify. Reputable seller kind of required though....
 
D

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
OK I can definitively say you're not going to get 32 cores, 20TB and 512GB RAM for 5-8k.

If you're spending that kind of money, you really should be doing some performance gathering beforehand. What kind of workload? read, write? mixed? What's AVG / CPU load, how many VM's, etc.

Shooting blindly is just going to be a big disappointment. ie. Spending money on CPU's when you should be spending it on RAM or SSD's. And do you really need 20TB of fast, or can you split it into fast/slow, or do you need flash cached slow storage?
 
S

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
Last edited:
S

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
Last edited:
D

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
StammesOpfer said:
I could piece together a 24c/48t dual Xeon v2 w/384GB ram and Spinning storage with a big flash cache and some dedicated flash well within that budget. I am sure you could get into some newer CPUs too but I don't know the pricing on V3+ and ddr4 is expensive.
Click to expand...
Sure, an R720xd, C240. But can you in good conscience recommend putting potentially 5+ year old hardware in? No possible warranty, etc. He'd be better off going to one of the refurbishers and buying something with warranty from them.

Maybe if he were running 2 systems HC, but then you have to factor in the cost of that too....
 
S

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
dwright1542 said:
Sure, an R720xd, C240. But can you in good conscience recommend putting potentially 5+ year old hardware in? No possible warranty, etc. He'd be better off going to one of the refurbishers and buying something with warranty from them.

Maybe if he were running 2 systems HC, but then you have to factor in the cost of that too....
Click to expand...

For personal use to play with absolutely. For production use/critical data no way. But I would think about running 2 or 3 of them with Failover/HA instead of paying full price for new.
 
A

AJXCR

Active Member
Jan 20, 2017
565
96
28
35
I can help you configure and sell you a server that would meet or exceed your requirements if you'll work with me a bit to determine exactly how you want it spec'd. I have more equipment laying around a the moment than I know what to do with.

I pulled a new Supermicro SYS-1028UX out of the box last night which includes:
  • 10x SAS3 2.5" hotswapable back plane with caddies
  • SAS3 LSI HBA
  • Chelsio T580 NIC (or Solarflare 10G if you prefer)
  • X10DRU-XLL motherboard (ultra series)
    • Integrated hyper speed
    • 160W TDP/socket support
    • Redundant platinum power supplies
    • Integrated IPMI 2.0 and KVM with Dedicated LAN
  • We can configure the drives, mem, and processors based on how much you want to spend. For example purposes, however:
Where are you located?
 
I

i386

Well-Known Member
Mar 18, 2016
4,250
1,548
113
34
Germany
If it's just for learning/playing than I would build two systems, one storage node and one vm host.

Storage node with 100gbe/edr infiniband + optane ssd(s) for slog (900p or p4800x), max 128gb ddr3/4
Vm host with 100gbe/edr infiniband and two cpu (8+ cores & 2+ ghz each)

Put everthing that you're not spending on hdds/ssds (capacity) in ram for the vm host.
 
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
i386 said:
If it's just for learning/playing than I would build two systems, one storage node and one vm host.

Storage node with 100gbe/edr infiniband + optane ssd(s) for slog (900p or p4800x), max 128gb ddr3/4
Vm host with 100gbe/edr infiniband and two cpu (8+ cores & 2+ ghz each)

Put everthing that you're not spending on hdds/ssds (capacity) in ram for the vm host.
Click to expand...
This would be a good idea.. and when I said 20TB of storage, that was slow storage... for fast stuff was thinking ~4TB to answer someone else's question.
 
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
AJXCR said:
I can help you configure and sell you a server that would meet or exceed your requirements if you'll work with me a bit to determine exactly how you want it spec'd. I have more equipment laying around a the moment than I know what to do with.

I pulled a new Supermicro SYS-1028UX out of the box last night which includes:
  • 10x SAS3 2.5" hotswapable back plane with caddies
  • SAS3 LSI HBA
  • Chelsio T580 NIC (or Solarflare 10G if you prefer)
  • X10DRU-XLL motherboard (ultra series)
    • Integrated hyper speed
    • 160W TDP/socket support
    • Redundant platinum power supplies
    • Integrated IPMI 2.0 and KVM with Dedicated LAN
  • We can configure the drives, mem, and processors based on how much you want to spend. For example purposes, however:
Where are you located?
Click to expand...
Austin, TX... but I'll be installing this in Kansas City.

Assuming I got it built kind of the way you suggested above but without any drives... what price range are we at?
 
B

BigDaddy

Member
Aug 8, 2016
38
20
8
Wish you lived closer to me lol. I have a dual e5-2686 v3 (36 cores) 128gb ddr4 machine with 1tb of nvme storage that'd I've been debating selling. Board even supports bios turbo mod for an all core of 2.8-2.9. Do you ever end up around NC?
 
A

AJXCR

Active Member
Jan 20, 2017
565
96
28
35
El Chorizo said:
Austin, TX... but I'll be installing this in Kansas City.

Assuming I got it built kind of the way you suggested above but without any drives... what price range are we at?
Click to expand...

Austin? ..That's awesome! I graduated with an engineering degree from UT and now live right down the street in Houston.

Just to clarify, regarding the buying and selling of computer/server related gear, I am an individual and it's a hobby. I tend to cycle through a lot of gear, but that is more an attempt to make the hobby self supporting rather than a money pit.

Concerning price- You asked about the machine less drives. In my mind, the availability of these drives are part of what creates the opportunity to get a machine of this caliber well below a price point I think you'll find elsewhere.

A couple of points:
  • The components listed above were merely for example purposes. We could easily go with a different chassis, processors, memory, drives, NIC's, HBA's, etc.
  • An earlier poster asked a few specific questions about your intended use case to which I never saw a direct response.
    • How many VM's do you intend to run?
    • What will they be used for?
    • What else might the machine be used for?
    • What guest operating systems will be used?
    • Are you sure you really need that much power/hardware?
  • Do you have a rack?
Concerning price, I'd be more than happy to work with you to tailor the final machine to your desired specs and price point. I haven't sat down and really crunched any numbers, but off the top of my head, if you wanted exactly the machine described above with (5x) 1.92TB SAS3 SSD's, I'd probably be willing to let it go fully assembled in the original packaging (w/rails, cables, etc) for $9500 ish?

-Craig
 
A

AJXCR

Active Member
Jan 20, 2017
565
96
28
35
El Chorizo said:
This would be a good idea.. and when I said 20TB of storage, that was slow storage... for fast stuff was thinking ~4TB to answer someone else's question.
Click to expand...
If you wanted to go a route more like what you've described here, we could do something more like:
  • 4U Supermicro 846/7 SAS3 chassis w/24x hotswappable front 3.5" drive bays (can be provided with regular 3.5" caddies, 3.5" to 2.5" converter caddies, or a combination of both). Can also add a second rear backplane to provide up to an additional 24x SAS3 2.5" slots or 12x 3.5" slots.
  • 2x 920W Silent/Redundant power supplies
  • X10DAC board w/built in LSI SAS3 3008 controller- Note that this board does not have ipmi, but there are other MB options here.
  • (4x or more) 6TB HGST HE8's
  • Optane/P3700 NVMe drive for caching
  • P3700/Intel 750 1.2TB for sys drive
  • Your choice of processors
  • T580 (40G or 4x 10G)/Solarflare 2x 10G

OR

2U Intel chassis with up to 24x front 2.5" hotswappable bays and your choice of configurations.. this one has a built in Intel 2x 10G add on card

Idk.. Give it some thought and feel free to give me a call or come buy and take a look. Happy to discuss/answer questions/let you test out hw configs/or source parts.

-Craig
 
E

El Chorizo

New Member
Jul 29, 2017
20
3
3
45
El Chorizo said:
This would be a good idea.. and when I said 20TB of storage, that was slow storage... for fast stuff was thinking ~4TB to answer someone else's question.
Click to expand...
The more I'm thinking the more I like this idea... because I can add an adapter to existing servers and get storage for those too all on the storage host. Nothing I've done before, so I'm a newb at it but I like the idea of this.
 
You must log in or register to reply here.
Top Bottom