Hyper-Threading in ESXi

[Danic]

I'm planning on buying replacement parts for my Home ESXi server. I don't need anything powerful. But in anyone's experience does hyperthreading really help? Should I spend the extra $ for 4 core w/HT? I currently run 5 1vCPU VM's. Which is a almost perfect fit for current AMD X6 1090T. VM's mostly sit idle, but my NAS and game server VMs take a toll on overall system performance when in heavy use.

Can anyone share experiences with overloading (more virtual than physical cores) a 4 core xeon with and without hyperthreading?

Thanks

 

[Rhinox]

Can anyone share experiences with overloading (more virtual than physical cores) a 4 core xeon with and without hyperthreading?

More vCPUs than pCores is not overloading, just overcommiting (or overcommitment? not sure what is correct). And it is quite normal and frequent with ESXi, because very rarely all your VMs are loading all vCPUs to the max, at the same time. How much you can overcommit CPU depends on your scenario. The best way is to observe pCPU-load on your ESXi-server for some time (i.e. with esxtop) and slowly increase number of VMs.

My ESXi-server has 8 logical cores (single xeon with 4 physical cores, plus HT), and I have 12 VMs running, with altogether 18 vCPUs and never had any problems. Most of my VMs are very lightly loaded because they are running services not very cpu-intensive (i.e. dns, ftp, ntp). Just a few of them can load vCPU seriously (i.e. web-server, database, etc).

When comparing CPUs with/out HT, general rule is: you can overcommit pCPUs with 8 cores more, than pCPU with 4cores +HT (although both have 8 logical cores), and both can be overcommited more than pCPU with just 4 pCores without HT.

 

[Danic]

Thanks for the fast response. Overcommit is the correct word. It just slipped my mind when posting. 18 vCPUs sounds very high 4 core/8 thread CPU. I'm impressed. But in my experience anything over 1:1 (pCPU:vCPU) would cause problems. Maybe AMD's Phenom II is just a poor CPU when it comes to virtual machine performance. Maybe some other mis-configuration is causing my crappy performance. Well my plan is to upgrade to E3-1220 v3+Supermicro X10SLH-F hopefully it helps.

 

[CreoleLakerFan]

AMD cores != Intel cores. Plus that Thuban is sucking some serious wattage; the E3 1220 v3 should given you 3-4 times the performance at about 1/2 the peak power draw, and will utterly destroy it in idle power consumption.

 

[Chuckleb]

I utterly agree with Intel cores and AMD cores. If you look at how AMD lays out their chips, they share parts which makes it more akin to hyper threading. You get better value, but lower performance. We have lots of AMD and they run 50% slower at same price point.

 

[Danic]

Thanks for the input everyone and if anyone is curious, I did the upgraded. I got Intel E3-1231v3 and SuperMicro X10SLH-F. I choose the one with hyper-threading because the price was only little more than E3-1220v3 and allows me to overcommit a little more. First off I can officially take advantage the 4x8GB ECC memory I had. The old Gigabyte AM3+ motherboard I used did not support ECC, as far as I can test to is only ASUS's AM3+ motherboards do. Also the new hardware fixed some random boot issues I had in my FreeBSD VMs. I did some benchmarks of before and after, nothing too in depth. All these tests were done while 3 of my 'home' critical VMs were running.
On a Windows 2012 Server 4vCPU VM on ESXi 5.5: (Old AMD 1090T -> New Intel 1231v3)

• Novabench Overall 545 -> 667
• Novabench RAM 122 -> 156
  
• Novabench CPU 402 -> 482
• Novabench Graphics 3 -> 5
• Novabench HW 18 - > 23
• AS SSD Overall Score 742 -> 947 (This amazed me the most, it's the same SSD)
• 7-Zip Overall Rating 2987 / 11591 -> 3979 / 14909

On Ubuntu 14.04 4vCPU VM I used phoronix test suite. Results can be seen here.
Esxi-testing Benchmarks - OpenBenchmarking.org
Summary: Everything is at least 20% faster.
Lastly, my kill-a-watt shows a lower idle as well. My server now runs about 110 watts. Before the upgrade it was at 165 watts. That is going to save me $12.85 a month in power. ($.319 kWh).
Looking forward to lag free game, file, and voice servers at the next LAN I host.

 

[NeverDie]

We have lots of AMD and they run 50% slower at same price point.

Wow. I didn't know it was that bad. In that case, is there ever a good reason to buy an AMD CPU rather than an Intel CPU?

 

[NeverDie]

More vCPUs than pCores is not overloading, just overcommiting (or overcommitment? not sure what is correct). And it is quite normal and frequent with ESXi, because very rarely all your VMs are loading all vCPUs to the max, at the same time. How much you can overcommit CPU depends on your scenario. The best way is to observe pCPU-load on your ESXi-server for some time (i.e. with esxtop) and slowly increase number of VMs.

My ESXi-server has 8 logical cores (single xeon with 4 physical cores, plus HT), and I have 12 VMs running, with altogether 18 vCPUs and never had any problems. Most of my VMs are very lightly loaded because they are running services not very cpu-intensive (i.e. dns, ftp, ntp). Just a few of them can load vCPU seriously (i.e. web-server, database, etc).

When comparing CPUs with/out HT, general rule is: you can overcommit pCPUs with 8 cores more, than pCPU with 4cores +HT (although both have 8 logical cores), and both can be overcommited more than pCPU with just 4 pCores without HT.

So, if you are careful never to overcommit, will that guarantee "lag free" performance, or will the hypervisor's overhead necessarily introduce at least some lag?

 

[T_Minus]

Wow. I didn't know it was that bad. In that case, is there ever a good reason to buy an AMD CPU rather than an Intel CPU?

To save $ AND to get more cores for higher simultaneous usage.

You can do it with Intel, it would just cost more to get 12core, 14, 16, 18, etc.... vs AMD is <50% used per-CPU with that many cores they're just not = to Intel performance.

With that said I'm only using AMD to play around and compare the difference to intel I haven't used AMD daily for ~10 years or so.

 

[HellDiverUK]

I've moved over to AMD for my unRAID box. It uses a little more power, but it has 8 cores at 3.2-4GHz, so it can really motor through VMs and Dockers. I'm running the FX8320E on an Asus M5A78L-M board with 16GB ECC DDR3-1600 RAM. ECC is working.

 

[T_Minus]

I've moved over to AMD for my unRAID box. It uses a little more power, but it has 8 cores at 3.2-4GHz, so it can really motor through VMs and Dockers. I'm running the FX8320E on an Asus M5A78L-M board with 16GB ECC DDR3-1600 RAM. ECC is working.

What did you "move over" from? Curious the comparisons you've ran too.

 

[HellDiverUK]

What did you "move over" from? Curious the comparisons you've ran too.

E3-1225v3 on an Asus P9D-M.

 

[badskater]

I've been using Opterons for years, because they are cheap for a home lab. (Opteron 6238 vs Xeon E5-2620 for example) But mostly, all servers i advice at work uses Intel, because of how much stronger the Intel core is compared to AMD. (Yes, I got at one point cheap Intel servers on the home lab, but not enough ram slots 72gb vs 128gb with 4gb dimms can make a huge difference in a vCloud/Horizon home lab...)

Also, I wonder why you used the igfx version of the Xeon, while the motherboard has one integrated on it. I think you could have gotten a E3-1230v3 for almost the same price... (Well, it is here in Canada)

 

[Chuckleb]

Yeah memory slots was another advantage for AMD. They had more slots especially since they had relatively affordable quad socket boards giving you lots of memory slots...

 

[HellDiverUK]

Also, I wonder why you used the igfx version of the Xeon, while the motherboard has one integrated on it. I think you could have gotten a E3-1230v3 for almost the same price... (Well, it is here in Canada)

I used that, because it came in the Lenovo TS140 I bought to strip for parts. For a little over £100 after MIR, I got 4GB ECC RAM, a 460W 80Plus Platinum PSU, a 500GB WD RE4, two very nice fans, a slimline DVDRW drive, a copy of Server 2012R2 Essentials, and finally the Xeon.

 

[badskater]

I used that, because it came in the Lenovo TS140 I bought to strip for parts. For a little over £100 after MIR, I got 4GB ECC RAM, a 460W 80Plus Platinum PSU, a 500GB WD RE4, two very nice fans, a slimline DVDRW drive, a copy of Server 2012R2 Essentials, and finally the Xeon.

Now, i quite well understand. For that price, I would have done the same most likely.