ESXi 5.1 on C1100 and "low power" profile issues

spectdev

New Member
May 25, 2013
14
1
3
I have a C1100 with the following config:
- 2x L5520
- 72GB RAM
- 3 SSDs
- 1 HDD

Firmware Version 1.82.53138
BIOS Version S99C3B22

The system is surprisingly quiet and idles at 100 watts in ESXi low power profile and 150 watts in balanced power profile.
For my light VM usage, the low power profile make more sense, and that's what I am trying to use.

However, I have issues with ESXi freezing from time to time in that mode.
The error I get from IPMI every time it happens is: "PROC_IERR_STATUS: Processor sensor, IERR was asserted".
I don't get the issue in balanced power profile from testing over several days, but I might need to test further to confirm.

I have load tested the box by running several instance of y-cruncher - A Multi-Threaded Pi Program in a few VMs and maxing out the memory for hours.
The temps under load stay below 85C and stability is fine.

I have noticed the issue to arise in all cases thus far when the system is idle for a while and I execute a CPU intensive task in a VM thereafter.
It does not happen every time, but when it happens the above is the typical case that leads to it.

Right now, I have set ACPI to C2 and disabled C6 in the bios and currently waiting to see if it happens again.

Is anyone having similar experience under ESXi in low power profile or do I just have a faulty system?

Thanks.
 
Last edited:

spectdev

New Member
May 25, 2013
14
1
3
I have.
I believe the options were for either Dell Active Power Controller or OS.
I have had it set to OS for all tested scenarios.

Too bad there is no option in the bios for "Low Power".

Thanks.
 

spectdev

New Member
May 25, 2013
14
1
3
Quick update: ACPI to C2 and disabling C6 made no difference.

Back to balanced profile and will test it for a few days.
 

spectdev

New Member
May 25, 2013
14
1
3
So, the system is has been up for a few days now on balanced profile.

Anyone with C1100 that can test low power profile?
 

tjk

Active Member
Mar 3, 2013
288
51
28
www.servercentral.com
Sorry for the off-topic question, but has anyone been able to update the bios/bmc/drac firmware on these? I picked up a batch, and no luck with any methods I try nor can I find much online about updating these.

Thanks,
Tom
 

PersonalJ

Member
May 17, 2013
127
11
18
Sorry for the off-topic question, but has anyone been able to update the bios/bmc/drac firmware on these? I picked up a batch, and no luck with any methods I try nor can I find much online about updating these.

Thanks,
Tom
No, I was not able to. You may be able to force a flash but I was afraid it would brick the server.
 

spectdev

New Member
May 25, 2013
14
1
3
Sorry for the off-topic question, but has anyone been able to update the bios/bmc/drac firmware on these? I picked up a batch, and no luck with any methods I try nor can I find much online about updating these.

Thanks,
Tom
The newest bios is B23 (I have B22).
You should not update your machine to the latest as per the accounts of those that succeeded in updating, ESXi stopped working and who knows what else.
If you have a version older than B22, then do update to B22 and then update the BMC to 1.82.

Back to the topic though, can someone help me out and run ESXi in low power mode for a few days and report back on stability?

Thanks.
 

spectdev

New Member
May 25, 2013
14
1
3
So, it looks like I have a defective system as the system gave the same error even at balanced power settings after a few days.
I bought a replacement.
 

kenop

New Member
Jul 8, 2013
1
0
0
So, it looks like I have a defective system as the system gave the same error even at balanced power settings after a few days.
I bought a replacement.
Hello Spectdev I am just curious did your replacement system work out for you, and where did you get your server from?

I got one from DDS/Lextec and am having the same issue, where the system seems to lock up under load.

Haven't tried changing the power profiles yet but hoping it doesn't need a replacement like yours.
 

badskater

Automation Architect
May 8, 2013
129
44
28
Canada
Hello Spectdev,

i have 6 C1100s at home for a vCloud Suite 5.1 Cluster with a NetApp iSCSI SAN. I personally never setup the power profile, cause i don't mind using 50w more of power. I prefer to do this, so i'm sure i won't encounter bugs. I have B22 on all my servers, and no problems with ESXi.

6 C1100 configs:

2 x L5639
72GB of DDR3
No HDDs (Internal USB that's all)
Quad-Nic for iSCSI and View

Personally, i wouldn't recommend using low power mode, but well, it's your choice at the end.
 

Fzdog2

Member
Sep 21, 2012
92
14
8
Hello Spectdev,

i have 6 C1100s at home for a vCloud Suite 5.1 Cluster with a NetApp iSCSI SAN. I personally never setup the power profile, cause i don't mind using 50w more of power. I prefer to do this, so i'm sure i won't encounter bugs. I have B22 on all my servers, and no problems with ESXi.

6 C1100 configs:

2 x L5639
72GB of DDR3
No HDDs (Internal USB that's all)
Quad-Nic for iSCSI and View

Personally, i wouldn't recommend using low power mode, but well, it's your choice at the end.
I believe the rules of the forum are you can't tease all of us by posting a 6 server vCloud Suite @home without posting up a thread with pics/description.
 

Biren78

Active Member
Jan 16, 2013
550
94
28
I believe the rules of the forum are you can't tease all of us by posting a 6 server vCloud Suite @home without posting up a thread with pics/description.
Can I second your motion?! We need to see pics of this.
 

PigLover

Moderator
Jan 26, 2011
3,011
1,313
113
Yeah. Pictures. Mandatory. You can't tease like that and get away without following up.
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
The tylersberg chipset was notoriously buggy with power management. On top of that ESXi doesn't do well since it throws your junk into P12 maximum slowness and parks a second socket in default it sets the threshold to 20% (two sockets feel even more pain). so it samples this very often and what you end up with is instability and really poor performance. Every answer to various errata is simple: Disable C1E,C-states, and set the profiles to MAX/MAX/MAX and life will be good.

the earlier chipsets even changed the power TDP envelope because they could not reliably work with power management. Very similar to how earlier tylersberg chipsets did not grant 5600 series cpu features - many motherboards had to be replaced when the customers figured out their SR-IOV, VT-D, and AES-NI was not present even though the cpu's supported the features! :)

What I found for business applications was reducing the CPU to 1 socket increased the load enough that the ESXi would not force it down as far into power savings and that 30 watts was the difference between a slow 2-socket and very fast 1-socket.

The i/o was slowed down which then caused the cpu to drop load and vicious low cpu cycle :(
 

Navy_BOFH

Active Member
Aug 2, 2013
150
58
28
Hello Spectdev,

i have 6 C1100s at home for a vCloud Suite 5.1 Cluster with a NetApp iSCSI SAN. I personally never setup the power profile, cause i don't mind using 50w more of power. I prefer to do this, so i'm sure i won't encounter bugs. I have B22 on all my servers, and no problems with ESXi.

6 C1100 configs:

2 x L5639
72GB of DDR3
No HDDs (Internal USB that's all)
Quad-Nic for iSCSI and View

Personally, i wouldn't recommend using low power mode, but well, it's your choice at the end.
Help me out here - since I have a CS24-TY and trying to go ESXi on it...

Where/how did you hook up the internal USB? And if you do that - can you still have your HDDs? I am trying to run ESXi off USB but utilize all 4 drive bays for storage. Right now I have 2 filled with 2Tb WD RE4s, but trying to get 2 more drives in and set up a ZFS share to then store my additional VMs along with all my media.

I have been trying to figure out all the details (along with updating BIOS) but I'm deployed and our bandwidth is quite limited.
 

Aliv Cherevko

New Member
Jul 3, 2015
15
0
1
So, it looks like I have a defective system as the system gave the same error even at balanced power settings after a few days.
I bought a replacement.
spectdev, your system wasn't defective. I had exactly same issues with both of my C1100 CS24-TY. And this happened exactly in the same manner like the issues you described.

And I fixed the issues. I updated BIOS from DELL to original BIOS from Quanta. After that I also installed XenServer 6.2 instead of ESXi. The machine is up an running about a year from that time . No issues at all.
 
Last edited: