HP DL180 G6

mattlach

Member
Aug 1, 2014
153
14
18
I am actually considering getting an Arduino and writing a small program to take care of the fan problem
On second thought this project rapidly grew in complexity. I am going to try for a simpler solution. The arduino solution IS the most elegant, and I may pursue it long term when I have time.
 

Marsh

Moderator
May 12, 2013
2,294
1,109
113
I am 59 years , I learn thru out the years, simplicity have its place in life.
I spend 1/2 hour and few bucks, I am satisfy with the fan mod outcome. Next step is to replace the fans if I am not happy with the mod.

If you like, I'll make a fan rpm connector for you.
 

zoroyoshi

New Member
Dec 10, 2013
20
6
3

(sorry for my poor english)
- I've made two boards, one is for FAN1/2 and another is for FAN3/5. (SENS pins are individual)
- some newer version of BMC firmware checks as if the fans are controllable by PWM, so I used a version of 4.23.
- this analog method is very cheap, but I think it's more better to use microcontroller.
 

mattlach

Member
Aug 1, 2014
153
14
18

(sorry for my poor english)
- I've made two boards, one is for FAN1/2 and another is for FAN3/5. (SENS pins are individual)
- some newer version of BMC firmware checks as if the fans are controllable by PWM, so I used a version of 4.23.
- this analog method is very cheap, but I think it's more better to use microcontroller.
Thank you for that description!

Unfortunately my electronics skills are very limited. How does this analog controller work? Is it a fixed speed stepdown, or can it somehow change fan speeds based on temperature?
 

mattlach

Member
Aug 1, 2014
153
14
18
Meanwhile I hope I haven't made a serious and expensive mistake...

Originally I looked at the HP manual for the server to confirm VT-D support. In the BIOS settings section (page 143 of the pdf) it mentioned enable/disable VT-D so I assumed I was safe.

Then I came across this post.

So now it looks like I have conflicting information.

The Xeon L5640's support VT-D according to intel, but that won't do much good if the server refuses to enable it.

I absolutely need VT-d support for my setup. Can anyone confirm whether or not it works?

Did I make an expensive mistake by ordering one of these? If so, I hope the seller will take it back, as the specs suggest VT-d support. (and they offer a 30 day warranty)

Appreciate any thoughts!
Thanks,
Matt
 

mattlach

Member
Aug 1, 2014
153
14
18
Meanwhile I hope I haven't made a serious and expensive mistake...

Originally I looked at the HP manual for the server to confirm VT-D support. In the BIOS settings section (page 143 of the pdf) it mentioned enable/disable VT-D so I assumed I was safe.

Then I came across this post.

So now it looks like I have conflicting information.

The Xeon L5640's support VT-D according to intel, but that won't do much good if the server refuses to enable it.

I absolutely need VT-d support for my setup. Can anyone confirm whether or not it works?

Did I make an expensive mistake by ordering one of these? If so, I hope the seller will take it back, as the specs suggest VT-d support. (and they offer a 30 day warranty)

Appreciate any thoughts!
Thanks,
Matt
Actually, never mind. I think I got my answer here.

This was likely either a problem with early BIOS releases for our servers back in 2009, or the person having the issue had a non VT-D capable Xeon.
 

cptbjorn

Member
Aug 16, 2013
100
19
18
appears to be a variable pulse width generator. the diode to the pwm pin on the motherboard makes it so both outputs need to be high to give the fan a high value (AND logic), basically at 100% pulse width it will be identical to stock performance and the pot will turn fan speed down from there. you could do same w/ a uC with pwm out (such as arduino), but would want a series resistor.

also passthrough works fine w/ dl180 g6 and esxi 5.x
 

pgh5278

Active Member
Oct 25, 2012
473
126
43
Australia

(sorry for my poor english)
- I've made two boards, one is for FAN1/2 and another is for FAN3/5. (SENS pins are individual)
- some newer version of BMC firmware checks as if the fans are controllable by PWM, so I used a version of 4.23.
- this analog method is very cheap, but I think it's more better to use microcontroller.
Hello, good idea. Have a question, is the PWN signal going back to the DL180 board, have a high rpm rate via PWM, while the fan is actually spinning slower?
Thanks P
 

zoroyoshi

New Member
Dec 10, 2013
20
6
3
(PWM on FAN)=(PWM from board generated by BMC) AND (PWM signal created by NE555),
not backing to DL180 board.(NE555 DIScharge signal is open collector)

VT-d: I used 2012R2, SR-IOV can be checked on motherboard-LAN(82576).
 

zoroyoshi

New Member
Dec 10, 2013
20
6
3
I'm very sorry about VT-d (SR-IOV) capable on my DL180...
get-netadaptersriov from powershell causes the following:
Name : Ethenet 2
InterfaceDescription : Intel(R) 82576 Gigabit Dual Port Network Connection #2
Enabled : True
SriovSupport : NoIoMmuSupport
SwitchName : DefaultSwitchName
NumVFs : 6

Name : Ethernet
InterfaceDescription : Intel(R) 82576 Gigabit Dual Port Network Connection
Enabled : True
SriovSupport : NoIoMmuSupport
SwitchName : DefaultSwitchName
NumVFs : 6
NoIoMmuSupport.... not capable of SR-IOV.
 

mattlach

Member
Aug 1, 2014
153
14
18
I remember that I had tried pass through , vt-d worked ok with esxi 5.x with X5650
Excellent.

I just received mine yesterday, and apart from a hiccup where I didn't get the riser card I had specicifed (they are hipping it separately now) specified in my order, everything is working well!

VT-d is showing up in the BIOS, so it was probably just a matter of early BIOS revisions not having it implemented yet. (fingers crossed) (Mine cam ewith the June 2011 BIOS on it, will be updating to latest as soon as I get ILO working.

ILO configured nicely using the dedicated management port on the back. I have an IP set up, and it gives me the logon screen remotely. As others have experienced, the default login credentials for ILO aren't working though (presumably because the previous user changed them). I tried resetting them using the il100cfg utility as described below from the old DL180 thread:


In case anyone is wondering how to access the lo100 admin account I tried to reset the password by updating the lo100 firmware and then tried with the Intel ipmitool however both did not work. What I ended up doing was downloading the SmartStart Scripting Toolkit Win32 Edition part 1 which contains the lo100cfg utility. From there I ran lo100cfg.exe -s and had it output the current settings and I saw the following:
<users>
<user id="2" name="ROUSER" privilege_level="operator" />
<user id="3" name="USERID" privilege_level="administrator" />
</users>
I was then able to login with the USERID account using the password "admin".
I ran into trouble though as I only had client versions of Windows to install, as I don't normally run Windows (even my desktop is a linux machine) No love from the lo100cfg utility under Windows 7 Professional. I was able to find an ISO of Windows Server 2008 online. Hopefully it will install in a 30 day trial mode, just like Windows 7 does on the desktop, so I won't need a key, and I can use that to reset the ILO admin account and then wipe the drive and go about installing ESXi.

A quick note for anyone else using one of these, as it might save them a lot of time in trouble shooting: I got stuck for hours with a system that would not boot off of any media. First I tried off of a USB stick. Never booted, just a blinking cursor. Then I tried using a USB DVD drive, again, just a blinking cursor. Then I opened the case and installed a DVD drive direcly using one of the motherboard SATA ports. Still no love, just a blinking cursor. Was tearing my hear out trying to figure out what BIOS setting I must have missed, and wondering if I had wound up with a defective unit until I on a whim, found and disabled the hot plug USB floppy setting, which finally allowed me to boot.

Apparently it creates a fake floppy so that you can hot plug a USB floppy drive for maintenance. Problem was my unit was trying to boot off of this fake floppy at the expense of everything else (and it wasn't apparentl either, as floppy was not listed in the boot order menu in the BIOS). Once I disabled it, everything was good!

Also wasted a shitload of time by accidentally disabling quick boot when I was troubleshooting my boot problems. It is amazing how long it takes to count up to 96GB of RAM :p

Next step, install Windows 2008 Server (hopefully in trial mode) to reset ILO, then do some baseline temperature testing by loading all the cores with mprime in linux (or prime95 in Windows) so I know where I stand before my fan mod.

Does anyone off hand know what the max temp is of the L5640 CPU's? The desktop LGA1366 chips can go up to 100C - if I recall - but I am not sure if the low power Xeons have a lower limit. It also depends on what the server management firmware thinks is an over temperature condition.

That being said, I should be able to figure out what the stock target temperature is by loading up the cores without any PCIe boards installed, jotting down fan RPM's and core temps as they rise to a stable temp, and then plot temps vs fan RPM's and fit a line to try to see what core temp corresponds to a 100% 14krpm duty cycle, and then set up my fan mod to stay under this temperature (with a little safety margin, of course)

Wish me luck!
 

mattlach

Member
Aug 1, 2014
153
14
18
Oh, and remind me to wear a set of these next time I work on this thing :p



I think my ears were ringing after I hung out waiting for the 96GB memory countdown so I could hit F10 and enter the BIOS right when it was donen to turn quick boot back on :p
 
Last edited:

mattlach

Member
Aug 1, 2014
153
14
18
Alright,

Brief update.

The fan controller solution did not pan out, primarily because there is no good location around the CPU's to place a thermal probe such that it both is close enough to the CPU to pick up a close approximation of the core temp AND is sufficiently shielded from the airflow, that it isn't cooler down itself by the flowing air, and reads too low.

I tried Marsh's suggestion to use two of the original fans, disconnect the rest, and fake the motherboard using the RPM signal. This worked, but not as well as for Marsh, as my expansion card forces the fans into the higher RPM value, so with two of them spinning they are still producing ~76dB (and a bad 76dB at that, due to the high pitched whine) as compared to ~86dB with all 8 redundant fans spinning.) At this level I can still hear the server one floor up from the basement in my livingroom. (100 year old house, thin walls and floors I guess.)

At some point in this latest step, my IPMI management port stopped responding to network requests. Don't know what happened or why, but it still appears to be working internally, as the BIOS can still read fan RPM's and temperatures, and it still detects fan failures when I disconnect RPM signals. Hoping it is not related. May try reseating the dedicated management port module

Today after work, I am going to - instead - try to hook up my 60mm delta fans at full speed (6800rpm). It may not be any quieter, but I'm hoping the noise profile is more "whoosh" like, rather than the high pitched whine of the original 14krpm sunons (even when operating at a lower duty cycle)

The Delta's do have a lower CFM airflow rating though and a much lower static pressure rating than the Sunon's. To compensate for this, I'm also going to zip-tie one super narrow, ultra quiet 60mm fan to each CPU heatsink to make sure they get enough airflow and don't overheat.

Wish me luck!
 

Marsh

Moderator
May 12, 2013
2,294
1,109
113
@zoroyoshi

give (get-vmhost).IovSupportReasons a try within powershell
I am starting my journey with Hyper-v SR-IOV setup, here is my vm host setup.

ASUS Z9NA-D6 LGA1356 server board with 2 E5-2430 ES chip. Intel E1G42ET network card

PS C:\Users\Fun> (get-vmhost).IovSupportReasons
This system has a security vulnerability in the system I/O remapping hardware. As a precaution, the ability to use SR-I
OV has been disabled. You should contact your system manufacturer for an updated BIOS which enables Root Port Alternate
Error Delivery mechanism. If all Virtual Machines intended to use SR-IOV run trusted workloads, SR-IOV may be enabled
by adding a registry key of type DWORD with value 1 named IOVEnableOverride under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft
\Windows NT\CurrentVersion\Virtualization and changing state of the trusted virtual machines. If the system exhibits re
duced performance or instability after SR-IOV devices are assigned to Virtual Machines, consider disabling the use of S
R-IOV.

I made the registry change, it may work now

PS C:\Users\Fun> (get-vmhost).IovSupportReasons
OK
PS C:\Users\Fun> Get-NetAdapterSriov

Name : Ethernet 3
InterfaceDescription : Intel(R) 82576 Gigabit Dual Port Network Connection
Enabled : True
SriovSupport : Supported
SwitchName : DefaultSwitchName
NumVFs : 6

Name : Ethernet 4
InterfaceDescription : Intel(R) 82576 Gigabit Dual Port Network Connection #2
Enabled : True
SriovSupport : Supported
SwitchName : DefaultSwitchName
NumVFs : 6

Confirm WS12r2 vm client is using SR-IOV.
Key steps,
physical host > enable VT-D,
in my case, made the registry change,
VM host , create hypre-v switch with SR-IOV enabled.
VM client settings > network adapter > Hardware Acceleration > Enable SR-IOV
boot vm ws12r2 client > Device Manager > Network adapters > Intel 82576 Virtual Function
 
Last edited:

zoroyoshi

New Member
Dec 10, 2013
20
6
3
@Marsh

I installed english language pack and get-vmhost | FL from powershell, it says
IovSupportReasons : {The chipset on the system does not do interrupt remapping, without which S
R-IOV cannot be supported., To use SR-IOV on this system, the system BIOS m
ust be updated to allow Windows to control PCI Express. Contact your system
manufacturer for an update., SR-IOV cannot be used on this system as the P
CI Express hardware does not support Access Control Services (ACS) at any r
oot port. Contact your system vendor for further information., This system
has a security vulnerability in the system I/O remapping hardware. As a pre
caution, the ability to use SR-IOV has been disabled. You should contact yo
ur system manufacturer for an updated BIOS which enables Root Port Alternat
e Error Delivery mechanism. If all Virtual Machines intended to use SR-IOV
run trusted workloads, SR-IOV may be enabled by adding a registry key of ty
pe DWORD with value 1 named IOVEnableOverride under HKEY_LOCAL_MACHINE\SOFT
WARE\Microsoft\Windows NT\CurrentVersion\Virtualization and changing state
of the trusted virtual machines. If the system exhibits reduced performance
or instability after SR-IOV devices are assigned to Virtual Machines, cons
ider disabling the use of SR-IOV.}

Too many problems to use SR-IOV on DL180 G6 :-<
 

Marsh

Moderator
May 12, 2013
2,294
1,109
113
@zoroyoshi

Did you tried adding IOVEnableOverride in the registry?

SR-IOV may be enabled
by adding a registry key of type DWORD with value 1 named IOVEnableOverride under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft
\Windows NT\CurrentVersion\Virtualization

I could fire up my HP to to give a try , but HP does not have any free PCI slot for network card though.
 

mattlach

Member
Aug 1, 2014
153
14
18
@zoroyoshi

Did you tried adding IOVEnableOverride in the registry?

SR-IOV may be enabled
by adding a registry key of type DWORD with value 1 named IOVEnableOverride under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft
\Windows NT\CurrentVersion\Virtualization

I could fire up my HP to to give a try , but HP does not have any free PCI slot for network card though.
Luckily for me, I don't plan on using SR-IOV, just straight 1:1 Direct I/O mapping. I hope you guys figure this out. Best of Luck!