Tested: DL580G8 (Gen8) and G9 (Gen9) *Lack of* Compatibility

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
Funny enough, if we talk about E7 v3 + DDR3 - almost every vendor except HP and Dell supported those configuration. Cisco UCS series support v3 with DDR3 ram, Oracle (which in my opinion almost always best engineered servers), even Huawei supported DDR3 with v3 Xeons. I had some experience with almost every quadsocket Brickland platform released, and in my experience DL580 Gen8/Gen9 is actually one of the worst ones, only IBM with their awful X6 is worse. And my favorites are Cisco UCS C460 and Oracle X4-4/X5-4 machines. Although nothing new here, DL560 Gen8 were one of the worst "el-cheapo" quad socket machines... So what else to expect?
What did you dislike about the IBM/Lenovo X6? And what did you prefer about the Cisco and Oracle platforms?
 

RussianE39

New Member
May 22, 2021
13
2
3
What did you dislike about the IBM/Lenovo X6? And what did you prefer about the Cisco and Oracle platforms?
IBM - Price, amount of bugs, performance, support. The only cool thing about X6 were their DDR SSD sticks IMHO. Cisco and Huawei is probably most flexible ones from this generation. Oracle is best considering HW and FW engineering and the only one where you can get E7-8895v2 CPUs officially with their TurboBoost tricks, also all MSR registers are not latched on those. So in bare tests, Oracle machines would be benchmark winners. They have their own disadvantages - like only LP expansion cards, PITA to get firmware, VERY noisy (thx to power supplies). Getting cheap parts for Oracle machines is another thing. I personally use DL580 Gen8 as one of my workstations, but I DONT like how they designed and made it, I just use it from price/performance ratio, probably DL580 were most popular Brickland platform ever made, so its cheapest to maintain.
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
IBM - Price, amount of bugs, performance, support. The only cool thing about X6 were their DDR SSD sticks IMHO. Cisco and Huawei is probably most flexible ones from this generation. Oracle is best considering HW and FW engineering and the only one where you can get E7-8895v2 CPUs officially with their TurboBoost tricks, also all MSR registers are not latched on those. So in bare tests, Oracle machines would be benchmark winners. They have their own disadvantages - like only LP expansion cards, PITA to get firmware, VERY noisy (thx to power supplies). Getting cheap parts for Oracle machines is another thing. I personally use DL580 Gen8 as one of my workstations, but I DONT like how they designed and made it, I just use it from price/performance ratio, probably DL580 were most popular Brickland platform ever made, so its cheapest to maintain.
Oops - I just bought two X6 recently!

I have 2 DL580G8, 2 DL580G9, 2 X3850 X6

Isn't IBM the only one which actually supports 8CPUs this generation?

What's the backstory with the 8895v2? Do they not use their highest P-States on most of these servers? Do you have any interesting reading links on this?

I feel like the HP is really well assembled from a hardware POV, and it's the BIOS restrictions which make it the worst platform. Being able to swap the whole CPU tray is nice, and if you could simply arbitrarily run memory and CPU combinations like the hardware actually supports (decoupling CPUs from DDR3/DDR4), it would be very nice.
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
So, some interesting findings from a per-outlet managed PDU:
The IPMI/Standby Power for IBM/Lenovo X3850 X6 to just be plugged into the wall with a single PSU is about ~26.5W of active power! That's shockingly high to me. I'll have to measure the HP DL580 Gen9 and such in a bit for comparison, but I feel like it's nowhere near this high for pure standby ILO.

Edit: I started a power consumption thread here: Power Consumption Thread | ServeTheHome Forums
 
Last edited:

MichalPL

Active Member
Feb 10, 2019
189
25
28
Few weeks ago I bought another 4x 8895v2 for second DL580 G8 (to replace 4890v2) (today they are really for free, about $40each lol), but decision for future is to switch to TR (32cores or more), it's faster for massive C++ compiling tested on slightly overclocked TR3970.

I personally use DL580 Gen8 as one of my workstations
How did you managed the noise that DL5800G8 generate? Have you replaced fans?
lol, 1.5 year ago - before Ryzen 5000, and intel 12gen it was one of the fastest machines on the planet even single core performance was good.

Isn't IBM the only one which actually supports 8CPUs this generation?

TurboBoost tricks
Have you managed to run 8895v2 3.6GHz all core all the time ?
 
Apr 27, 2022
3
5
3
Just joined to update this thread and say that it is indeed possible to run v4 Broadwell-EX CPUs on an HP DL580 Gen8 (original hardware), even with DDR3 1333Mhz RAM (PC3-10600R).

I was able to confirm two very important details in my testing:

*Swapping a DL580 G8's SPI board with a DL580 G9's SPI board*
  • Results in a 'fully converted' G8 into a G9 with no issues and no side-effects (everything works as if it were a G9)
  • All iLO functionality works as expected
  • G8 chassis no longer accepts v1/v2 CPUs, but does accept v3/v4 CPUs (expected with a G9)
  • G8 chassis no longer accepts DDR3 RAM/cartridges, but does accept DDR4 RAM/cartridges (expected with a G9)
*Cross-flashing a DL580 G8's SPI board with the iLO/BIOS firmware from a donor DL580 G9's SPI board*
  • Results in a 'mostly converted' G8 into a G9 with some side-effects (detailed further below)
  • G8 chassis no longer accepts v1/v2 CPUs, but does accept v3/v4 CPUs (expected with a G9)
  • G8 chassis accepts either DDR3 RAM/cartridges *or* DDR4 RAM/cartridges as long as the CPU supports it (unexpected, but welcomed!)
  • HTML5-based Remote Console works just fine with the G8 SPI board hardware!

*HP DL580 Gen8 SPI Cross-flash/Broadwell-EX/DDR3 Project*


Project synopsis:
  • Sandy Bridge-EX (v1), Ivy Bridge-EX (v2), Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs (5 years worth of server CPU generations) all use the same LGA2011-X socket (electrically-compatible)
  • Sandy Bridge-EX (v1), Ivy Bridge-EX (v2), Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs (5 years worth of server CPU generations) can run on each other's chipsets
  • Sandy Bridge-EX (v1) and Ivy Bridge-EX (v2) Xeon CPUs use external voltage regulators (built into motherboard)
  • Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs use internal voltage regulators (built into CPU)
  • Due to the above, v1/v2 Motherboards can accept v3/v4 CPUs without issue *as long as the BIOS allows it* (but most don't... especially Dell and HP BIOSes)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) support both DDR3 and DDR4 RAM
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) include on-die eDRAM cache (doesn't exist on v1/v2 CPUs)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) support *both* AVX and AVX2 instructions (v1/v2 CPUs *only* provide AVX)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) are a die-shrunk (14nm) version of v3 CPUs (22nm), and thus are more efficient/cooler than v1/v2/v3 CPUs
  • HP DL580 Gen8 Servers (my server) supports only v1/v2 CPUs and DDR3 RAM (won't accept v3/v4 CPUs or DDR4 RAM)
  • HP DL580 Gen9 Servers (newer version of my server, which I don't own) supports only v3/v4 CPUs and DDR4 RAM (won't accept v1/v2 CPUs or DDR3 RAM)
  • I want to run the v4 CPUs, with my DDR3 RAM, on my HP DL580 Gen8 Server... but the G8 BIOS won't allow it (refuses to boot with v4 CPUs, even if I inject the Broadwell microcode into the G8 BIOS it still won't boot)
  • Despite the G8 BIOS not working with v4 CPUs or their microcode, I still managed to get this wacky combo (G8 running v4 CPUs and DDR3 RAM) working by dumping the G9 iLO/BIOS firmware from a damaged G9 SPI board and programming it onto my working G8 SPI board with an external programmer (BlackCat USB)
Observed side-effects from programming G9 iLO (2.77)/BIOS (U17) onto G8 SPI board:
  • G8 (w/G9 iLO and BIOS flashed) fans run at 50% load 24/7 - fans do not ramp up or down, ever, regardless of Power Settings.
  • G8 (w/G9 iLO and BIOS flashed) Serial Number and Product ID show as blank (until I fixed it in the BIOS service menus, easy-peasy)
  • G8 (w/G9 iLO and BIOS flashed) boots normally, but I get a yellow-colored (non-blocking) warning/error regarding the REST API (seems the BIOS is sending an API call to the iLO interface/webserver that is returning a 500 Internal Server Error response)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Agentless Management Service' as 'Not available'
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Fans' as 'Not available' (which explains the lack of fan control)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Power Supplies' as 'Not available' (however, all Power-related functions in iLO still work as expected - such as the Server Power buttons, Power Regulator, Power Capping, and Power Meter sections)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Temperatures' as 'Not available' (unlike the Power Supplies, the temperature data is indeed missing/non-functional elsewhere in iLO... however the POST screen shows correct Inlet temp, and OS-level utilities also report correct temps such as ESXi)
Aside from the above side-effects, the G8 is totally solid with v4 CPUs and DDR3 RAM running iLO/BIOS firmware from a G9.

I would attach the dumps of the iLO/BIOS firmware from the G9, but i'm guessing it's not allowed - for now I have attached pictures for reference.
 

Attachments

seanneko

New Member
Feb 1, 2022
17
5
3
Just joined to update this thread and say that it is indeed possible to run v4 Broadwell-EX CPUs on an HP DL580 Gen8 (original hardware), even with DDR3 1333Mhz RAM (PC3-10600R).
Great that you were able to prove that the hardware fully supports it, it's just a limitation of the BIOS. Wonder if there would be a way to merge the Gen8 and Gen9 BIOS to give Gen9 CPU support, but still work correctly with the rest of the Gen8 platform e.g. fans. Being able to use v4 CPUs with the fans still working correctly would give these systems years more life going into the future.
 
  • Like
Reactions: Layla

MichalPL

Active Member
Feb 10, 2019
189
25
28
Just joined to update this thread and say that it is indeed possible to run v4 Broadwell-EX CPUs on an HP DL580 Gen8 (original hardware), even with DDR3 1333Mhz RAM (PC3-10600R).
Just bought G9, do you think it is possible to use DDR3 cassettes with 1866MHz DDR3 inside (4x E7-8894v4) ?
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
Just joined to update this thread and say that it is indeed possible to run v4 Broadwell-EX CPUs on an HP DL580 Gen8 (original hardware), even with DDR3 1333Mhz RAM (PC3-10600R).

I was able to confirm two very important details in my testing:

*Swapping a DL580 G8's SPI board with a DL580 G9's SPI board*
  • Results in a 'fully converted' G8 into a G9 with no issues and no side-effects (everything works as if it were a G9)
  • All iLO functionality works as expected
  • G8 chassis no longer accepts v1/v2 CPUs, but does accept v3/v4 CPUs (expected with a G9)
  • G8 chassis no longer accepts DDR3 RAM/cartridges, but does accept DDR4 RAM/cartridges (expected with a G9)
*Cross-flashing a DL580 G8's SPI board with the iLO/BIOS firmware from a donor DL580 G9's SPI board*
  • Results in a 'mostly converted' G8 into a G9 with some side-effects (detailed further below)
  • G8 chassis no longer accepts v1/v2 CPUs, but does accept v3/v4 CPUs (expected with a G9)
  • G8 chassis accepts either DDR3 RAM/cartridges *or* DDR4 RAM/cartridges as long as the CPU supports it (unexpected, but welcomed!)
  • HTML5-based Remote Console works just fine with the G8 SPI board hardware!

*HP DL580 Gen8 SPI Cross-flash/Broadwell-EX/DDR3 Project*


Project synopsis:
  • Sandy Bridge-EX (v1), Ivy Bridge-EX (v2), Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs (5 years worth of server CPU generations) all use the same LGA2011-X socket (electrically-compatible)
  • Sandy Bridge-EX (v1), Ivy Bridge-EX (v2), Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs (5 years worth of server CPU generations) can run on each other's chipsets
  • Sandy Bridge-EX (v1) and Ivy Bridge-EX (v2) Xeon CPUs use external voltage regulators (built into motherboard)
  • Haswell-EX (v3) and Broadwell-EX (v4) Xeon CPUs use internal voltage regulators (built into CPU)
  • Due to the above, v1/v2 Motherboards can accept v3/v4 CPUs without issue *as long as the BIOS allows it* (but most don't... especially Dell and HP BIOSes)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) support both DDR3 and DDR4 RAM
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) include on-die eDRAM cache (doesn't exist on v1/v2 CPUs)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) support *both* AVX and AVX2 instructions (v1/v2 CPUs *only* provide AVX)
  • E7 v4 CPUs (Broadwell-EX, a.k.a. the "top dog" LGA2011-X CPUs) are a die-shrunk (14nm) version of v3 CPUs (22nm), and thus are more efficient/cooler than v1/v2/v3 CPUs
  • HP DL580 Gen8 Servers (my server) supports only v1/v2 CPUs and DDR3 RAM (won't accept v3/v4 CPUs or DDR4 RAM)
  • HP DL580 Gen9 Servers (newer version of my server, which I don't own) supports only v3/v4 CPUs and DDR4 RAM (won't accept v1/v2 CPUs or DDR3 RAM)
  • I want to run the v4 CPUs, with my DDR3 RAM, on my HP DL580 Gen8 Server... but the G8 BIOS won't allow it (refuses to boot with v4 CPUs, even if I inject the Broadwell microcode into the G8 BIOS it still won't boot)
  • Despite the G8 BIOS not working with v4 CPUs or their microcode, I still managed to get this wacky combo (G8 running v4 CPUs and DDR3 RAM) working by dumping the G9 iLO/BIOS firmware from a damaged G9 SPI board and programming it onto my working G8 SPI board with an external programmer (BlackCat USB)
Observed side-effects from programming G9 iLO (2.77)/BIOS (U17) onto G8 SPI board:
  • G8 (w/G9 iLO and BIOS flashed) fans run at 50% load 24/7 - fans do not ramp up or down, ever, regardless of Power Settings.
  • G8 (w/G9 iLO and BIOS flashed) Serial Number and Product ID show as blank (until I fixed it in the BIOS service menus, easy-peasy)
  • G8 (w/G9 iLO and BIOS flashed) boots normally, but I get a yellow-colored (non-blocking) warning/error regarding the REST API (seems the BIOS is sending an API call to the iLO interface/webserver that is returning a 500 Internal Server Error response)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Agentless Management Service' as 'Not available'
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Fans' as 'Not available' (which explains the lack of fan control)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Power Supplies' as 'Not available' (however, all Power-related functions in iLO still work as expected - such as the Server Power buttons, Power Regulator, Power Capping, and Power Meter sections)
  • G8 (w/G9 iLO and BIOS flashed)'s iLO interface reports 'Temperatures' as 'Not available' (unlike the Power Supplies, the temperature data is indeed missing/non-functional elsewhere in iLO... however the POST screen shows correct Inlet temp, and OS-level utilities also report correct temps such as ESXi)
Aside from the above side-effects, the G8 is totally solid with v4 CPUs and DDR3 RAM running iLO/BIOS firmware from a G9.

I would attach the dumps of the iLO/BIOS firmware from the G9, but i'm guessing it's not allowed - for now I have attached pictures for reference.
Awesome post and awesome work! Thanks for sharing!!!
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
Just bought G9, do you think it is possible to use DDR3 cassettes with 1866MHz DDR3 inside (4x E7-8894v4) ?
Even if you could (which you probably can if you follow the above instructions, and are willing to deal with the compromises), IIRC, the DDR3 is limited to 1600MHz on 8894v4.

DDR4-1333/1600/1866

DDR3-1066/1333/1600

 

nutsnax

Active Member
Nov 6, 2014
247
92
28
113
So if one wanted to go this route with a quad Xeon E7 v4 setup on one of these E7 platforms (i.e. Cisco C460 M4,DL580 G8/9,Dell R8XX, Supermicro X10QBI)

....which would be the most straightforward/easiest to do?
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
So if one wanted to go this route with a quad Xeon E7 v4 setup on one of these E7 platforms (i.e. Cisco C460 M4,DL580 G8/9,Dell R8XX, Supermicro X10QBI)

....which would be the most straightforward/easiest to do?
I think it's confirmed that Cisco and Supermicro support this out of the box. AFAIK, it's only Dell and HP that are known to attempt to prevent this.
 

nutsnax

Active Member
Nov 6, 2014
247
92
28
113
I think it's confirmed that Cisco and Supermicro support this out of the box. AFAIK, it's only Dell and HP that are known to attempt to prevent this.
So a c460 m4 with v2 xeons and ddr3 should take v4 xeons out of the box? Is there any way to get BIOS updates for these?
 

MichalPL

Active Member
Feb 10, 2019
189
25
28
Ok confirmed, DL580 G9 8894v4 is working with DDR3 too. Almost identical speed on DDR4 (2133MHz @ 1600MHz CL12 "octa chanel") and DDR3 (1866MHz @ 1333MHz CL9 "octa chanel") same single core speed.
 

nutsnax

Active Member
Nov 6, 2014
247
92
28
113
Ok confirmed, DL580 G9 8894v4 is working with DDR3 too. Almost identical speed on DDR4 (2133MHz @ 1600MHz CL12 "octa chanel") and DDR3 (1866MHz @ 1333MHz CL9 "octa chanel") same single core speed.
What BIOS version?
 

seanneko

New Member
Feb 1, 2022
17
5
3
Ok confirmed, DL580 G9 8894v4 is working with DDR3 too. Almost identical speed on DDR4 (2133MHz @ 1600MHz CL12 "octa chanel") and DDR3 (1866MHz @ 1333MHz CL9 "octa chanel") same single core speed.
Is this by cross flashing a G8 SPI board with G9 bios, or is it working with DDR3 out of the box without needing to do anything?
 

MichalPL

Active Member
Feb 10, 2019
189
25
28
What BIOS version?
System ROMU17 v2.76 (10/21/2019)

Is this by cross flashing a G8 SPI board with G9 bios, or is it working with DDR3 out of the box without needing to do anything?
I just removed all eight DDR4 RAM cassettes and insert the DDR3 one from dl580 G8, and vice versa ;)

G9 is running on DDR3 without any problems, G8 not running on DDR4 (stop at memory checking and display that this memory type is incompatible)
 

seanneko

New Member
Feb 1, 2022
17
5
3
I just removed all eight DDR4 RAM cassettes and insert the DDR3 one from dl580 G8, and vice versa ;)

G9 is running on DDR3 without any problems, G8 not running on DDR4 (stop at memory checking and display that this memory type is incompatible)
Nice! I'm tempted to buy a Gen9 SPI board and some v4 Xeons and install them into my Gen8. Seems like it's still not clear what works and what doesn't work though... Some people said that for them it had compatibility issues. Wonder if it's the specific model of CPU that does it.