Mezzanine and Backplane Connectors

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
A few tests and some initial results:

Test 1 - 1 Blade, 1 PSU, 1 Gbe pass-through module - no OA module, no Fans
- psu turns on and supplies 12V to the chassis, psu led green
- blade power led amber -> standby
- no signs of life in the pass-through module or Insight display (was to be expected w/o OA)
- any attempt to power on the blade results in blade health led blinking red -> critical condition and psu shutting down (led going from green to amber -> power supply failure)

Test 2 - 1 Blade, 1 PSU, 1 Gbe pass-through, 4 fans (minimum config) - no OA
- psu turns on - > green led
- blade power led amber -> standby
- all 4 fans run at full blast
- no signs of life in pass-through or Insight
- any attempt to power on blade results in blade health led blinking red -> critical condition
- psu stays on -> led green

Test 3 - 1 Blade, 1 PSU, 1 Gbe pass-through, OA module installed - no fans
- psu turns on - > green led
- blade power led amber -> standby
- Insight display turns on showing fan subsystem failure, wants at least 4 fans installed
- Blade health led blinks red even without trying to power it on - > critical condition

Test 4 - 1 Blade, 1 PSU, 1 Gbe pass-through, OA module and 4 fans installed
- psu turns on - > green led
- blade power led amber -> standby
- Insight display turns on showing all green -> happy with the situation
- Blade powers on (led amber -> green)
- Both link and network activity on the pass-through module

So it looks like you have to bite the bullet and get the OA, unfortunately....
Oh, thank you very much! Looks like OA is a big boss there... Nothing will be done without it. But if it will not work even without fans I doubt I will benefit from acquiring OA module alone. I still will need fans and maybe something else. Basically full enclosure without enclosure :) The basic idea here is to use bare minimum. I mean really minimum :) Ok, thank you once again, need to rethink all that info. Anyways blade and midplane with switch are already coming. I'll try to play with that first before ordering something more. Let's see what I will be able to achieve though...
 

nallar

New Member
Jan 27, 2018
8
5
3
34
Been doing some reverse engineering of the OA recently, trying to set the fan speed manually.

The OA communicates with the blades using i2c and serial ports.

You can unpack the firmware image using binwalk (matroshka mode, as there are nested images).

On boot of the OA `/etc/rc.transition2` starts some HP services and sets the stage numbers on the insight display.

it runs `/etc/diagcheck`. It checks for a specific device on the i2c bus, a "HP Diagnostic Blade".
If found it will enable root shell access with the username "udog" and the password of the Administrator account.

Code:
Manufacturing diagnostics blade detected.  Allowing shell access.
/etc/grantroot
/usr/sbin/allfan 180 > /dev/null 2>&1
Reverse engineering of `/sbin/mgmt` would allow finding what it sends over i2c, it should be possible to use a cheap microcontroller then instead of the OA. Would also need to work out which pins are for i2c.
 
Last edited:
  • Like
Reactions: tim.yoshi

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Been doing some reverse engineering of the OA recently, trying to set the fan speed manually.

The OA communicates with the blades using i2c and serial ports.

You can unpack the firmware image using binwalk (matroshka mode, as there are nested images).

On boot of the OA `/etc/rc.transition2` starts some HP services and sets the stage numbers on the insight display.

it runs `/etc/diagcheck`. It checks for a specific device on the i2c bus, a "HP Diagnostic Blade".
If found it will enable root shell access with the username "udog" and the password of the Administrator account.

Code:
Manufacturing diagnostics blade detected.  Allowing shell access.
/etc/grantroot
/usr/sbin/allfan 180 > /dev/null 2>&1
Reverse engineering of `/sbin/mgmt` would allow finding what it sends over i2c, it should be possible to use a cheap microcontroller then instead of the OA. Would also need to work out which pins are for i2c.
Wow! That approach is quite awesome! Can you share all the source code or file which you are reverse engineering? I have some experience with micro-controllers and my brother is a programmer. Together I hope we can do something about it...
 

nallar

New Member
Jan 27, 2018
8
5
3
34
You can extract the firmware images using binwalk with `binwalk -e -M hpoa480.bin`.

I was reversing the binaries using binary ninja. It's not free but it's a lot cheaper than IDA and pretty good.

Radare2 is completely free and you might be able to use it instead if you want to try that.

The c3000 OA has multiple I2C muxes and I2C switches. The circled 0x75 mux is the one which diagcheck talks to.



I've just mucked up the board trying to solder fine wires onto that I2C bus to snoop on it. Can't find any solder wick so can't run the bladesystem until some arrives. Oops. :(
 
  • Like
Reactions: tim.yoshi

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
You can extract the firmware images using binwalk with `binwalk -e -M hpoa480.bin`.

I was reversing the binaries using binary ninja. It's not free but it's a lot cheaper than IDA and pretty good.

Radare2 is completely free and you might be able to use it instead if you want to try that.
Sorry but I don't have OA module, that's why actually I'd like to imitate it :)

I've just mucked up the board trying to solder fine wires onto that I2C bus to snoop on it. Can't find any solder wick so can't run the bladesystem until some arrives. Oops. :(
Oh, that sucks. I always keep spare solder wire.
 

nallar

New Member
Jan 27, 2018
8
5
3
34
Got it working again and found a way to get root access without tinkering with hardware.

If HP had a bug bounty program I would report this exploit to them, but they do not.

As I now have root (and can set fan speeds) that's all I needed... but I am happy to provide dumps of what communications the OA is doing to help out. You will have to figure out which pins to use yourself though.
 
  • Like
Reactions: tim.yoshi

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Got it working again and found a way to get root access without tinkering with hardware.

If HP had a bug bounty program I would report this exploit to them, but they do not.

As I now have root (and can set fan speeds) that's all I needed... but I am happy to provide dumps of what communications the OA is doing to help out. You will have to figure out which pins to use yourself though.
That's cool! Thanks for your work. So could you share all that knowledge with us here?
 

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Hi im to planing to build a custim middplane, did you cam anny further with it?
Hi there.
Sorry, but you know, just as with a lot of such a projects, I had to put it on hold for now. Had many other urgent business. I had blade on hand along with a midplane and Cisco switch, tied it all together once, tried to figure out pinout, but abandoned it quite fast. I'm still on plan to return to it, I'm for sure will not let it sit on hold for a long time, but still I doubt it will be earlier than this winter.
And can you share, do you had any further progress?
 

TomasHC

New Member
Oct 23, 2018
19
0
1
Hello, finally I found something relevant to my project of standalone BL460c server :) I'm also digging into this, at least tried to hardwire ILO network, but haven't got NW link yet - have to check with oscilloscope what's happening on the wires. Maybe there's some standby power for ILO, which is not enabled by default. Also something like blade presence detection via backplane connector. I have to check linked docu (Technologies in the HP BladeSystem c7000 Enclosure.pdf) and try to identify what peripherals are routed via PCIx, directly (I hope for ILO NW), SerDes, I2C.

I wasn't able to power on standalone BL460c G7, but G6 is working fine. My goal is to reach the ILO and at least one gigabit. Also would be nice to allow WOL or power control via ILO (if hardwired). I hope the "power button" is also routed to backplane. In a worst case, I should control Enable pin on external server PSU via blade's power button via some latch circuit.
20181014_231756.jpg
 

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Hello, finally I found something relevant to my project of standalone BL460c server :) I'm also digging into this, at least tried to hardwire ILO network, but haven't got NW link yet - have to check with oscilloscope what's happening on the wires. Maybe there's some standby power for ILO, which is not enabled by default. Also something like blade presence detection via backplane connector. I have to check linked docu (Technologies in the HP BladeSystem c7000 Enclosure.pdf) and try to identify what peripherals are routed via PCIx, directly (I hope for ILO NW), SerDes, I2C.

I wasn't able to power on standalone BL460c G7, but G6 is working fine. My goal is to reach the ILO and at least one gigabit. Also would be nice to allow WOL or power control via ILO (if hardwired). I hope the "power button" is also routed to backplane. In a worst case, I should control Enable pin on external server PSU via blade's power button via some latch circuit.
View attachment 9521
You could made it simpler by connecting directly to back connector.
I made found with multimeter that iLO network is directly wired to connector. Here is the pins:

A 10 - iLO Rx-
B 10 - iLO Rx+

A 9 - iLO Tx-
B 9 - iLO Tx+

Pin naming as per follows:
blade_connector_pinout_wip1.jpg

But I have not powered it yet. Not even tested. Just when I was testing connections with probe I had an emergency and abandoned that project for awhile. As I said I plan to go back to it, but not now, maybe in few months.
Also from what I know from now from internet and all this documents - you can't really directly connect to onboard LAN - it is going by SerDes chips and I believe they are quite proprietary. Way easier is to buy it's supplemental Cisco gigabit switch which will do the job just fine. It cost some really small price. here is the link: WS-CBS3020-HPQ | eBay
The trick is how to power it on. I believe I found the necessary info, but as i said i never got a chance to tinker with it for a enough time.
Also I doubt power button is wired to back :) I guess they made it in mode advanced fashion by transistors or even i2c, or even more proprietary with iLo.

So the bottom line - I'd also like to be able to turn it on or off remotely with the iLo, but from early test made here by fellow members I believe this will be complicated - iLo will be very upset about absence of it's native fans, OA module and possibly signals from PSUs. If one will be able to supplement all that data to it ( i believe it all can easily be faked even with arduino, but one should clearly know WHICH EXACTLY data to fake) it could work this way. But where we'd take from all that data?
 

TomasHC

New Member
Oct 23, 2018
19
0
1
Thanks for sharing the ILO pinout. I wasn't so brave to check with multimeter every pin - afraid of killing some sensitive parts. I just checked mainboard of 460c G7, and can't see some extra chips responsible for SerDes conversion. BCM57711 is native PCI-x, same as QMH2562 in Mezzanine slot. Only in case they natively support serialization, but who can confirm, while datasheets are not available.

If there will be strong team, everything is possible - from modifying ILO FW to tinkering PCI-x slot NIC to mezzanine port :) I did few times grabbing I2C stream (Saleae Logic) and simulating it with Arduino, but what we can do is decode the stream, but who knows the syntax, commands... Looks like a playground for old hackers with no personal life or/and kids :)
 

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Thanks for sharing the ILO pinout. I wasn't so brave to check with multimeter every pin - afraid of killing some sensitive parts.
It's a server) you just can't kill it :)

I just checked mainboard of 460c G7, and can't see some extra chips responsible for SerDes conversion. BCM57711 is native PCI-x, same as QMH2562 in Mezzanine slot. Only in case they natively support serialization, but who can confirm, while datasheets are not available.
If I recall correctly I tried to calculate all possible combinations and came to conclusion that there is indeed SerDes in there - they are connecting to switches and by 4 wires only. And it's Gigabit connection which is not possible with 4 wires - only 8 in conventional copper twisted pair. If used 8 pairs - then there will be no space left in it's 100 contact connector, but we know for sure that there is a lot of space in fact used by something else. So SerDes takes place for sure. Also it is drawn this way in documents - iLO - plain old good 4 copper Ethernet twisted pair for 100MB connection and special highspeed connection for two onboard networks and plus reserved for two or more possible external NICs.

If there will be strong team, everything is possible - from modifying ILO FW to tinkering PCI-x slot NIC to mezzanine port :) I did few times grabbing I2C stream (Saleae Logic) and simulating it with Arduino, but what we can do is decode the stream, but who knows the syntax, commands... Looks like a playground for old hackers with no personal life or/and kids :)
Yeah, sure :)
 

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
Also if it could be beneficial or provide some insights I have made comprehensive electrical testing of it's mighty 100 pin connector in relation to the common GND - i.e. resistance, capacitance and diode test. Here in table are the results. if cell is empty - then there was "nothing" .
While patterns is clearly visible yet results interpretation is yet a good question.

Here is the table:
blade_pinout.xlsx
 

TomasHC

New Member
Oct 23, 2018
19
0
1
So far it looks good :) ILO working, remote power off too, power on unfortunately not. Console didn't work on W10/IE11, but there's quite old ILO FW (from 2011), but I'm a bit afraid to update...
I need to enhance cooling, then will do more tests. Also waiting for the SUV cable.20181026_235440-small.jpg ilo-summary.png ilo-temp.png
 

tim.yoshi

Member
Jun 25, 2017
40
0
6
36
Kiev, Ukraine
So far it looks good :) ILO working, remote power off too, power on unfortunately not. Console didn't work on W10/IE11, but there's quite old ILO FW (from 2011), but I'm a bit afraid to update...
I need to enhance cooling, then will do more tests. Also waiting for the SUV cable.

OMG! Really? I thought it was not possible without pleasuring demanding iLO with many different OK signals.

Yeah. you better take off the lid and blow directly on CPU's - your current set-up for sure will not do the job for a long time.

But why it won't on? that's strange... and how about power cycle? Could you just reboot it instead of power off?

Overall it's a great success! really. congrats. I wish I also had time for tinkering with it now :(