Yet another problem with IPMI (Asus Z9PE-D8 WS & ASMB6-iKVM)

[grad]

My Asus ASMB6-iKVM on my Asus Z9PE-D8 WS motherboard does not work when I turn off the system. When the server is turned on I can ping and login to the Web-based UI, check sensors, etc but as soon as I shutdown the server I lose connection for good.

I noticed that, when the server is on, I can keep pinging to its ASMB6 remotely, when I turn it off, it loses 2 ICMP requests, then it responds to 2 more requests, then nothing, I just get request timeouts.

1) I have tried with setting static IPs for the ASMB6 at the Z9PE-D8 WS motherboard's BIOS and with DHCP.
2) I have tried flashing the BIOS/firmware for both Z9PE-D8 WS and ASMB6 with the latest versions, 5802 and 1.16 respectively.
3) I have tried resitting the ASMB6-iKVM board multiple times
4) I have tried with different ethernet cables and an alternative/cheap Netgear switch with nothing but the LAN1/IPMI port of the Asus Z9PE-D8 WS server and my client computer.
5) I have tried placing ASMB6's IP/MAC on my client's cache using "arp -s", as suggested here.
6) I do not know how to enable "gratuitous ARPs" on the ASMB6-iKVM (if that is the problem)

Anyone has a clue what could I be doing wrong ?

 

[RolloZ170]

you may have misconfigurated the system.
set all to enable WOL, this lets the LAN port powered at power down/standby state.

 

[grad]

you may have misconfigurated the system.
set all to enable WOL, this lets the LAN port powered at power down/standby state.

There are no WOL options in the BIOS. At the APM setting I have enabled all, Power On By PS2 Keyboard/PS2 Mouse/PCIE/Ring/RTC.

The Asus Z9PE-D8 WS motherboard has 2 Intel WG82574L Gigabit ports, showing as enp7s0 and enp7s1 on the host OS (Proxmox 7.2-1). One of them is used as the LAN for the IPMI BMC network, getting the ASMB6-iKVM MAC address.

I can only check WOL on the LAN ports at the host/Proxmox (for both it is enabled, "Wake-on: g").

I can ssh to ASMB6 IPMI but it runs a read-only busybox environment. It shows lo, eth0, sit0, ip6tnl0 interfaces.

The problem is that once I shut down the system the IPMI board is not available, I can not ping or access it, the BMC_LED1 stops blinking too.

 

[RolloZ170]

I can only check WOL on the LAN ports at the host/Proxmox (for both it is enabled, "Wake-on: g").

thats what i meant.

the BMC_LED1 stops blinking too.

then the ASPEED is unpowered.

 

[RolloZ170]

your PSU may have insufficient +5V standby power.
#Manual#
2. Baseboard Management Controller LED (BMC_LED1)
The BMC LED works with the ASUS ASMB6 management device and indicates its initiation status.
When the PSU is plugged and the system is OFF, ASUS ASMB6 management device starts system initiation for about one (1) minute.
The BMC LED blinks after system initiation finishes

 

[grad]

My PSU is a 1200W Corsair, I doubt that it is the culprit but you never know, I will try with another PSU.

The BMC LED works with the ASUS ASMB6 management device and indicates its initiation status.
When the PSU is plugged and the system is OFF, ASUS ASMB6 management device starts system initiation for about one (1) minute.
The BMC LED blinks after system initiation finishes

When the PSU is plugged and the system off my Asus ASMB6 does not initiate and certainly doesn't ever blink as long as I leave it. If I turn on the system then sure, it initiates and starts blinking after half a minute and is accessible to ping. When I turn off the computer then all is gone.

 

[RolloZ170]

When I turn off the computer then all is gone

then your PSU don't have +5VStandby thought. very old caps or damaged PSU.

 

[grad]

I checked with another PSU, Corsair AX1200i. ASMB6 only works when the computer is on. When I shut the computer down, it blinks/responds to pings 2-3 more times then IPMI is not available remotely anymore.

 

[RolloZ170]

I checked with another PSU, Corsair AX1200i. ASMB6 only works when the computer is on. When I shut the computer down, it blinks/responds to pings 2-3 more times then IPMI is not available remotely anymore.

something bad on the motherboard.

 

[grad]

I hope it is something else, I can not afford to get a replacement. I will keep trying.

Thanks for your time anyway.

 

[grad]

OK, I guess I have resolved the issue. It was not the PSU. Being desperate, I tried many things, including changing cables, switches, etc. I am just too happy right now to care about the origin of the problem or the solution, it might have been more than 1 problem at the same time (isn't it always ?). I didn't always have the patience to disconnect power, wait a minute and cold boot when I changed things. For sure there was a stupid MAC address and DHCP misconfiguration on my part at my OPNsense router.

1) I stayed with DHCP for the IPMI, being worried after reading this. I am not sure if it applies but anyway.
2) ASMB6-iKVM is RMCP+ capable but it only supports up to 12 (0,1,2,3,6,7,8,11,12) Cipher Suites. In order for me to use lanplus interface (-I lanplus) with ipmitool I had to add -C 8 as an argument.
3) I do not know if gratuitous ARPs are required for the BMC. I enabled them e.g. with

ipmitool -H 192.168.0.5 -U admin -P admin lan set 1 arp generate on

 

[RolloZ170]

and the BMC heartbeat LED is flashing now in standby as mentioned in the manual ?

 

[grad]

and the BMC heartbeat LED is flashing now in standby as mentioned in the manual ?

@RolloZ170 sorry for the delayed reply.

Yes, it now does. Apparently, it was a router (OPNsense) or switch (Zyxel XGS1250-12) misconfiguration or incompatibility rather than a hardware (motherboard, IPMI, PSU, etc) issue (but a few other matters, e.g. an incompatible RAM module, made debugging a PITA). For now, I am setting a static IP for the IPMI on a different subnet than my client. I can ping and monitor the Asus Z9PE-D8 server while it is turned off and I can cold boot and power cycle it remotely.

What hasn't worked so far is remote controlling it using the Java console. What works for my Asus Z10PA-D8/ASMB8 server, i.e. using ipmi-kvm-docker, does not seem to work for Asus Z9PE-D8/ASMB6, I am getting a blank screen.

 

[RolloZ170]

check this, also very i nteresting:

https://forums.servethehome.com/index.php?threads/power-supply-vs-bmc.37967/

 

[grad]

check this, also very i nteresting:

https://forums.servethehome.com/index.php?threads/power-supply-vs-bmc.37967/

Thanks, maybe it was that.