Patrick

How to fix: BMC can't force ME to recovery mode when updating BIOS

Patrick

Administrator
Staff member
Dec 21, 2010
12,382
5,529
113
Patrick submitted a new resource:

How to fix: BMC can't force ME to recovery mode when updating BIOS - Tested on a Supermicro motherboard

I recently ran into an issue where I purchased a Supermicro license key for remote BIOS upgrade only to find that the system rebooted to "System initializing" and post code 19. Normally the remote BIOS update key allows you to update BIOS remotely without issue, even when you have some strange POST code failure. Instead, the remote flash would fail and I saw this message:


View attachment 5265
Here was the fix:
  • Power off system completely
  • Unplug system so BMC turns off...
Read more about this resource...
 
  • Like
Reactions: i386 and PigLover

i386

Well-Known Member
Mar 18, 2016
3,397
1,141
113
33
Germany
If this doesn't work for you, try to update bios with SUM* and the "--force_update" parameter:

Code:
./SUM -i 192.168.0.107 -u ADMIN -p ADMIN -c UpdateBios --file <PATH TO BIOS FILE> --force_update --reboot
*executable is shipped with the superdoctor package, download from supermicro ftp is possible
 

Panagiotis

New Member
Nov 5, 2019
15
3
3
If this doesn't work for you, try to update bios with SUM* and the "--force_update" parameter:

Code:
./SUM -i 192.168.0.107 -u ADMIN -p ADMIN -c UpdateBios --file <PATH TO BIOS FILE> --force_update --reboot
*executable is shipped with the superdoctor package, download from supermicro ftp is possible
You rock. I just recovered the BIOS on my X10dri with this method, also suggested by Supermicro in their FAQs. I tried Patrick's method first, but hadn't considered setting Restore on AC loss to Stay Off before flashing, so that didn't work. Thank you for the reference, bookmarked :)
 

sko

Member
Jun 11, 2021
94
53
18
Thanks for this info!

I just had a X10DRU-i+ here that didn't POST after flashing the latest BIOS (2 similar systems were upgraded last week without any issues..)

Some additional findings: In case SUM will give you this error

Code:
********************************<<<<<ERROR>>>>>*********************************

ExitCode                = 148
Description             = IPMI execution exception occurred
Program Error Code      = 216.55
Error message:
    Other user is updating. Please try again later.

********************************************************************************
Try another (newer) Version of SUM - I had SUM v2.3 lying on my disk which didn't work; the current version 2.8 was able to flash the BIOS.
Also try changing the "ME manufacturing mode select" jumper position. Strangely with the manufacture mode enabled I couldn't flash via SUM (or the web-IMPI), setting the jumper back to 1-2 worked with SUM.
 

oneplane

Active Member
Jul 23, 2021
305
164
43
If I'm not mistaken this is because the ME update and some other classes of signed updates can only be pre-loaded after which a system reset has to happen before the firmware will read the pre-loaded update capsule and then start the update process itself. On top of that, to pre-load an ME update, the ME has to be switched to update mode, reset, then preloaded, then reset again. This mostly happens when a firmware update contains a large jump in ME versions or updater versions because it will do a prep+update first and then once the system resets the ME hasn't gone into update mode yet and can't start the update, but the new firmware expects the new ME firmware so it can't boot either.
 

sko

Member
Jun 11, 2021
94
53
18
The ME wasn't updated on those systems (they were/are on 3.91; 3.94 is the latest). I actually tried to fix the problem on that particular host by also updating the ME/IPMI, which didn't help with the error.
 

acquacow

Well-Known Member
Feb 15, 2017
698
379
63
41
So I'm dealing with this on an X10SRH-CF right now.


It arrived with bios 9/9/2014 Rev 1.0a and I updated to 6.5.2021 Rev 3.4
It now spends a "LONG time at DXE--BIOS SMM Initialization 79"
When posting and in the BIOS it reports:
Operational Firmware Version: N/A
ME Firmware Type: N/A
Recovery Firmware Version N/A
ME Firmware Features: N/A
ME Firmware Status #1: 0x00003401
ME Firmware Status #2: 0x00080000
Current state: Initializing
Error Code: Image Failure


I have been able to downgrade/upgrade the bios, but no matter which way I go or which flags I feed to AFUDOS, I can't seem to get the ME firmware to write.

I've tried popping the battery, but when you reset the bios, it defaults to last-state or power-on and it immediately turns on when I feed it power, so I can't do Patrick's sequence in the first post. Doesn't matter which position I have JPME2 in (there is no JPME1). The flash updaters say that it can't enter/unlock the ME Manufacturer mode.

Thoughts?

Thanks,

-- Dave
 
Last edited:

acquacow

Well-Known Member
Feb 15, 2017
698
379
63
41
OMG, saw the update up top about sum having to be 2.8 (I was using 2.1, which I use for my x9 systems... ).

Finally got ME Firmware to update!

1666149579018.png

This is still a 2014 bios, which is what the board shipped with, I guess I can try the 2021 bios now?

*crosses fingers*

Thanks for all the info in this post. It's silly that sum is the only tool that can update this.
My jumper settings are JPME2: 1-2, JBR: 1-2

-- Dave
 

acquacow

Well-Known Member
Feb 15, 2017
698
379
63
41
Yup, managed to flash right to the 2021 bios/ME just fine using sum.

1666150616808.png
 

Noob321

New Member
Nov 4, 2022
9
1
3
You rock. I just recovered the BIOS on my X10dri with this method, also suggested by Supermicro in their FAQs. I tried Patrick's method first, but hadn't considered setting Restore on AC loss to Stay Off before flashing, so that didn't work. Thank you for the reference, bookmarked :)
Hi, where do I type in this code?
 

Noob321

New Member
Nov 4, 2022
9
1
3
Hi,
Where do I type in this code. My server won't boot up after I updated Bios. There is just a blank screen. Server does have power, fans running etc.
I get error "BMC can't reset ME to operational mode" when I try update Bios again

./SUM -i 192.168.0.107 -u ADMIN -p ADMIN -c UpdateBios --file <PATH TO BIOS FILE> --force_update --reboot
 

i386

Well-Known Member
Mar 18, 2016
3,397
1,141
113
33
Germany
first you need a licensed ipmi (x12/h12 or newer have it already integrated)
then you need the sum tool, it's included with superdoctor5 (you can find it on supermicros "ftp" server)
then open a shell/command prompt (not sure if you need admin rights on windows machines) and type in that command with the ip address of the bmc of the server and your credentials
 

Noob321

New Member
Nov 4, 2022
9
1
3
Thanks, I have 2 servers that don't show any video output after I updated Bios through IPMI.
Both times it said update was successfull and asked to reboot, said Yes.
One powers up and stays up but with no video output so I can't do anything. Wanted to retry Bios update but getting "BMC can't reset ME to operational mode"
The second, only the fans turn on, red LEDS turn on on HDDS and then fans turn off, then on again and continues in a loop like that.
When I retry Bios update I get "BMC can't force ME to recovery mode"
Have tried moving jumper JPME2 to pins 2/3 on both servers (manufacture mode) but same result.
I have the SUM tool but I cannot open a command prompt because no video.
 

i386

Well-Known Member
Mar 18, 2016
3,397
1,141
113
33
Germany
I have the SUM tool but I cannot open a command prompt because no video.
This guide is for remotely updating the bios not local.
If you run it locally it will very likely render the server unbootable.

Use a laptop/another pc or server for the procedure while the target server is shut down.
 

Noob321

New Member
Nov 4, 2022
9
1
3
I am using laptop.
Stupid question... server shutdown but power cable attached or not?
 

Noob321

New Member
Nov 4, 2022
9
1
3
I am getting error saying "." is not recognized as an internal or external command.
If I have the file on my desktop am I putting in c:\desktop\filneame for Path to Bios file
Do I use the < > ?
 

Noob321

New Member
Nov 4, 2022
9
1
3
Am I supposed to run the sum executable on my laptop and then open a command prompt on my laptop and then type in command
 

Noob321

New Member
Nov 4, 2022
9
1
3
I assume the IP address to type in is the same as the address used through IPMI?
Proving I am really a NOOB here