Asrock BMC no longer accessible after SSL certificate install

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tfboy

New Member
Mar 19, 2023
17
3
3
Since purchasing an Asrock rack X570D4U-2L2T, it's been fine and the BMC IPMI has been really useful.
Today, I tried installing a certificate so have proper SSL (certificate generated from my own private PKI CA), and I must have done something wrong or it has serious QA issues as I can no longer access the web interace.

I can reach the IP over SSH (I get the Smashlite Scorpio Console), but that's about it.

Unfortunately, I'm physically away so cannot reset anything physically.

So I have two question:

1. Is there a way to reset / fix / return to the default self-signed certificate using the SSH console or do I have to wait till I have physical access to the machine again?

2. Clearly, I've had an issue with the certificate used. It was with a .pem extension for both the cert and the key and in PEM format. Are there any funny restrictions?
 

casperghst42

Member
Sep 14, 2015
118
23
18
56
When you say "not longer accessible", is that fully no access, or that the browser complains that the BMC no longer have a valid certificate?
 

RolloZ170

Well-Known Member
Apr 24, 2016
7,404
2,312
113
1. Is there a way to reset / fix / return to the default self-signed certificate using the SSH console or do I have to wait till I have physical access to the machine again?
for BMC reset without PW you need physical access. would be strange if not.
 

tfboy

New Member
Mar 19, 2023
17
3
3
When you say "not longer accessible", is that fully no access, or that the browser complains that the BMC no longer have a valid certificate?
I get an error, cannot connect. Not an invalid cert.
for BMC reset without PW you need physical access. would be strange if not.
I have the password and can login over SSH. Is there a way to reset the BMC over SSH? I've found how to reset the server but not the BMC itself.
 

tfboy

New Member
Mar 19, 2023
17
3
3
Yes, but I'm completely flummuxed how this works.
Their website tells you to install ipmitool which is straight foward, then jumps straight to that second command.
No information how you connect to it!
I've tried ipmitool -H <BMC IP address> -U admin and then entering the password, but I just get :

Code:
Authentication type NONE not supported
Error: Unable to establish LAN session
Error: Unable to establish IPMI v1.5 / RMCP session
 

RolloZ170

Well-Known Member
Apr 24, 2016
7,404
2,312
113
heir website tells you to install ipmitool which is straight foward, then jumps straight to that second command.
No information how you connect to it!
run on local machine don't need connection path(ip i.e.)
 

tfboy

New Member
Mar 19, 2023
17
3
3
run on local machine don't need connection path(ip i.e.)
Ah. OK.
Only issue, I'm running vSphere on it, so no bare metal OS access.
I guess this is serial? If I can have a passthrough serial or something... I'll give it a try.
 

tfboy

New Member
Mar 19, 2023
17
3
3
Aha.
I've had some success!
I stumbled across this site ipmitool 1.8.11 vib for ESXi
And managed to get it installed.
I ran the command and it just returned a prompt. I assumed it worked because shortly after, the IP side went down (ping dropped) and after a few minutes, came back up. So the BMC was reset! :)
But still no https interface :(
My guess is the reset doesn't restore the SSL side to the factory-installed self-signed certificate.
I might just have to wait till I can access the box physically to reset.

I have emailed Asrock support so will see what they suggest.

It's probably worth digging a little more with the ipmi, maybe another command can help me out :)
 

tfboy

New Member
Mar 19, 2023
17
3
3
Final update:
The reset was successful in that the user account and password was reset to admin / admin. I can se this with the ipmitool - if I have the wrong credentials, I get a Error: Unable to establish IPMI v2 / RMCP+ session message.

However, when I use the correct credentials, I'm still stuck:
Code:
$ ipmitool -H 192.168.18.200 -U admin -I lanplus sol activate
Password:
Error activating SOL payload: Unknown (0x18)
If I can't get this resolved remotely (looking unlikely now!), then I hope I can do a deeper reset than what I've been able to do so far as that doesn't reset the SSL certificates...
 

RolloZ170

Well-Known Member
Apr 24, 2016
7,404
2,312
113
"I was able to set the password using ipmitool locally with the command "ipmitool -I open user set password 2 [password]", cf.)"
 

twin_savage

Active Member
Jan 26, 2018
107
73
28
34
When/If you do a system reset, ensure the BMC actually has a MAC address after the reboot. I had an Asrock BMC "forget" its MAC address after an update once and all kinds of weird problems ensued until I re-established it.
 

tfboy

New Member
Mar 19, 2023
17
3
3
ok, took me a while to sort this out.
A full reset, clearing of the BIOS didn't help.
In the end, I had to do a full reflash of the BIOS. This was the only solution to this particular problem which I find a little worrying: supplying a certificate that, for some reason, it doesn't like, completely crashes the web server side to the point that a full re-writing of the flash is required and not even a full BMC BIOS reset will work.
The flashing takes a while too...
 

agross

New Member
Dec 16, 2024
1
1
3
I have a script that uses curl and the ASRock BMC API. It is invoked as a deploy hook by certbot. It likely needs adjustment for acme.sh. I consider it a hack, but it works well with a X570D4U-2L2T board.

Bash:
#!/usr/bin/env bash

set -euo pipefail

bmc=https://172.16.0.253
bmc_user=some-admin
bmc_password=secret
curl_cookie_jar="$(mktemp)"
curl=(curl --insecure --silent --show-error --fail --cookie-jar "$curl_cookie_jar" --cookie "$curl_cookie_jar")

# RENEWED_DOMAINS is set by certbot after obtaining a new cert.
for domain in $RENEWED_DOMAINS; do
  case $domain in
    example.com)
      >&2 printf "Updating cert on BMC for domain %s\n" "$domain"

      if ! token="$("${curl[@]}" \
                         --data-raw "username=$bmc_user&password=$bmc_password" \
                         "$bmc/api/session" \
                      | jq --raw-output .CSRFToken)"; then
        >&2 printf 'Login failed\n'
        exit 1
      fi

      >&2 printf 'Logged in with CSRF token %s\n' "$token"

      "${curl[@]}" \
           --header "X-CSRFTOKEN: $token" \
           --header "Content-Type: application/json" \
           --data-raw '{"uid_command":1}' \
           "$bmc/api/actions/chassis-led"

      >&2 printf '\nUID LED blinks temporarily\n'

      "${curl[@]}" \
           --header "X-CSRFTOKEN: $token" \
           --form "new_certificate=@$RENEWED_LINEAGE/fullchain.pem" \
           --form "new_private_key=@$RENEWED_LINEAGE/privkey.pem" \
           "$bmc/api/settings/ssl/certificate"

      >&2 printf '\nDone\n'
    ;;
  esac
done
 
  • Like
Reactions: scyto

scyto

New Member
Jun 22, 2024
7
1
3
I consider it a hack, but it works well with a X570D4U-2L2T board.
What you consider a hack i consider pragmatically elegant :)

thanks, that looks simple enough, i will be using the neilpang acme.sh container so i am sure i can figure it out, might have to wait for a few months as i will shrtly be out of action for 6 weeks). [edit, heck i can check certbot too]

thanks for sharing, really appreciate it

just as this might tickle you interest, i did find this after i posted, but a)super complext and b)super complex so i am glad to have what you shared
(it uses the redfish API to generate a CSR, the passes that to LE using an acme client called lego, then uploads the resulting cert [and it does it for all the machines they have stored in some asset system, very clever, too clever for me)
this is cool, that you now educated me there is a /api endpoint in addition to the /redfish/v1/CertificateService endpoint too. The API endpoint looks better as you can upload the key, the redfish endpoint doesn't seem to have that
 

scyto

New Member
Jun 22, 2024
7
1
3
thanks @agross and i was suprrised that copilot got me from his script to this, which is working (run manually after running acme.sh docker container... i will find out in Feb if it worked as --renew_hook.... i need to convert the script from bash to sh (that was 4 things and co-pilot did all of that and it suggested the code to iterate through the dirs, and showed me how to derive the domain name and certname from the directory name when it gave an example (for reference i cant write a lick of code from scratch). Next up to make this into a docker stack that uses scret for the user and the name.

I leave this here as-is here as it does the main thing for a prototype - takes certs from an acme.sh output folder and uploads it to the BMC, as i think about it more, iterating though the dirs make no sense, better to place in output dir, capture a $PWD and use that to inform that paths, will refactor it later....


Bash:
#!/bin/sh

# THIS SCRIPT SHOULD BE PLACED IN THE /ACME.SH FOLDER
set -eu

bmc=https://192.168.1.118
bmc_user=some user
bmc_password=some admin
curl_cookie_jar="$(mktemp)"
curl="curl --insecure --silent --show-error --fail --cookie-jar $curl_cookie_jar --cookie $curl_cookie_jar"

#at the moment it iterates for the subdir and processes files in every folder with _ecc - either this should be changed to look for a specifc folder called by the --renew-hook command, or by a variable, bur for now this is what it is, or the script could be place in the _ecc folder and called by hand - i need to think about that

for dir in */; do
#  if [ -d "$dir" ]; then
if [ -d "$dir" ] && echo "$dir" | grep -q "ecc"; then
    echo "Directory: $dir"

    # Extract the base directory name without the trailing slash and "_ecc"
    base_dir=$(basename "$dir" _ecc)

    printf "Updating cert on BMC for domain %s\n" "$base_dir" >&2

    token=$($curl --data-raw "username=$bmc_user&password=$bmc_password" "$bmc/api/session" | jq --raw-output .CSRFToken)
    if [ -z "$token" ]; then
      printf 'Login failed\n' >&2
      exit 1
    fi

    printf 'Logged in with CSRF token %s\n' "$token" >&2

    $curl --header "X-CSRFTOKEN: $token" \
          --header "Content-Type: application/json" \
          --data-raw '{"uid_command":1}' \
          "$bmc/api/actions/chassis-led"

    printf '\nUID LED blinks temporarily\n' >&2

    $curl --header "X-CSRFTOKEN: $token" \
          --form "new_certificate=@./$dir/fullchain.cer" \
          --form "new_private_key=@./$dir/${base_dir}.key" \
          "$bmc/api/settings/ssl/certificate"

    printf '\nDone\n' >&2
  fi
done