[Solved], [Documented]: C6100 Nodes - Can't set BMC IP, BMC broken

DavidRa

Infrastructure Architect
Aug 3, 2015
272
122
43
Central Coast of NSW
www.pdconsec.net
I'm powering up the second C6100 chassis and nodes I bought back in September this year. The first went perfectly well - all 4 nodes got BMC/IPMI addresses, booted, installed OS and are now running production. The second is a ... difficult birth, shall we say.

So far node 2 is fine. Node 1 I can't set an IPMI address in the BIOS - if I type in an address, it's lost straightaway and not applied. There's a thread in the Dell forums that looks appropriate, but it seems risky - has anyone tried it, or another procedure?

Node 3 is also a problem child, same problem with setting IPMI address and the bonus of a reboot every time it gets halfway through booting WinPE so the fix above is going to be tough. I'm thinking there's probably a HW fault of some type, but without an OS and IPMI, troubleshooting is rather difficult, so alternative suggestions are welcome.

Actually Node 3 and 4 show the same symptom setting BMC - upon entering BIOS Setup, the BMC is shown with the right version, and all is fine until I try to set an IP - then the BIOS basically hangs (or very close to it) and the BMC shows Wrong Channel.
 
Last edited:

azev

Active Member
Jan 18, 2013
738
211
43
I've had similar problem with mine, I ended up just using static dhcp reservation to go around the issue.
 

DavidRa

Infrastructure Architect
Aug 3, 2015
272
122
43
Central Coast of NSW
www.pdconsec.net
Right. Here's how you fix it (cobbled info together from forum posts, Google searches etc).

WARNING: Running the following procedure with the wrong BIOS or BMC can, and PROBABLY WILL permanently brick your node. Running the commands below can permanently brick your node ANYWAY if you look at it the wrong way.

To continue, open Notepad or another text editor. Create a new document and type in this text:

I understand that what I am about to do can permanently destroy, mutilate, damage and render into a useless pile of boards, components and metal, this node, the other nodes in the chassis, all the nodes in my rack or racks and the powered off computers on the floor. It may also shave my cat, give my dog diabetes, cause my car to rust and a meteorite to crash through the roof of my house, turning me into a wheelchair-bound quadriplegic.

Done that? OK save it to your desktop as "I DIDNT LISTEN TO THE INSTRUCTIONS.TXT". Let's begin.

You will need:
BMC and BIOS ROMs
Product Support | Dell US

FreeDOS Images
Index of /pub/FreeDOS/bootable-usb/
I used: http://ftp.chtaube.eu/pub/FreeDOS/bootable-usb/FreeDOS-1.1-memstick-3-30M.img.bz2

Win32 Disk Imager
http://internode.dl.sourceforge.net...ger/Archive/Win32DiskImager-0.9.5-install.exe

PLD Rescue CD
http://rescuecd.pld-linux.org/beta/RCDx86_13_03_10.iso

USB Key
At least 32MB in size, and it's going to be completely wiped, so back up NOW. 256MB is better, in which case you can use one of the larger FreeDOS images.

1. Setup
  • Download FreeDOS boot image, Image Writer, ROMs, PLD Linux image
  • Install Win32DiskImager and write the FreeDOS image to USB key
2. Use SOCFLASH to flash the BMC
  • Copy extracted files from Dell BMC download to USB key "BMC" directory - use the "Floppy" version, File Name: PEC6100BMC133.exe
  • Boot FreeDOS USB key
  • Specify A:\FDOS\COMMAND.COM /E:1024 /P for the location of the command interpreter, if prompted by FDOS boot code
  • Run these commands:
    • cd BMC\SOCFLASH\DOS
    • SOCFLASH -sf ..\5442M133.ROM
  • Remove power from chassis for 30 seconds, then plug back in and continue
  • This process will reset all BMC configuration (e.g. Shared/Dedicated NIC) and may ZERO the MAC (see step 4 to fix that).
3. Use AFUDOS to flash BIOS
  • Copy extracted files from Dell BIOS download to USB "BIOS" directory - use the "Floppy" version, File Name: PEC6100BIOS017100.exe
  • Boot FreeDOS USB key
  • Specify A:\FDOS\COMMAND.COM /E:1024 /P for the location of the command interpreter, if prompted by FDOS boot code
  • Run these commands:
    • cd BIOS
    • AFUDOS 6100v171.ROM -x -reboot
4. Use PLD to force IPMI MAC
  • Boot PLD CD image (I used my IP KVM, you could also try UNetBootIn, or real media ...)
  • Run these three commands to load IPMI drivers
    • modprobe ipmi_devintf
    • modprobe ipmi_si
    • modprobe ipmi_msghandler
  • Decide your new MAC address (in the form: NN:NN:NN:NN:NN:NN) and convert each hex digit and colon to its ASCII equivalent. I matched the first 8 digits to other BMC MAC addresses I have, then numbered from 02:00 to 02:03 based on the chassis and blade number. It's "Good Enough" (if you have hundreds of nodes and possible conflicts, you can easily choose your own scheme). Here's the mapping table you need:
    • 0x30 == "0"
    • 0x31 == "1"
    • 0x32 == "2"
    • 0x33 == "3"
    • 0x34 == "4"
    • 0x35 == "5"
    • 0x36 == "6"
    • 0x37 == "7"
    • 0x38 == "8"
    • 0x39 == "9"
    • 0x61 == "A"
    • 0x62 == "B"
    • 0x63 == "C"
    • 0x64 == "D"
    • 0x65 == "E"
    • 0x66 == "F"
    • 0x3a == ":"
  • Convert your MAC address into a series of hex values separated by spaces, and end it with 0x00. For example:
    • 00:14:25:8d:30:7f
  • Becomes this string which I'll call $MYDATA (yes you could set environment variables etc)
    • 0x30 0x30 0x3a 0x31 0x34 0x3a 0x32 0x35 0x3a 0x38 0x64 0x3a 0x33 0x30 0x3a 0x37 0x66 0x00
  • Run the following command:
    • ipmitool lan print 1 | grep "MAC Address"
      This should show the current address is 00:00:00:00:00:00
  • Run the following command, substitute your own numbers for $MYDATA:
    • ipmitool raw 0x2e 0x21 $MYDATA
    • ipmitool raw 0x2e 0x23 $MYDATA
    • ipmitool mc reset cold
  • Wait 30s
  • Run the following command:
    • ipmitool lan print 1 | grep "MAC Address"
      This should show your shiny new MAC address
  • Reboot, do your thing (install your OS, switch NIC to dedicated NIC etc)
References:
PLD LINUX Rescue CD
Dell C6100 BMC / BIOS HELP - Cloud Services - Forum - Cloud Services - Dell Community
Dell C6100 BMC / IPMI Mac address all zeros after update? 00:00:00:00:00:00 | Web Hosting Talk
ipmi_on_linux [Adam Sweet's Wiki]
IPMI HOWTO for Debian GNU/Linux on the Intel SR2300 (Server Board SE7501WV2)
 
Last edited: