JBOD adventures with Dell PERC H710P Mini (LSI 2208)

Discussion in 'RAID Controllers and Host Bus Adapters' started by Malvineous, Sep 29, 2018.

  1. Malvineous

    Malvineous New Member

    Joined:
    Sep 28, 2018
    Messages:
    4
    Likes Received:
    1
    Hi all,

    I've recently gotten hold of a Dell PowerEdge R720 with a PERC H710P Mini controller. Wanting to try out ZFS I had hoped to make the individual drives available to the host (as JBOD/passthrough drives) but sadly the Dell firmware does not support JBOD for this controller. Other cards based on the same LSI 2208 "Thunderbolt" controller do, so after reading a number of posts about people who had talked about flashing alternate firmware on the card (but none that I could see who had tried), I thought I would give it a go.

    Unfortunately, none of the official LSI firmwares seem to work. They flash successfully but the card boots up with a firmware failure and no drives are seen. I haven't tried firmwares from other manufacturers, but the purpose of this post is to explain how to recover from a failed flash, in case anyone else wants to give it a go with other firmwares.

    So, to recover the H710 you will need:

    1. A bootable USB stick with the LSI MegaRAID recovery image on it.

    The instructions for making the bootable USB under Linux didn't work for me, so I had to manually create a bootable USB with SYSLinux and copy across all the configs and programs. Make sure you get this to work before starting! Since rebooting the R720 is quite slow, you can test it out locally with QEMU first:

    Code:
    qemu-system-x86_64 -hda /dev/sdx
    
    If it boots (and you can run the MegaRAID commands and just get errors about no controllers found) then you're good to go.

    They have a bootable ISO as well which would've been a lot easier, but this didn't work for me because they don't load any CD drivers, so there's no drive letter with the firmware on it after you've booted so that was completely useless.

    2. A copy of the latest Dell firmware for the PERC.

    I downloaded mine from the Dell support site. Put in your service tag and change the category to "SAS RAID". In my case, the download was called "Dell PERC H710P Mini Monolithic Adapter firmware release 21.3.5-0002". The .exe download can be extracted under Linux with "unzip", and contains a file called FW8148PM.rom. This is the file you will need. Make sure it's accessible from the USB stick.

    3. Patience. If the wrong firmware is flashed, the system will take about 30 mins to boot, and will appear stuck on 'initializing firmware'. Just give it time and it will boot eventually. With the wrong firmware loaded, none of the usual tools will be able to see the card. Only the LSI recovery mode tool will recognise and be able to reflash it.

    In order to do the recovery for the H710P, you'll need to boot the USB, go into the MegaRAID menu and then boot the recovery image.

    dell-h710-2-lsi_recovery.png

    In my case the USB drive was drive C, adjust as needed. Once booted, you can:

    Code:
    set path=a:\;a:\fdos;a:\fdos\bin   # Needed to avoid "DOS4GW not found" error
    c:
    cd\boot\lsi\fw\recover
    # Try the LSI ROM (this flashes but doesn't work)
    megacli -m0flash 0 ..\9286\mr2208fw.rom
    
    # Go back to the Dell ROM
    megacli -m0flash 0 \fw8148pm.rom
    
    Note that although the command here is "megacli", it's actually megarec.exe that has been renamed for some reason.

    The LSI ROM does flash:

    dell-h710-1-2208_flash.png

    But during boot it still says there was a FW failure. Linux dmesg says this:

    Code:
    megaraid_sas 0000:02:00.0: Waiting for FW to come to ready state
    megaraid_sas 0000:02:00.0: FW in FAULT state!!
    megaraid_sas 0000:02:00.0: FW restarted successfully from megasas_init_fw!
    megaraid_sas 0000:02:00.0: Waiting for FW to come to ready state
    megaraid_sas 0000:02:00.0: FW in FAULT state!!
    megaraid_sas 0000:02:00.0: Failed from megasas_init_fw 5583
    
    Flashing the Dell image works fine though and gets everything back to normal:

    dell-h710-3-dell_recovery.png

    After restoring the Dell FW, I didn't even lose my RAID config. But looks like I'll have to be content with two RAID0 virtual disks, each with only one physical disk, instead of true JBOD mode.

    If anyone else decides to have a go with firmwares from other manufacturers, please let me know! (Also if you're reasonably certain of one that might work, let me know why and where it is and I might give it a go. I'd be more eager except it's such a time consuming process, having to wait 30 mins for the server to boot!)
     
    #1
    fohdeesha likes this.
  2. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,019
    Likes Received:
    236
    Check the posts by @vanfawx here in the forums. He posted in detail on how to flash lsi 9270-8i (2208 chip) RAID cards to 2308 HBA like 9207-8i as well as back to MegaRAID FW. Since yours is a Dell branded card, not sure if that has any implications. I believe he also has a blog post on the interwebs.
     
    #2
  3. Malvineous

    Malvineous New Member

    Joined:
    Sep 28, 2018
    Messages:
    4
    Likes Received:
    1
    Hmm, very interesting. It seems the Dell branded 2208 cards are not compatible with the LSI ones. One of the many differences seems to be that the SAS BIOS is part of the main system BIOS in a Dell, while with the other cards it's part of the option ROM on the card itself. I am assuming this because even after you reflash (or indeed, completely wipe) the "BIOS" component on the card, you still get the original POST message about pressing Ctrl+R to configure the card.

    However if the LSI 2208 and 2308 firmwares are compatible, I wonder whether there's a Dell-branded 2308 card that I could grab the firmware from? Any tips on which Dell cards use the 2308 chipset?
     
    #3
  4. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,019
    Likes Received:
    236
    I believe 9217-8i is Dell's nomenclature. You should be able to grab the latest from LSI/Avago/Broadcom support section. If you use megarec and clear the FW and BIOS, It should just behave like a plain HBA.

    smart @$$ in me wanted to post this link at first:
    LMGTFY
     
    #4
  5. Malvineous

    Malvineous New Member

    Joined:
    Sep 28, 2018
    Messages:
    4
    Likes Received:
    1
    Haha I did do some Googling before replying but all I could find are the same links you posted.

    I'm reasonably sure those ones won't work, because the 9217-8i and similar are all LSI model numbers, so they would be firmwares for LSI cards that have been bundled with Dell hardware. The actual Dell devices have names like H710P, H310, etc. and there is very little mention of what controller chips they're running. The H710P firmware I flashed to recover the card for example, had no mention of LSI, 2208, Thunderbolt, or any of the other details about the card's chipset.

    I can confirm that clearing the FW and BIOS does not work, because when booting Linux in this state the MegaRAID driver comes up saying the firmware is in fault and then gives up.

    The only chance of this working is finding out which of Dell's PERC cards (not the LSI ones) are using the 2308 chipset. I will keep looking, but the PERC chipsets don't seem to be that widely documented.
     
    #5
  6. Malvineous

    Malvineous New Member

    Joined:
    Sep 28, 2018
    Messages:
    4
    Likes Received:
    1
    Just had another go at this. Dell don't seem to have released any Dell-branded adapters based on the SAS2308 chipset so unfortunately there isn't anything there to flash. I tried flashing the H700 (SAS2108) firmware and the H730 (SAS3108) but even the recovery tool refused to flash them saying they were incompatible. I tried hex editing the H700 firmware file to make the chipset and PCI IDs match my H710P, but sadly that didn't make any difference.

    I tried booting with a cleared flash and as expected this just results in a bunch of kernel warnings about hung tasks, as the driver keeps "waiting for FW to initialize".

    Annoyingly, the H700 and H730 firmwares have references to the SAS2208 chipset, so it looks like they might actually be compatible, if anyone can figure out how to get a flash utility to accept the image.

    Weirdly after all this, Dell's perccli tool that originally flashed the bad firmware no longer recognises the card, saying there are no controllers installed on the system!

    EDIT: Also, in case anyone finds it useful, the Linux PCI ID database is handy to figure out what cards share the same chipsets. Here are links for the SAS2208 and SAS2308 device list.
     
    #6
Similar Threads: JBOD adventures
Forum Title Date
RAID Controllers and Host Bus Adapters LSI SAS9205-8e with Hitachi JBOD SAS Drive Spinup Question Tuesday at 3:14 PM
RAID Controllers and Host Bus Adapters Searching for an affordable RAID card that can also function in MIXED mode (RAID + JBOD) Sep 25, 2018
RAID Controllers and Host Bus Adapters NetApp DE6600 disk shelf as regular JBOD? Sep 18, 2018
RAID Controllers and Host Bus Adapters JBOD controller May 1, 2018
RAID Controllers and Host Bus Adapters Dell MD1200 JBOD (zfs) LSI SAS HBA support? Apr 20, 2018

Share This Page