MegaRAID cannot import foreign config (CacheCade too)

[TheBR]

Hey All, Have a disaster on my hands, got a LSI 9266-8i controller with cachecade (2 x SSD in RAID 1) and 4x WD RED in RAID10 running an ESXi 6 Homelab in a Lenovo TS140.

The system fell over this morning and upon reboot didn't recognize the config and is saying it cant import the foreign config. Looking in the MegaRAID BIOS i can see that one disk from EACH array is saying REBUILD, the partner disks are green.

Any attempt to import the foreign config from here is met with an error.

I booted up an Ubuntu Live USB and installed MegaCLI from which i have been able to do some diagnostics, but im NOT an expert, in fact its pretty much new to me (MegaCLi not Ubuntu)

I ran the following commands:-

./MegaCli64 -CfgForeign -Dsply -a0 | grep "Foreign configuration"

Foreign configuration 0 on controller 0:

Foreign configuration 1 on controller 0:

Foreign configuration 2 on controller 0:

Foreign configuration 3 on controller 0:

./MegaCli64 -CfgForeign -Dsply -a0 >> foreign.txt (to have a look at the config)

and finally tried to import the first config:-

./MegaCli64 -CfgForeign -Import 0 -a0

However i then get :-

root@ubuntu:/opt/MegaRAID/MegaCli# ./MegaCli64 -CfgForeign -Import 0 -a0

FW does not have Security key, create Security key first.
Failed to import foreign configuration 0 on controller 0.

FW error description:
The requested command has invalid arguments.

Now why is it asking for a security key? I took a look at the docs and it seems to indicate that it could be linked to SafeStore but i DONT have safestore enabled.

Looking through the foreign config text file i created above all drives show as
Secured: Unsecured
Locked: Unlocked

So im not sure why its asking to create a key?

So has anyone seen this before? Am i on the right track here? Any Ideas (Help!)?

Is there any way to even confirm Safestore is off - im not sure if this is just a red herring.

EDIT : Did a bit more digging - and had a look through the docs and the foreign config on the drives with MegaCli.

(In Safestore section)

Import a Foreign Configuration After you create a security key, you can run a scan for a foreign configuration and import a LOCKED configuration. (You can import unsecured or unlocked configurations when security is disabled.) A foreign configuration is a RAID configuration that already exists on a replacement set of drives that you install in a computer system. WebBIOS Configuration Utility and MSM allows you to import the existing configuration to the RAID controller or clear the configuration so you can create a new one.

To import a foreign configuration, you must first enable security to allow importation of LOCKED foreign drives. If the drives are locked and the controller security is disabled, you cannot import the foreign drives. Only unlocked drives can be imported when security is disabled.

Ok, so i went and took a look at the foreign config i have and each drive says:-

Secured: Unsecured
Locked: Unlocked

So why the hell is it asking me for a security key to be created?

 

[TheBR]

Hey All, Have a disaster on my hands, got a LSI 9266-8i controller with cachecade (2 x SSD in RAID 1) and 4x WD RED in RAID10 running an ESXi 6 Homelab in a Lenovo TS140.

The system fell over this morning and upon reboot didn't recognize the config and is saying it cant import the foreign config. Looking in the MegaRAID BIOS i can see that one disk from EACH array is saying REBUILD, the partner disks are green.

Any attempt to import the foreign config from here is met with an error.

I booted up an Ubuntu Live USB and installed MegaCLI from which i have been able to do some diagnostics, but im NOT an expert, in fact its pretty much new to me (MegaCLi not Ubuntu)

I ran the following commands:-

./MegaCli64 -CfgForeign -Dsply -a0 | grep "Foreign configuration"

Foreign configuration 0 on controller 0:

Foreign configuration 1 on controller 0:

Foreign configuration 2 on controller 0:

Foreign configuration 3 on controller 0:

./MegaCli64 -CfgForeign -Dsply -a0 >> foreign.txt (to have a look at the config)

and finally tried to import the first config:-

./MegaCli64 -CfgForeign -Import 0 -a0

However i then get :-

root@ubuntu:/opt/MegaRAID/MegaCli# ./MegaCli64 -CfgForeign -Import 0 -a0

FW does not have Security key, create Security key first.
Failed to import foreign configuration 0 on controller 0.

FW error description:
The requested command has invalid arguments.

Now why is it asking for a security key? I took a look at the docs and it seems to indicate that it could be linked to SafeStore but i DONT have safestore enabled.

Looking through the foreign config text file i created above all drives show as
Secured: Unsecured
Locked: Unlocked

So im not sure why its asking to create a key?

So has anyone seen this before? Am i on the right track here? Any Ideas (Help!)?

Is there any way to even confirm Safestore is off - im not sure if this is just a red herring.

EDIT : Did a bit more digging - and had a look through the docs and the foreign config on the drives with MegaCli.

(In Safestore section)

Import a Foreign Configuration After you create a security key, you can run a scan for a foreign configuration and import a LOCKED configuration. (You can import unsecured or unlocked configurations when security is disabled.) A foreign configuration is a RAID configuration that already exists on a replacement set of drives that you install in a computer system. WebBIOS Configuration Utility and MSM allows you to import the existing configuration to the RAID controller or clear the configuration so you can create a new one.

To import a foreign configuration, you must first enable security to allow importation of LOCKED foreign drives. If the drives are locked and the controller security is disabled, you cannot import the foreign drives. Only unlocked drives can be imported when security is disabled.

Ok, so i went and took a look at the foreign config i have and each drive says:-

Secured: Unsecured
Locked: Unlocked

So why the hell is it asking me for a security key to be created?

Creating a Key stops the message BUT still cant import.

Right - Went last ditch and WAS going to follow this article which can help rebuild the drives without wasting the data:-

http://mycusthelp.info/LSI/_cs/AnswerDetail.aspx?sSessionID=8612912037OZHNRNWPEJMMKCLUDHJUHSMCIKO[FL&inc=8186&caller=1&txtCriteria=Configuration%20Loss

In this, I followed the first stage

Select the ‘Configuration Wizard’ from the WebBIOS menu; a1)Select ‘Clear Configuration’; click Next;

Reboot

However, Upon reboot the sun was shining as when i went into the BIOS the drives were back, albeit with 1 offline drive for each of the RAID 10 volumes. WTF!!!!

I did not give a damn, booted it up, got everything i need and now i can start sifting through the logs trying to find out why this has happened.

Data saved though - PHEW!

No idea why, or even how, clearing the configuration A) Kept the right configuration OR any config for that matter and B) got rid of the wrong configuration.

 

[iq100]

Your experience is exactly why I prefer not to use LSI or any other 3rd party subsystems for RAID. Keep a daily, or more frequent, standard OS image, even bootable, and check out you can boot from it BEFORE you start each day. Then remove the bootable from your system to protect it. Unfortunate, but really TRUE. Until RAID is a standard OS based structure and NOT a manufacturer's proprietary design, unless you buy two of everything, you introduce new vulnerabilities.

 

[mrkrad]

I think the cachecade ssd setup has to be intact otherwise you can't rely on the underlying hard drives. Why I don't bother with cachecade!