LSI 9270-8i IT mode

[Toasty]

Sorry if this has already been answered elsewhere. I did a forum search but couldn't find exactly what I was looking for.

I have a client with an LSI 9270-8i, with 8 drives currently connected. No RAID, just pass-thru to the OS (currently Server 2012 R2). This works fine for their current setup, but I am planning on installing CentOS and configuring the drives for a ZFS mirrored array.

I have done this many times before with different controllers, but they were all capable of being put in IT mode, where I could actually see the physical drives and get SMART data and whatnot. I don't have any prior experience working with the 9270-8i in particular, so I'm unaware if there is such a mode available. I can't seem to find anything about flashing the firmware for IT mode. I would love to poke around the setup interface available on boot, but the server is recording surveillance footage and state law requires no downtime without a prior approval (which is a bit of a hassle if I'm just taking it offline for a few minutes). I don't see anything in the MegaRAID Storage Manager interface that tells me what I need to know either.

Does anyone know if it's possible to configure the 9270-8i for IT mode or as a "dumb" HBA? I'd prefer to use my client's existing hardware, but if this isn't possible, I'll also take recommendations for replacements (I have one in mind, but the more opinions the merrier).

 

[nthu9280]

Refer to the LSI sticky thread.

It's based on 2208 chip. It can't be flashed to IT mode. You need to get either 2008 or 2308 based cards.

Computer Parts, Laptops, Electronics, and More - Newegg.com


Sent from my Nexus 6 using Tapatalk

 

[w0mbl3]

Refer to the LSI sticky thread.

It's based on 2208 chip. It can't be flashed to IT mode. You need to get either 2008 or 2308 based cards.

I didn't know that when I started, so I did it anyway

I just flashed my 2208 (Supermicro X9DRH-7F motherboard onboard) to an IT firmware from a 2308 as an experiment. (I've flashed various LSI devices from Dell/HP with firmwares so aware the some of the differences of versions of sas2flash/megarec over time).

Seems to work so far - have booted Fedora off an Intel SSD.

Obviously not a supported solution given I've crossed hardware boundaries with the ROM's and firmwares (controller now reports as a 2308 in sas2flash).


Here's the info from Sas2flash after the flashing, and Megarec before I started (can check device ID and Vendor ID from the megarec output to confirm its a 2208 - reference the ID's here http://pciids.sourceforge.net/v2.2/pci.ids )

After:


Adapter Selected is a LSI SAS: SAS2308_2(D1)

Controller Number : 0
Controller : SAS2308_2(D1)
PCI Address : 00:01:00:00
SAS Address : 5003048-0-195E-4500
NVDATA Version (Default) : 14.01.00.06
NVDATA Version (Persistent) : 14.01.00.06
Firmware Product ID : 0x2214 (IT)
Firmware Version : 20.00.07.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9207-8i
BIOS Version : 07.39.02.00
UEFI BSD Version : 07.27.01.01
FCODE Version : N/A
Board Name : SAS9207-8i
Board Assembly : N/A
Board Tracer Number : N/A

Finished Processing Commands Successfully.
Exiting SAS2Flash.

Before:


Adapter #0

==============================================================================
Versions
================
Product Name : LSI MegaRAID ROMB
Serial No :
FW Package Build: 23.28.0-0010

Mfg. Data
================
Mfg. Date : 00/00/00
Rework Date : 00/00/00
Revision No :
Battery FRU : N/A

Image Versions in Flash:
================
BIOS Version : 5.46.02.0_4.16.08.00_0x06060900
WebBIOS Version : 6.1-71-e_71-Rel
Preboot CLI Version: 05.07-00:#%00011
FW Version : 3.400.05-3175
NVDATA Version : 2.1403.03-0128
Boot Block Version : 2.05.00.00-0010
BOOT Version : 07.26.26.219

Pending Images in Flash
================
None

PCI Info
================
Controller Id : 0000
Vendor Id : 1000
Device Id : 005b
SubVendorId : 15d9
SubDeviceId : 0690

Host Interface : PCIE

ChipRevision : D1

Link Speed : 0
Number of Frontend Port: 0
Device Interface : PCIE

Number of Backend Port: 8
Port : Address
0 0000000000000000
1 0000000000000000
2 0000000000000000
3 0000000000000000
4 0000000000000000
5 0000000000000000
6 0000000000000000
7 0000000000000000

HW Configuration
================
SAS Address : 50030480195e4500
BBU : Absent
Alarm : Present
NVRAM : Present
Serial Debugger : Present
Memory : Present
Flash : Present
Memory Size : 1024MB
TPM : Absent
On board Expander: Absent
Upgrade Key : Absent
Temperature sensor for ROC : Present
Temperature sensor for controller : Absent

Here's Smartctl info from a drive after flashing the 2208 with IT firmware, to show its working:

smartctl 6.5 2016-05-07 r4318 [x86_64-linux-4.11.9-300.fc26.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model: INTEL SSDSC1NB240G4R
Serial Number: BTWL4466037D240HGN
LU WWN Device Id: 5 5cd2e4 04b71e409
Add. Product Id: DELL(tm)
Firmware Version: D201DL13
User Capacity: 240,057,409,536 bytes [240 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 1.8 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-2 T13/2015-D revision 3
SATA Version is: SATA 2.6, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Fri Aug 25 10:03:09 2017 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

 

[w0mbl3]

I'm now attempting to reverse the IT firmware flash on my 2208 - putting it back to stock (latest from Supermicro's ftp server) Megaraid ROM version.

If that all works, and my crazy cross-hardware flash reverses successfully, I'll write up how I did it along with which versions of the tools I used

 

[nthu9280]

Wow. I've never tried cross flashing the FW of a different chip architecture (2208 vs 2308). While it may be technically possible since flash chip is a separate from 2x08 ROC, don't know how it affects the data that is passed through the card.


Sent from my Nexus 6 using Tapatalk

 

[w0mbl3]

I was surprised it works - seems to have no issues, but I'll be doing some extensive soak tests to verify, as well as using it for VM OS drives rather than data I want to keep I do note that the 2208 and 2308 share the powerpc architecture (altho' 2208 has much more processor grunt), but also realise this will be way off the scale of risk for many.

Plan is to use it to run a RaidZ across 5x 512GB Samsun 840 Pro SSD's, using PCI passthrough to a Freenas (or maybe Nas4free) VM, which will serve up iSCSI volumes to other VM's. I since managed to flash it back to stock, and then reflash to the 2308 firmware.

I have screenshots via BMC of each step of the flash (and back-out), and will be writing it up when I get some spare time, but obviously its a highly not supported configuration and only insane idiots would be willing to actually use this in any build. Having proven I can reflash it back to stock, IMO all I'm risking is non-critical data.

If it shows up any data errors during soak test, I'll just disable it and swap in one of my spare HP H220 (2308-based) PCIe-3.0 cards with the same IT firmware on them, but at this point its just a bit of fun

 

[vanfawx]

Wow, that's some pretty impressive black magic there. Is this only doable between an 2208 and a 2308? Or any chance of this working with 2108 based MegaRaid cards (like the 9260-8i)?

 

[w0mbl3]

2108 looks like it may work too - same Powerpc 440 chip @ 800MHz as the 2308 (2208 is a dual-core version of same). However - there is a risk that it'll brick the card.

Note that I've not tested this extensively - only a sample of a single 2208 onboard LSI on my supermicro motherboard

There was a while before I managed to get the LSI recovery firmware to work, where I thought I may have bricked the controller when trying to revert to being a 2208 again.

It was a risk I was willing to take as RAID is useless to me (I'm not a fan of hardware RAID). I need IT firmware for ZFS appliance - and I have a couple of spare 2308 cards sitting on the shelf, so if I broke the 2208, I'd just write it to experience and use a 2308.

I've got screenshots of each step and command, including how to back up the SAS address (as I wiped it in the process - deliberately, but then restored the same address later), and will post them up somewhere on the weekend, with a lot of disclaimers

 

[vanfawx]

I look forward to giving your instructions a shot once you have them pulled together online! I have a bunch of 9260's and a 9270 I'd love to try this out on to create some HBA's.

Thanks for sharing what you've learned so far!

 

[w0mbl3]

Here's a first pass at documenting what I did..

Flashing LSI 2208 with IT firmware to use as an HBA - My Wired House

 

[vanfawx]

Great writeup, very clear. Are the zip bundles you reference on your site available via google? If not, would you be willing to package them up?

I look forward to trying this. I have a 9270-8i that I'm looking forward to converting to IT mode.

Thanks again and I'll update this with my experience going through the process!

 

[vanfawx]

Actually google came through as always and I've acquired the files. I look forward to trying this out!

 

[w0mbl3]

Actually google came through as always and I've acquired the files. I look forward to trying this out!

Cool, I figured the filenames would be enough to track them down.

There are multiple versions of the mega* tools - I had at least 3 versions of megarec.exe sitting around on my file server where I keep old stuff, and 2 of them didn't recognise the card - but thats par for flashing LSI devices. e.g. when cross-flashing the H220 HP cards to LSI firmware, the v20 sas2flsh.exe won't do it, but the one from the v14 driver pack would using the v20 FW.

I'm pretty sure the one in lsi_dos.zip was the one which worked, but if not, the sas2008.zip file has a different version in it. Same applies for megacli.exe, and sas2flash. (note that megaoem.exe also supports some of the same commands as megacli).

Trying to work out where I got the sas2flash.efi I used - its 387 bytes and dated 2016.

edit: Installer_P20_for_UEFI.zip has the sas2flash.efi I used

If you get stumped on tools, I can work on pulling together a zip file of the ones which worked for me, but that machine is in pieces currently (changing PSU and installing a 6x2.5" drive bay adapter), so it'll be a week or so as this weekend is busy.

 

[w0mbl3]

I did just notice the 2108 has a PCIe 2.0 interface, and not PCIe 3.0 like the 2208/2308 - which may prove a challenge, so please don't give this a shot unless you're 100% prepared to risk a bricked card.

 

[vanfawx]

I'm hoping it won't make a difference, but I do have a stack of 9260's so I can sacrifice one of them. Considering this was always "impossible", I'm willing to risk a little to make a pile of fairly useless raid cards (to me) into useful HBA's.

I went through your instructions and found them easy to understand and link to the files and versions required. I'll be able to do some testing this week once I free up a machine to do testing on.

 

[vanfawx]

@w0mbl3 So I went through your steps today, and they work exactly as you documented. I ran into one thing that I'm guessing you didn't. The retail 9270-8i card I have has a speaker on it. When I converted it to a 9207-8i, the speaker starting screaming. Since I'm not planning on going back to the megaraid firmware, I removed the speaker.

I have screen shots of each step, but the steps worked as you laid them out. The only thing you didn't have in your steps, is you need to reboot after you wipe the flash via megarec otherwise sas2flash won't pick up the card.

Sorry for the delay on this, things have been crazy lately

 

[vanfawx]

I did just notice the 2108 has a PCIe 2.0 interface, and not PCIe 3.0 like the 2208/2308 - which may prove a challenge, so please don't give this a shot unless you're 100% prepared to risk a bricked card.

So funny story...the 2308 RoC actually supports both PCIe 2.0 and 3.0. The 9205-8i is the PCIe 2.0 version of the 9207-8i. Both use the same 2308 chip, ones just an earlier rev than the other. So I wonder if I can flash the 9205-8i firmware to a 9260-8i.

I'll give it a shot and let you know ;-)

 

[nthu9280]

So funny story...the 2308 RoC actually supports both PCIe 2.0 and 3.0. The 9205-8i is the PCIe 2.0 version of the 9207-8i. Both use the same 2308 chip, ones just an earlier rev than the other. So I wonder if I can flash the 9205-8i firmware to a 9260-8i.

I'll give it a shot and let you know ;-)

2308 was made in two different steppings - C0 & D1, IIRC.
The cards with older stepping are only PCIe v2 and can only be flashed with 9205-8i(e). This was documented in Xeon e5-2670 thread. OEM XS had mixed stock of these Intel branded cards.
Newer stepping can do PCIe v3 and can be flashed as 9207-8i

Also, SBR files may be required for certain OEM cards eg., Fujitsu 2607-A21(/A11) cards. They are SAS2008 based cards like H310, H200 etc. Without correct SBR file the card can only see 4 drives.

I have a Dell H700 (SAS2108) should be equivalent to 9260-8i. This thread almost makes me want to tinker with it.

Another great LSI tool if you guys want to get into the weeds is LSIUTIL. Not most user friendly with 90 main menu options and then nested submenus. I think there is GitHub version too. Broadcom / LSI site has an older version. If someone has a Oracle /Sun support there may be there is a current version. There is more recent version on the internet but at this day in age, I'm kind of leery of getting files from unknown sites





Sent from my Nexus 6 using Tapatalk

 

[w0mbl3]

So I went through your steps today, and they work exactly as you documented. I ran into one thing that I'm guessing you didn't. The retail 9270-8i card I have has a speaker on it. When I converted it to a 9207-8i, the speaker starting screaming. Since I'm not planning on going back to the megaraid firmware, I removed the speaker.

I have screen shots of each step, but the steps worked as you laid them out. The only thing you didn't have in your steps, is you need to reboot after you wipe the flash via megarec otherwise sas2flash won't pick up the card.

Sorry for the delay on this, things have been crazy lately

Awesome - I didn't have a speaker as mine is onboard on my supermicro motherboard so no screaming on my end.

Really interesting that it works on your card tho' - its a retail LSI card rather than a Dell/HP/licensee version? Did your 9270 card end up with an identity crisis and show as a 2308 after flashing instead of 2208? Can it see all 8 channels?

Noted on the reboot - I reconstructed my steps from screenshots, so may well have rebooted between flashes thanks, I'll add a note to my steps.

2308 was made in two different steppings - C0 & D1, IIRC.
The cards with older stepping are only PCIe v2 and can only be flashed with 9205-8i(e). This was documented in Xeon e5-2670 thread. OEM XS had mixed stock of these Intel branded cards.
Newer stepping can do PCIe v3 and can be flashed as 9207-8i

Thanks nthu9280 forthe info - my HP H220 cards must be the newer stepping, as I've flashed them with 9207-8i (same firmware file I used for the 2208 above). Somewhere I've got a photo of the chip from when I took the heatsink off to verify it was a 2308 on the card.. will have a look and see if I can find it and whether it shows stepping.

Note that Vanfawx didn't flash his 2108 card (LSI 9260) - so only 2208->2308 flash has been done in this thread, speaker issue notwithstanding.
From what you're saying on PCIe 2.0 vs 3.0 - sounds like if the 2108 card is a PCIe 2.0 variant, then the 9205-8i firmware has a better shot of working, although I suspect flashing a 2.0 card with PCIe 3.0 microcode might be enough to brick it.

 

[nthu9280]

Typing from my memory and it's 2:30am us central

Sas2flash -list ( or -listall ) should get you the chip info


Sent from my Nexus 6 using Tapatalk