IBM M5015 experiences

[mobilenvidia]

IBM M5015/LSI9261 experiences

Finally in from the US is my IBM M5015 with BBU unit and cables.
I'll do some playing around with it as I don't have the drives that are going to be used with it yet.

But I do have 2x SATA3 SSD's that will test it's RAID 0, will probably get a 3rd to test RAID 5, possibly a 4th to test RAID 6

Speaking of RAID6 I also have the M5000 Advanced Feature key.

Anything you guys would like to know or have me try let me know.

Now to plug it in and see if it works



Latest SAS2108 Firmware and Driver download

 

[mobilenvidia]

Very first impressions:

Takes ages to initialise every time you reboot, this belongs in a server on 24/7
Can't get CTRL+H to bring up BIOS, possibly no room for the BIOS, I'll disable Intel RAID as I no longer need it will try again later.
The heatsink runs hot hot hot, will need to find a fan to attach.

MSM works straight away.
Able to setup a RAID 0 Array in a jiffy.
With the M5015 all the options are available, Read ahead, Write through/back etc, not available on M1015 that I used

Quick Performance tests
2x OCZ Solid3 60GB SSD's in RAID0, with settings recommended for SSD's
ATTO result 840MB/s write, 1000MB/s read.
About double what each drive is supposed to do as per OCZ specs, makes me very happy
I need to do some .htaccess mods to my webserver and I'll get some piccies up.

RAID0 with read ahead on, write through, cached IO
ATTO 830MB/s write, 2,600MB/s read

RAID0 with read ahead on, write back, cached IO
ATTO 830MB/s write (lots faster in small size files then Write through) read speed is around 1600MB/s
Speeds are all over the place with write back on
I assume battery is still charging so when setting to Write Back with BBU it selects Write through.

Strange thing HDTach reports 250MB/s read, which is what I get with iCHR10 controller (SATA2)

Updated to latest IBM firmware.
SAS2flash doesn't see the card so can't backup the BIOS/FW this way.
Megarec sees the card but SBR and SPD backup not much use
MegaCLi not sure of the commands, may need to do some homework before I plug the M5000 key in.
Anyone with knowledge on a way to backup the BIOS/FW etc let me know.
Never mind, LSI has a page with all settings (GRrrr, doesn't detect card)

 

[mobilenvidia]

Any one looking at one of these keep in mind:

With BBU = keep on 24/7, battery looses about 30%/day when off
Takes a few hours to charge, and below a certain charge level will resort to Write through mode.

RAID 0, 1 and 10 or JBOD get a M1015 this does it soooo much better
The extra layer that the SAS2108 brings between HDD and OS slows real world performance

RAID 5 and beyond the M5015 will excel as the SAS2108 eats XOR for B'fast


I'll probably remove the battery till I get my HDD's for a RAID 5/6 box
I've got a M1015 still coming to compare some results with.

Now to get an ASUS WS mobo to get both SAS cards and GTX470 all running at once.
Ultimate setup M1015 for RAID0 SSD's, M5015 for RAID6 storage.
If only my money tree would fruit

 

[sotech]

Can you provide a part number for the BBU? Didn't know there was one, would very much like one for ours...

If you're looking at the P8B WS for a couple of these cards: we're running that board and 2x M1015 boards + 2 NIC cards and the option ROM runs out really quickly and we can't get into the card BIOS most of the time. It's a right PITA - I opened a case with Asus about it and they basically said tough bikkies. Maybe next time they'll build a board with more option ROM... I don't know whether the same is true for other Asus WS boards.

 

[mobilenvidia]

Battery 46M0917, about $50+ for a kit on Ebay
Remote mount 68Y7396 (I havn't got this)

The battery comes with the M5015 as std.

Disable the Mobo onboard HDD controller and you can get to the Option ROM more easily.

I was looking at the P55 WS Mobo as I have 16GB RAM and a XEON 3470 already

 

[sotech]

Thanks for the info there.

Hm. That's one thing I didn't try disabling - mainly because the hard drives are in use. I was really hoping for a motherboard that I wouldn't have to fiddle with settings on each time I wanted to reboot and change something in the raid setup... ah well.

Battery 46M0917, about $50+ for a kit on Ebay
Remote mount 68Y7396 (I havn't got this)

The battery comes with the M5015 as std.

Disable the Mobo onboard HDD controller and you can get to the Option ROM more easily.

I was looking at the P55 WS Mobo as I have 16GB RAM and a XEON 3470 already

 

[Falloutboy]

Just for reference.

My setup let me access the web bios through my uefi BIOS

I can access the Web Bios interface by going into the system bios and telling the system bios to run the addon bios ; however I can not do so from the press CTRL+H or whatever it is.

It comes up with something like Controller BIOS on 0D 00 E0 B6

Although I did not have to flash the bios on this, my current setup seems quite stable:

Xigmatek Elysium Case fitted with ASUS Maximus IV Extreme - Z
16 GIG PC 1600 DDR3
Intel i7 2600K (not overclocked yet)
2 x EVGA 3072MB GTX 580's, 1 in the first slot, 1 in 2nd to last slot. in SLI
Onboard 1 ADATA 128GB SSD, Cacheing 1 x 1TB WD Caviar Black Edition Sata 3
IBM M1015 running 4 x 2TB WD Caviar Black Edition Sata 3
Onboard ASUS DVD /RW
Silverstone Strider ST1500W Power Supply

I also hope to add a 3444E in my last remaining slot to run 4 x WD Caviar Black edition Sata 2 drives

I will run my remaining to Sata II drives as data backups until the flooding ends in bangcock and we start getting more drives in.

I copied the entire contents of the windows 7 home premium 64 bit to one 2TB drive and from there to the other drive I moved it, now I know it was cached by memory but DAMN! sas is quick.

Also I have yet to try all the Maximus's on board controllers, Marvell, Intel, JMicron.

Hope this helps.

 

[mobilenvidia]

UEFI BIOS's are better as there seems to be options to run 'other' OptionROMS from the BIOS.
Alas the LA1156 mobo's were last of the old non UEFI BIOS

I only found the disable Intel SATA with this card yesterday.
There is no need to enter the optionROM really as there is nothing in there that MSM can't do.
But cool to impress your friends.

There seems to be a lot of fiddling in the BIOS to get any RAID to work
To enable Intel RAID, I have to enable RAID mode, setup the Array then reboot back into the BIOS and enable AHCI mode.
This gets rid of the Intel RAID boot screen and doesn't allow changes but keeps the array.
But now no longer needed, as the HDD's will go onto the SAS controller

I need the Marvell conroller to run the PATA DVD burner, would have been nice to loose this piece of $%!#

 

[mobilenvidia]

Updated to latest LSI 9260-8i BIOS 12.12.0-0073

Not as straight forward as it should have been

MegaCLI and SAS2Flash don't recognise the IBM M5015
But the latest IBM FW 12.12.0-0056 comes with a IBM specific Firmware updater app.
I modified the batch file that calls the FW updater to upload the latest LSI FW.
Took ages to reboot, I thought the worst after a few minutes but the heart-beat LED started up and all was well (phew)

Now do I rebrand the M5015 to a LSI 9260-8i to get access to the MegaCLi/SAS2Flash etc
Or just leave it IBM.
Other issue will the M5000 key still work ?
I may just have to try it to see

 

[mobilenvidia]

Flashing SBR to LSI9260 worked, boot is so much faster.
Flashing SBR ro Dell H700 worked, like above boot is faster
BUT I can't SBR to anything else, no matter what I do, the H700 stays.

Boot after power on to USB then I get a write error 8192 when megarec sbr
Next try to write works, even says success but H700 is staying.
Will try some more stuff tomorrow.

BTW, M5000 key works in LSI9260 SBR and H700 SBR as well.

 

[Patrick]

Sweet find! I have a feeling LSI controllers are going to be super interesting when they qualify everything with SandForce drive firmware.

 

[mobilenvidia]

Yes I saw that last week, LSI and Sandforce this is going to get very good.

Bit concerned about my now H700, refusing to write the SBR.
Need to find PDF with the "factory reset' jumper position.

 

[mobilenvidia]

phew got the SBR issue sorted.

Looks like the latest FW (v0073) from LSI stops SBR being modified (probably write protects it)
I went back a few versions then was able to write the SBR again.
Keep this in mind for future SBR changes.

Now running M5015 with Dell v0004 FW and SBR set to LSI9260-8i

SAS2FLASH still can't see the adaptor, I assume the SUBVENDOR which is still IBM is the issue here.
I doubt LSI would allow this to be changed with FW hash checks to no doubt keep an eye on it.

 

[mobilenvidia]

SBR changing is affected in the actual SAS BIOS, not the firmware.
The very latest FW v12.12.0.0073 stops SBR changes this is BIOS v3.23 (Sept 2011)
You either run this BIOS then flash to an older one, change the SBR then flash to the new one to get round the lock.
Or as I'm doing currently:

Flash the CacheCade v2.0 Firmware/BIOS
Firmware is relatively new (Aug 2011) but has a higher FW version number v12.13.0.0104
BIOS is v3.20 (Nov 2010)
This allows SBR changes
I've got BIOS v3.21 and 3.22 as well, I'll test at some point to see where the lock begins.
Differerent FW's have different BIOS parts, ie the SUN BIOS has 'HIIM' what ever that is.

I've tried these BIOS's
SUN - other that installing more BIOS bits, same OptionROM as LSI
IBM - IBM SBR, LSI OptionROM
Dell - Dell SBR, has its own OptionROM, not had a very close look, must do so, also at some point had a CTRL+C at boot I never seen before and could not enter.
AOC - Not seen a BIOS specifically for this, it does have it's own SBR though, LSI OptionROM
LSI - LSI OptionROM obviously
LSI CC - LSI OptionROM, Cachecade v2.0, has CC bits, obviously I don't have a key so disabled

From trial and error, getting to the OptionROM in non UEFI BIOS:
In system BIOS make sure that in boot options that the RAID LSI is there (ie select it as a HDD)
Boot with CTRL+F11 at System BIOS splash screen (or what ever you need to choose the boot drive)
At LSI BIOS confirm, press CTRL+H
Boot option screen should now appear, choose the RAID LSI option to boot from
This should now enter the LSI Option ROM

And to confirm, avoid IBM SBR unless you run your machine 24/7
This seems to take ages at LSI BIOS initialisation, almost like a very first boot after flash but every time.
All other SBR's are much faster.

If any body has a SUN or an AOC SAS2108 SBR they could share, I'd love to try these especially the SUN one (no prejudice against AOC owners ofcourse )

 

[mobilenvidia]

BIOS 3.21+ no writeSBR changes allowed

Dell BIOS v12.10.0.004 has the option to press CTRL+R at boot to bring up it's own option ROM.
Anything changed in here makes the Array(s) incompatible to any other Firmware version !!

All Firmware except LSI 12.2.0.0025, Dell 12.10.0.004 and CacheCade v2.0 12.13.0104 install these FW parts: APP, BIOS, HIIM, PCLI, BCON, NVDT, BTBL and BOOT
LSI 12.2.0.0025 and CC 12.13.0.0104 has everything but HIIM
Dell 12.10.0.0004 doesn't have HIIM or BCON but has CTLR instead.

Thats enough for now, may actually setup a bootable partition to test something other than FW

 

[mobilenvidia]

Just noticed while installing IBM's new 12.12.0.0065 FW.

If you use Megarec to install the FW it will notice a mismatch of vendor on FW and Hardware.
ie I have the SBR set to LSI running LSI's CacheCade v2 FW.
When I installed 0065 from IBM it noticed the Vendor mismatch and changed it to IBM automatically
Only noticed as for some reason IBM SBR takes forever to boot.

 

[mobilenvidia]

SUN FW 12.12.0.0048 = all drives can power down and power settings options in BIOS

Configured
Unconfigured
Hot Spare

Configured/Online drive options:
Auto
Controller
Max
Max without cache

MSM also has the features able to toggle

Configured/Online drive options:
Auto
Max
Max without cache

Max = avoid spin up if cache will hold/has the data

Intel FW 12.12.0.0045 also has an advanced button to allow the drives to be not spundown at certain times
Can't remember if the SUN version has this

 

[mobilenvidia]

New CacheCade v2.0 Firmware just out v12.12.0.0077

BIOS v3.22

Can't set 'Configured drive' power properties like in 0048

New versions also of MegaCLi available.

 

[mobilenvidia]

Powersave feature works well, running SUN FW 12.12.0.0048 with all drives not active for 1/2hr going to powersave mode (does not work on SSD's)

Now in MSM under Physcial and Logical drives the HDD with it enabled has a special icon and show as being in powersave mode.

Now why can't LSI have this feature in all their FW releases ?

 

[mobilenvidia]

Hmmm, looks like new FW is getting smarter in what SBR is loaded.

I now get error beeps with latest Cachecade FW if the SBR is anything but IBM
Beeps right after the drives are initialised.