HP MicroServer HBA controller recommendation

[sofianito]

Hi All,

This is my first post

I bought recently a HP MicroServer N40L and I am looking for a low profile eight internal 6 gb/s SATA ports controller to setup a SAN using Solaris 11, ZFS, iSCSI. The purpose of the SAN is to provide a shared storage for ESXi VMs and to backup personal data. The candidates I have in mind are actually:

1) IBM M1015
2) LSI 9211-8i

The M1015 seems cheaper (in ebay) than the 9211-8i. Is there a difference in terms of performance, though?

Will these HBAs fit without problems in the HP MicroServer? If so, in the case of the M1015, will I need a 90 degree minisas cable for the ports or a normal one would do the job?

Any other recommendations are welcome

Thanks.

 

[mobilenvidia]

There is only one choice really, the M1015

As you have already noticed the M1015 can be gotten cheaper
And better yet the M1015 can be turned into a LSI9211 where if run in IT mode ZFS will love it.

Have a look at the front page I've done a some benchies and a how to flash the M1015 into various modes

The M1015 is a Low profile card, if only LSI put the SAS connectors of LSI9240 (M1015) on the back of the card like the LSI9211 then it would have no issues with cabling.
In my Desktop case the cables have oodles of space to loop around, in a skinny server, ie a 1U you will need a 90deg cable, 2U you should get away with normal cables.

 

[Patrick]

I think you need a tape measure. The cards are half height, which should be fine, but the length is under 7" (IIRC around 6.6"). I don't have a HP MicroServer to measure, but it may be worth a shot since I know those are fairly tight areas where the expansion slots are in those. Also remember you need at least another 0.5" for the SFF-8087 breakout cables.

Any reason you need an 8-port internal card? What about looking at something with a SFF-8088 connector or two?

 

[sofianito]

Thanks mobilenvidia, Patrick for advices and help.

@mobilenvidia: I'll check your benchies, and will definitely follow your guide to turn it into LSI9211.

Regarding the minisas, the 90º cable is probably the way to go for the MicroServer. The M1015 will sit under the PSU separated by a panel. The distance between the MB and the panel is about 10cm (~4"). I need to check the distance between the minisas connector and the upper panel.


@Patrick: The max length I measured is about 17cm (~6.69"), so I believe the 6.6" will fit tightly . Simon Davies confirmed to me that the M1015 fits tightly.

Any reason you need an 8-port internal card? What about looking at something with a SFF-8088 connector or two?

Sorry, I am quite new in the HW domain. I googled and found out that SFF-8088 is apparently related to external storage, isn't it? Do you mean to use an HBA with external SFF-8088 connectors to add external storage? Actually the MicroServer has un eSata.

Here is my approach regarding the use of 8-port internal card. Please feel free to guide me or correct me if I am going in the wrong direction:

The MicroServer comes with 4x3.5" drives backplane connected to the 3gb/s onboard sata controller. I would connect the backplane to the minisas of the M1015 in order to use 3TB green disks, which allows me to have a max of 12TB of internal storage. To improve ZFS performance, I would use mirrored storage pool. In that case I would get a max of 6TB for data.

I also would like to use L2ARC + ZIL to improve further more the performance of ZFS. For that, I am planning to use the HP 5.25" ODD bay to add 2.5" SSDs. The options I am having in my mind are the following:

1) SATA QuickPort Intern 6-Bay: This allows me to have 6xSSDs with max height of 9.5mm. 2 mirrored MLC SSDs for OS connected to the 3gb/s onboard controller, 2 mirrored MLC SSDs for L2ARC and 2 mirrored SLC SSDs for ZIL connected to the M1015.

2) Icy Dock MB994SP-4S 4 in 1 SAS / SATA Hot Swap Backplane RAID cage: This allows me to have 4xSSDs with max height of 15mm. 1 MLC SSD fo OS, 1 MLC SSD for L2ARC, and 2 mirrored SLC SSDs for ZIL. All these 4xSSDs will be connected to the M1015 minisas.

Any thoughts are welcome

Best Regards.

 

[mobilenvidia]

You could connect the 4x 3TB drives to the M1015 with using one MiniSAS connector
Then go for option 2 and use the other miniSAS connector to hook up the 4x SSDs
Everything attached to the M1015 the way it was meant to be

You can boot from any drive attached to the M1015 and in LSI9211 mode also choose a secondary drive

Quite a cool little box the N40L

 

[Patrick]

Makes a lot of sense. I was thinking that you wanted to add eight more spindle disks which is why I was thinking external enclosure.

 

[sofianito]

I found a good price for a used one in ebay, but it is sold without bracket. I am wondering where can I buy a LP bracket. Any suggestion?

Thanks.

 

[mobilenvidia]

Ebay has them.

I had an old ServeRAID 6M, that bracket fitted perfectly.
If you are like me and horde stuff have a look around the pile of old cards.
I think it's a pretty universal fit the low profile backet.

 

[Patrick]

Another option is that if you haven't bought the one without the bracket, paying a few dollars more for one with a bracket may not be a bad idea. It will end up costing a few dollars to buy and ship anyway.

 

[sofianito]

@mobilenvidia: It is gonna be quite expensive... The cost including shipping to Spain will be around $150. If I sum duties, taxes and customs, it'll be around $180 (~142€) :/

@Patrick: A german vendor is selling brand new items for around 119€ (~$150) shipping not included. I have sent him an email to check if he ships to spain... Waiting eagerly for a reply... I hope to close a deal and get the card for next week...

I have already ordered the Icy Dock MB994SP-4S. Regarding SSDs, I am willing to get 2xCrucial M4 64Gb, and an Intel SSD 311 20Gb, but I need to get the M1015 before...

Cheers.

 

[mobilenvidia]

I see $32 postage for a little bracket is way too expensive

US$92 free shipping from Israel any good ? http://www.ebay.com/itm/IBM-ServeRA...tworking_SM&hash=item2c624b1725#ht_1668wt_952

The M4's are really good SSD's just got a FW update to see them past the 5184hour usage bug, today.

 

[sofianito]

Thanks a lot mobilenvidia!

I just ordered it for 74€ free shipping. I hope to get it in less than 25 business days , though.

 

[mobilenvidia]

You could walk to Israel and pick it up in less than 25 working days

 

[sofianito]

Great! I have ordered it on last Sunday and received it this morning!. Very fast shipping indeed! .

It came with a LP bracket installed and fits perfectly in the PCIE2 x16 slot of the MicroServer, but I had to fiddle while installing it because the minisas cable that comes with the microserver need to be redirected through another path in order to plug it in the card port. I'll post some pictures later on

Actually I have 1 boot drive (the 250gb that ships with the N40L) with Solaris 11 express installed on it, and 2 Western Digital WD20EARS (2TB Sata 3Gb/s) mirrored with ZFS. At boot time, the controller recognizes the 3 JBOD disks and I am able to get to the Solaris boot menu but unfortunately the OS doesn't load. A kinda of BSOD dump stack is displayed quickly on the screen and the system reboots. I tried booting in single user mode with no success. Could it be that this issue is related to the fact that I had installed the Solaris while connected to the onboard controller in AHCI mode?

The M1015's WebBIOS displays the following information regarding the firmware:
Firmware Version: 2.120.144-1325
Firmware Package Version: 20.10.1-0052
Firmware Time: Jul 25 2011;15:23:58
WebBIOS Version: 4.0-49-Rel

I am gonna flash it in IT-mode and see what happens...

Cheers.

Edit: As promised, here are some pictures of my new SFF SAN with the M1015 plugged in

Another picture of the HP MicroServer SAN and my new SFF ESXi5 mobo

 

[sofianito]

I have flashed firmware to IT mode. There are some typo errors in the instructions. The emptysbr.bin should be sbrempty.bin, and sas2flsh -o sasadd should be sas2flsh -o -sasadd:

Convert LSI9240(IBM M1015) to a LSI9211-IT mode:
Type in the following exactly:
megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0
<reboot, back to USB stick>
sas2flsh -o -f 2118it.bin -b mptsas2.rom
sas2flsh -o -sasadd 500605b0xxxxxxxx (x= numbers for SAS address)
<reboot>
Done!

After rebooting the second time I get the following messages displayed on the screen:
LSI Corporation MPT SAS2 BIOS
MPT2BIOS-7.23.01.00 (2011.11.17)
Copyright 2000-2011 LSI Corporation.

Press Ctrl-C to start LSI Corp Configuration Utility...


After hitting Ctrl-C, I see the following sequence of messages:


Please wait, invoking SAS Configuration Utility...

LSI Corp Configuration Utility will load following initialization!

LSI Corporation MPT2 boot ROM successfully installed!


The last message disappears quickly, but the SAS configuration utility is not loaded!. Disks are not detected, and then it tries to do a remote boot through PXE. Is this the expected behaviour ??

If I don't hit Ctrl-C, the disks are detected and grub is loaded.

Edit: I found a similar issue reported in post #55 of this thread. I disabled as much as I could in the BIOS with no success. Curiously, in LSI9240 mode I could hit Ctrl-H and load the WebBIOS...

Update Feb 08, 2012: Although I couldn't load the LSI configuration utility, the M1015 is working perfectly in my SAN HP MicroServer

 

[leagle]

I just picked up a N40L and I am about to drop in a spare M1015 that I have lying around, so this thread is very interesting. If things work out, this system may be able to replace my 2008 R2 server.

sofianito, my one concern is whether or not the stock 150w PSU can handle everything you are throwing at it. The SSDs won't draw much power, but I am worried about the cumulative effect of 4 spindles, 4 SSDs, and the M1015 running at full load. Have you taken any measurements or given any thought to that topic?

Thanks!

 

[sofianito]

I'm glad you bought it It is a very good choice for home NAS/SAN server and even for an ESXi lab. I am impressed how well it is made. It is quiet and is great to save on power bill

This PSU will handle with no problem 4 SSDs + 4 Spindles. If you google, you'll find some people have thrown 5 internal spindles at it with no problems.

In my config I have the following:
Oracle Solaris 11 express
x2 Crucial M4 64GB MLC SDD
x1 Intel 311 20GB SLC SDD
For SDDs cage, I got the Icy Dock I mentioned in the thread. I ended up disabling the cage fans because they were a little noisy...
x1 250GB 7200 rpm (the one that ships with N40L)
x2 WD20EARS 2TB 5400 rpm (I am waiting HDD prices to drop to get x4 3GB 7200 rpm)
IBM M1015 HBA Controller
HP NC360T dual nics ( I had to mod it to fit the x1 slot. Check my other thread)

The power meter shows the following values:
Idle: 58 watt
Active: I used Bonnie to do an hdd benchmark. The power reached 70 watt

Once I finish setting up my homelab, I'll do more tests.

Cheers!

 

[Nnyan]

Hello everone,

Just got my N40L, upgraded it to 8GB of ECC memory and I have my blu-ray player and SSD (for boot) on its way. I have six 3TB drives that I would like to use (I realize I won't be able to use them all) on this box and I am looking for a hardware raid controller that supports 3TB drives (data only is fine I'm not booting from the array). I've found some contradictory information online and wanted to see if the M1015 will support 3TB drives. Failing that suggestions for a relatively inexpensive card that will support 3TB drives in raid5.

Thank you in advance

 

[mobilenvidia]

2nd question on this in only a few hours

I have replied to a post here
Same applies for your setup.

Don't get an Advanced feature key thinking RAID5 will rule the day with the M1015
Write performance on the M1015 with RAID5 = slow
RAID 1 or 10 if you need redundancy and you'll be laughing

 

[Nnyan]

Awesome! Thank you for the answer! I have 6 3TB drives and I really want to maximize my disk space but with some redundancy (1 failed disk is good enough). I just found out that I can get the IBM M5015 for a very good price, has anyone used one of these on a Microserver?