HP H240 (SAS3 HBA) 30 GBP from EU seller / 34 USD from US seller

[nle]

How is this card for use with ZFS? Did I understand correctly that it will not show the smart info?

 

[RTM]

How is this card for use with ZFS? Did I understand correctly that it will not show the smart info?

I have been looking into it, it is my understanding that it may not be that useful on FreeNAS because FreeBSD does not have the new(ish) "hpsa" driver but relies on the older "cciss" driver which has been deprecated in favor of the "hpsa" driver. I think SMART data is accessible, at least it looks like they got it working in page 2 of this thread, but there may be other consequences:

HP H240 HBA Controller

That is why I said this is not optimal. You really need to look through the hardware guide: https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/ This hardware you are using is going to cause you problems down the line and if there is any way that you can get...

www.ixsystems.com

The hpsa driver should be present in newer linux distributions, so I imagine that it will work great on that.

In any case that is just research, some of it is fairly old (that FreeNas thread is from 2017) a lot of stuff could have changed since.

I intend to do some testing when the ones I ordered arrive.

 

[Markess]

HP redesigns their server chassis only every 2-3 generations. They changed the design when going from G7 to G8 and they were not going to change it again just for SAS3. So for G9 they decided to keep G8 drive cages and passive backplanes (with SFF-8087 connectors) and use custom designed SAS3 controllers with SFF-8087 connectors instead.

There are no performance or compatibility issues with the cards. Just make sure to use quality cables.

I don't need another HBA, but this is a really interesting thread.

 

[BeTeP]

I do not like Dell and I did not like VMWare even before it was acquired - so personally I do not care that much about ESXi7 dropping support for a lot of older but still popular hardware. But some people here on the forum seem to care a lot.

Nobody brought it up yet but this H240 controller is still supported by ESXi7.0 out of the box (requires firmware update to 7.00).

VMware Compatibility Guide - I/O Device Search

www.vmware.com

 

[RTM]

So I just got the H240's I ordered, I was thinking about installing one (or two) in a machine that already has H220's installed to do a performance and feature comparison.

Any ideas to a good suite of tests?
I was thinking testing various OS (Linux, ESXi and FreeNAS), availability of SMART information and some variation of a fio performance test.

 

[Patriot]

iometer and fio are the defaults, if you let me know what cpus and disks you are using I can help with a block/q depth sweep

 

[RTM]

iometer and fio are the defaults, if you let me know what cpus and disks you are using I can help with a block/q depth sweep

That would be great, in terms of hardware, if I remember correctly it is a dual E5-2670v2 as well as 8x Samsung SM1625's 400g (not SAS3, but I assume good enough to do a test).

btw. I need to free the hardware first, so I expect it will be in the weekend.

 

[nickf1227]

I have been looking into it, it is my understanding that it may not be that useful on FreeNAS because FreeBSD does not have the new(ish) "hpsa" driver but relies on the older "cciss" driver which has been deprecated in favor of the "hpsa" driver. I think SMART data is accessible, at least it looks like they got it working in page 2 of this thread, but there may be other consequences:

HP H240 HBA Controller

That is why I said this is not optimal. You really need to look through the hardware guide: https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/ This hardware you are using is going to cause you problems down the line and if there is any way that you can get...

www.ixsystems.com

The hpsa driver should be present in newer linux distributions, so I imagine that it will work great on that.

In any case that is just research, some of it is fairly old (that FreeNas thread is from 2017) a lot of stuff could have changed since.

I intend to do some testing when the ones I ordered arrive.

You appear to be correct, or I would be buying one right now. There is no mention of the hpsa driver in 11.3, 12 or 12.1.

I can say for sure, though, that the HP H220 works great in FreeBSD/FreeNAS and is selling for around $33 right now. It is LSI 9207 based and so it is also PCIE 3.0, but is SAS 6.

NEW HP H220 9205-8I IT PCI-e 3.0 x8 Host Bus Adapter 660088-001 638834-001 | eBay

ETA 3 to 6 Weeks. We will do our best and help you provide the solution. ※ All faulty due to improper installation or improper use will NOT be covered by the warranty.

www.ebay.com

 

[rcatank]

I just cant get over these cards. Its very strange. These cards are just sooo curious in their choice to use SFF 8087 instead of SFF 8643. I wonder if this was a decision to cut costs for the card? This also tells me that they are confident that SFF 8087 will transfer at 12Gbps? Very interesting.

If I am not mistaken, isn't SFF 8643 just a split SFF 8087 electrically on the pcb. Given you get passive cables that just convert one to the other, I don't see why it would have any performance issues.

I feel its better its its a SFF 8087 instead because at least its got a metal latch both on cable and card, and thus more secure than the plastic of the SFF 8643 male and female ends.

 

[RTM]

So here are the initial results of my tests (which are limited because I spent WAY too much time getting ESXi to boot on my old Asus machine):

• Cards came with firmware version 6.88
• Cards were in RAID mode
• Controllers and disks were detected in ESXi 7.0, Debian Linux 10.3 and FreeNAS 11.3-U3.1
• Only tested in ESXi: Disks were not detected when controller was in raidmode (I assume it was because a RAID array was not configured)
• FreeNAS specific: As assumed controllers were claimed by CISS driver and not a hpsa driver
• FreeNAS specific: (probably because of CISS driver) disks connected with a link rate of less than 150 MB/s (I guess this makes the H240 undesirable for FreeNAS)

To change the controller to HBA mode, I used the commands from this post:

https://forums.servethehome.com/index.php?threads/hp-pmc-sa-in-whitebox.4732/page-2#post-227583

For reference essentially this:

1. Download the offline SSA iso
https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_321a88714bff4279abf175b9bd

2. Boot from the ISO in SSA CLI mode

3. Get the serial numbers and copy the serial numbers
ctrl all show

4. Set to HBA mode
ctrl serialnumber=SERIAL modify clearconfigdata
ctrl serialnumber=SERIAL modify clearnvram
ctrl serialnumber=SERIAL modify bootvolume=clearprimary
ctrl serialnumber=SERIAL modify bootvolume=clearsecondary
ctrl serialnumber=SERIAL modify raidmode=off

 

[kroem]

Is there a 16 port version of this card?

 

[Tiberizzle]

PMC has always made much better RAID controllers than LSI. But LSI's low end models were less expensive than PMC's. Which made them more popular and better supported many years down the road.

There are no performance or compatibility issues with the cards. Just make sure to use quality cables.

I'm going to have to politely beg to differ on these points.

I have observed the H240 to have subtle but fatal flaws which render it incompatible with the vast majority of SATA SSDs I have tested when behind the HP SAS expander part.

The issue presents only under heavy and constant I/O pressure (deeply queued random I/O bound tasks for 24+ hours).

The controller locks up probabilistically during error recovery from certain operations in that state and does not recover from the lock up without warm reset.

This occurs whether or not TRIM/DISCARD operations are issued.

This occurs with every SSD, expander and controller firmware I have tested including the most recent releases.

So far I have only seen one SATA 3 SSD part the controller does not eventually lock up with.

In every case simply moving the affected disk to another node with an LSI controller resolved the issue for that disk.

I would recommend AVOIDING this controller at any costs if your intended application involves SATA 2 and older SSDs behind a SAS expander and regarding the controller with suspicion until confirmed to work in other applications.

Moreover, I suspect this is the reason people are dumping them at "please dispose of my e-waste" prices.

 

[abqcheeks]

I bought a couple of these. They came with firmware version 7.0 in HBA mode. I torture tested them a bit with MySQL / ZFS using proxmox on Debian 10, with the hpsa driver. They work fine, but have a fatal flaw (for me anyway): no hot-swapping.

If I pull a drive and replace it with a blank spare, the kernel reports seeing the insertion, and ssacli reports it is inserted, but the OS is not notified and no /dev device files are created. If I re-insert a drive that it knew about at boot time (in the original slot or a different slot) it is recognized. I just can't get it to see a new drive without rebooting.

In the system log, the difference between a drive that it exports to the OS and one it doesn't is whether it says "Exp=0" or "Exp=1" at the end of this line:

[10439.304095] hpsa 0000:04:00.0: scsi 0:0:6:0: removed Direct-Access     ATA      Samsung SSD 860  PHYS DRV SSDSmartPathCap- En- Exp=0

I haven't tried this on FreeBSD (or FreeNAS) with the ciss driver yet ... I suppose I will just to satisfy my curiosity, but I'm also ordering some LSI cards to get this project finished.

 

[Woogz]

I picked up one of these cards planning to use it with a SAS SSD but am finding it impossible to find a cable that supports 12Gbps. I'm beginning to think that one doesn't exist.

I found
Internal 68 pin Mini SAS (SFF-8087) - 12 GB SAS Drive (SFF-8680) x 4

Which doesn't make sense to me since I don't get how it can be 68 pin and still be 8087.

I've also found
SAS 36 Pin (SFF-8087) connector to 4 x Low Profile SAS 29 pin (SFF-8482)

But SFF-8482 doesn't support 12Gbps.

So this card says it can do 12Gbps but I guess only if you have a backplane?

 

[Woogz]

Hmm. That's interesting. I truly thought all 8087/8482 and the newer 8643 cables had the capability to do 12Gbps, but were limited by the controller.

I believe you though. I started taking a look at these cheap controllers myself, and I ordered one. I think these cards may become more popular soon.

So I made a similar post on r/Datahoarder and someone found this post
https://www.reddit.com/r/DataHoarder/comments/76oaga
So I went ahead an ordered a normal SFF-8087 to SFF-8482 and will give it a try. I made sure to get one smaller than 3 feet though in case that does make a difference.

 

[Woogz]

Yeah im leaning towards most 8087 cables working, but let us know what you find. Remember, unless you have 12Gbps drives, you wont see it reported. The cable will connect at whatever the speed of the drive is. In your case, its a 12Gbps SAS SSD, so it should show in the card's bios or reporting utility.

Sorry I'm a bit new to all this. My plan was to test it with CrystalDiskMark but you're saying the card should tell me if it's seeing 12Gbps. Do I get to the card's bios the same way as I do my mobo? Is the reporting utility something I need to download?

And yeah if not for that one Reddit post I never would have thought there was a chance it supports 12Gbps because literally everything I can find says that 8482 only supports 6Gbps

SFF-8482 - CS Electronics

sff-8482

cs-electronics.com

Although I suppose a lot of resources just don't specify a speed so perhaps it's because the connector isn't the limiting factor.

 

[Woogz]

So I finally got my card and my SFF-8087 to SFF-8482 cable.

Using Crystal Disk Mark running at 32GB sample size I'm getting roughly 1000 MB/s read and 950 MB/s write which translates to 8 and 7.6Gbps respectively. So I'm definitely getting over 6 but not quite to 12. I'm told there's some encoding overhead so it's looking like I may actually be limited to 10Gbps like is called out in this post:

HP 240 12Gbps SAS3 internal HBA - $34.00 OBO free shipping

HP 240 12Gbps SAS3 internal HBA - $34.00 OBO free shipping Uses SFF-8087 cables instead of more expensive SFF-8634 cables This will limit you to about 10Gbps per channel (still quite a bit faster than 6Gbps) For SATA drives only, use these cables: https://****/2XF1wvq For SAS/SATA...

forums.serverbuilds.net

My question is why? The card's bios utility reports its speed at 12Gbps. I know the drives can do higher.

These HP cards are just so bizarre. Any one have any thoughts?

I will add that I ordered a 20" cable. The shortest one I could find due to this reddit comment:
https://www.reddit.com/r/DataHoarder/comments/76oaga
However they actually sent me a 24" cable. But he said with 19" cable he was able to get 12Gbps.

 

[TheServerGuy]

See here. The 8087 cable is limited to 10Gbps. You will see the negotiated connection rated at 12Gbps but the top rated speed is 10Gbps.

 

[RTM]

A while back I did some preliminary comparison of this card and a HP H220 (LSI SAS2308) using an array of Samsung SM1625's (notably these are SAS2 drives), and this controller was a bit slower (FWIW I rememeber it as around 100MB/s slower for the array).

I have forgotten the exact configuration I used, but I remember the result being pretty consistent, so perhaps this card is just a little slower?

See here. The 8087 cable is limited to 10Gbps. You will see the negotiated connection rated at 12Gbps but the top rated speed is 10Gbps.

I am not reading the post, as it is limited to 10G, rather that it is what the standard specifies.
If the reality for SAS is anything like network cables, where you can do sometimes do 10G on Cat 5 cables, then you may be able to do 12G on this too (though you may want to ensure you have good/short cables).

 

[EasyRhino]

I got one of these, it came in HBA mode.

You don't need an HP server to configure this HBA. You can use Smart Storage Administrator CLI (download, manual) to manage RAID/HBA functionality from the OS. Alternatively, you can create a USB key to boot off of and setup your preferred configuration there.

I was able to use the command line tools to update the bios in linux.

However, I tried making the USB key to boot and configure, and it would crash during the modified linux boot process.

I'm glad I didn't actually have any configuration to do, just wanted to check it out.