Right OK thanks, sorry I misread. Now I'm confused about which combination I tested re using more than 4 ports - I'll have to re-try.
Now I'll be interested to know if you get the same SATA drive detection issues that I found in the 510A FW. You've only connected 1 drive so far, so you definitely wouldn't have noticed it yet.
Specifically, when I had 15 drives connected to the expander using 510A FW, I would find that only two or three drives showed up in my LSI BIOS and in the OS, unless the expander was powered up a few seconds before the drives/backplanes connected to it.
This 2-3 total drive number made no sense, as I had 4 x 8087 cables, each with 3 or 4 drives on them. I seem to remember that I worked out that two or three drives of one port were detected, then no drives on any other port. But swapping cables around didn't necessarily change which drives did detect, and it never detected all drives on a single port. So it seemed like the expander was simply quitting in the middle of drive enumeration, except when the expander had time to fully power on/initialise before it saw any connected disks.
You're already powering your expander from a server PCI-E slot. I don't know if your drives are currently powered from the server, or an external JBOD? Either way, I would expect that when you try connect many more drives - theoretically as few as four, but definitely if you use 12+ - you will find that unless the expander is turned on a few seconds before the drives, your HBA BIOS and OS will only detect two or three of them.
(Unless we find that it is something specific to my disks/backplanes - another thing I couldn't easily test. It could be that my disks spin up a little slower than others might, which creates the problem/makes it more likely.)
If you don't have this issue, then you might not need to use any FW later than 510A anyway - unless you do want the (theoretical) option of >16 drives per expander.
To re-iterate, I found myself stuck between two bugs:
- On 510A, I got full 2 x 6G bandwidth to the HBA.. but I had this power-on timing SATA detection issue, requiring the expander is always powered before connected drives.
- On later FW, the power-on timing SATA detection issue was gone.. but I could never get full 2 x 6G bandwidth to HBA.
I tried literally every FW and could not find any that solved both problems, so I ended up working around the SATA detection issue, powering the expander from my motherboard PSU instead of my JBOD PSU, and requiring that I always turned on the server before the JBOD, then waited a few seconds before turning on JBOD. Such that the expander would always be initialised before any disks/backplanes spun up.
It was much more important to me to get full BW, and although my workaround was a bit annoying, it was usable and wouldn't have presented much long-term pain, given that 99% of the time the server and JBOD would be running 24/7.
I would only experience the annoyance on the rare occasions I needed to power down the server, as I would then need to remember to also power down the JBOD, and then wait for 15-20 seconds before also turning the JBOD back on again.
It also made cabling a bit more awkward, given my expander is physically located inside my JBOD (on an external fake PCI-E slot for power), but must be powered from the motherboard, necessitating an extended molex cable running between the two. That wouldn't be an issue for you, as you're using a real PCI-E slot for power.
All this also means that if one did not have a JBOD - ie all drives were installed and powered in the server itself - things would be much worse, because there'd be no easy way to turn the drives on later than the expander. But I think you mentioned having a monster 84 drive array, so I assume you do have some kind of external chassis?
But it's promising that the top port works with 510A. Will try and update results here.
