Hmmmm, seems odd that they'd want you to be limited to the bandwidth of a single SAS3 connection when the box can hold 32 SAS3 devices and also can cascade. How would they expect to push all that bandwidth over a single connection? I guess the thought is you're only losing at most the 2GB/s (less with the overhead) difference between a single SAS3 connection (6GB/s) and the bandwidth limitation of PCIe 3.0 x8 (8GB/s).Not really, but the manual tells you that the 4 sas ports on each iom are intended for 4 hosts (theoretically one host on 2 ioms each accessing one zone per pair), and that IOM 1 is the secondary (" SIM 1 does not support SATA device status. ")
Generally speaking, I don't "trust it" until I've tested it. I'd use the MP device id's (in linux maybe? IDK for sure, in TNC mounts should use the part UUID right?)The bigger question is how safe do you think it would be to mount the SATA drives using their multipath device ID if the drives continue to be presented to the OS as such?
This is true. I just have to do more testing to figure out the way my SATA drives are behaving. I'm a little nervous about mounting the SATA drives using their multipath device ID as I don't want to cause anyDont forget the old IOPS vs throughput issue... at some point (user density) you want the former more than then latter
But you actually get 2 sas3 connections if you attach IOM0+1
Yup. Testing is a must. I'll do that this week and report back my findings.Generally speaking, I don't "trust it" until I've tested it. I'd use the MP device id's (in linux maybe? IDK for sure, in TNC mounts should use the part UUID right?)
To test it I guess I'd do the following:
Port1 -> sim 0
Port3 -> sim 1
this should be using both pathways and you should see the SATA drives as mp.
Have some "test" traffic running across an array on the sata drives and disconnect Port 1, see what happens. put it back, disconnect port 3 see what happens etc.
reading a bit more I'm not convinced there are integrated interposers in there and am wondering if the sim expanders are cross connected.
bandwidth - old school thinking / design, LFF = spinning rust.
Well this just brought another thing to my attention. I hadn't even considered there being two controllers on these boards when I updated the firmware. If there are indeed two different controller cores, then my guess is that storcli would see two controllers and thus I need to update the firmware on the second controller as well to match the first.to follow up form the pm thread - LSI 9201-16e/i, LSI 9305-16e/i, LSI 9400-16e/i all appear to share the same physical design pattern which is basically presenting a single pci id device but doubling up the SAS controller cores on the card. In the case of the LSI 9305-24i, I'll guess they tripled up the cores. For reference the 9400-8e/8i refers to the core as a the 3408 and in the 9400-16e/i its referred to as the 3416 however LSI marketing material shows the core for the -16e/i as a 3408 and as the user manuall referred to it then core 0 and core 1 presumably of 3408. Single die per? so two chips or did they cram two 3408's on a single die? dunno - doesn't really matter at that level.
one or more sas controller cores is not necessarily the same as a second HBA which is what I'd expect storcli to see.then my guess is that storcli would see two controllers and thus I need to update the firmware on the second controller as well to match the first.
mpatha (35000cca08c01db20) dm-44 HGST,HBTAC2DH2SUN3.2T
size=2.9T features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 8:0:24:0 sdy 65:128 active ready running
`-+- policy='service-time 0' prio=50 status=enabled
`- 8:0:57:0 sdbe 67:128 active ready running
mpathaa (35000cca266e96620) dm-56 ATA,WDC WD101KRYZ-01
size=9.1T features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 8:0:4:0 sde 8:64 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
`- 8:0:37:0 sdak 66:64 active ready running
Yes the difference was there before..Was that difference there before? I mean hwhandler=0 seems pretty indicative...
I intend to move all my disk config files (fstab, crypttab, multipath, etc) over. But thanks for the tip .Just remember to do the same on the second box too unless you move the multipath config over as well