Enclosure with 2x Broadcom SAS3x28R expanders, dual link?

[nihonjin]

Hello, I've got an AIC J3016-01-03X with dual expanders and I wonder if anyone knows if dual link is possible with those. Will I need 2 controllers? or with one would do the trick. Before I do start the adventure of buying cables, would be great to know.

Some photos (mine doesn't have BMC)

 

[ari2asem]

dual expander is for sas drives. like high availability, in case one expander dies you stil can work with your jbob because of dual ports of sas drives.

dual port doesn't mean dual performance

 

[nihonjin]

I know, all my drives on that enclosure will be sas drives. But as in other expanders (supermicro etc), some people connect two cables in order to get dual link and get more bandwidth, I wondered if besides redundancy I could got that extra with an extra cable

 

[Sean Ho]

If it's like the SM EL2 backplanes, the second expander is for failover rather than additional bandwidth. If you're after bandwidth, you can dual-link from your HBA to one of the expanders/IOMs: 2x 8644-8644 cables, from the two ports on your -8e HBA to the two ports on a single IOM in the DAS.

 

[nihonjin]

Thanks, that could be great

 

[Rand__]

dual expander is for sas drives. like high availability
dual port doesn't mean dual performance

Thats not entirely correct - if you have a dual port SAS drive and run it multipathed (on an EL2 to 2 HBAs for example) you can utilize both ports at the same time and exceed single port SAS performance.

How much you gain depends on the drive, but some SAS3 drives claim 1600+ MB/s which is not reachable with a single port.

 

[nihonjin]

Thats not entirely correct - if you have a dual port SAS drive and run it multipathed (on an EL2 to 2 HBAs for example) you can utilize both ports at the same time and exceed single port SAS performance.

How much you gain depends on the drive, but some SAS3 drives claim 1600+ MB/s which is not reachable with a single port.

so, you need two HBAs?

 

[Rand__]

for multipathing? Yes. And OS support (which most modern OS' have)

 

[nihonjin]

so... my motherboard has a 3008 HBA on board (MD70-HB0 (rev. 1.2)), the enclosure has a SAS3 backplane (Datto RM23812) which is connected to the motherboard SAS controller through 2 SFFs... if I do install a second 3008 (destined to be connected to the AIC J3016-01-03X) card and cross connect cables between both backplanes (1 to each SAS controller), would get dual link in... both backplanes?

 

[Rand__]

I have no idea how they will work that out
I'd prolly just do 1 uplink to each enclosure & J3016) from each of your two controllers
That would imply that you evenly distribute drives in enclosure and J3016 to distribute the data to both locations to get the maximum theoretical performance

 

[nihonjin]

I have no idea how they will work that out
I'd prolly just do 1 uplink to each enclosure & J3016) from each of your two controllers
That would imply that you evenly distribute drives in enclosure and J3016 to distribute the data to both locations to get the maximum theoretical performance

and cheaper

 

[Rand__]

less cables, yes

 

[Rand__]

But that means that an External HBA might not your best choice, better get an internal one and then an Int to Ext adapter which then can connect the two hbas to the external box

 

[nihonjin]

But that means that an External HBA might not your best choice, better get an internal one and then an Int to Ext adapter which then can connect the two hbas to the external box

true

 

[nihonjin]

But that means that an External HBA might not your best choice, better get an internal one and then an Int to Ext adapter which then can connect the two hbas to the external box

Bad news, dorsn't work, some drives appears as locked. No problem if each HBA in connected to only one expander.

 

[Rand__]

So whats the current setup and which ones behave how?

 

[nihonjin]

Two 3008 (9300-8i) one on board, both on IT mode flashed with h16. The issue must be on the Datto RM23812 expander, there is were the "blocked" drives appear if I do the "dual link wichtcraft" (just where I wanted to install the SSDs).

 

[Rand__]

So each hba connected to one controller of the Datto. All drives are sas dual ports.
What do the sg_utils say about layout? Do the drives show up on both paths?

 

[nihonjin]

Can't check sg_utils atm, I'll test... tomorrow's early morning. But yeah, both controllers showed all the drives.

All drives are SAS, the SSD's are like the Samsungs I have on my market thread and the drives are SAS2 4TB Seagate ST4000NM0043.

 

[nihonjin]

btw didn't solve the issue with the sas controller/drives, on it mode can't create anything over linux (blk_update_request: protection error, and something about the cache), no problem at all on ir mode