My new setup, questions on details - XFS, SAS3 expander, SAS3 HBA

[Lleo_]

Hi All,
Long time reader - first post!
I have received the following parts from which I am building an additional storage enclosure to attach to my current server.
- Norco DS-24D
- Intel RES3TV360 SAS3 expander
- Intel RS3GC008 HBA

My current server is a Supermicro X9SIL-F with a L3426 Xeon. Using a PCIE M.2 adapter as boot drive and 6 SATA2 storage drives attached to the onboard SATA2 ports with software raid6
It runs linux and for most is a home server, serving media, DVR storage, etc. with occasional virtual machines also running.

The HBA has an IT firmware. I populated 12 of the 24 slots and these are visible to the server.
My questions (and sorry if these sound like a noob which I am when it comes to storage)
1. Is there benefit in connecting both external ports of the HBA to the SAS expander?

2. Plan to use ZFS with multiple vdevs, but if I add an L2ARC cache, should it be attached to onboard SATA2 port or expander ?

3. The only tool to read the HBA under linux is "sas3ircu" but provides very limited information. Is there another tool that may work with this HBA, to verify sync rates, channels, what not?

Thanks!

 

[Lleo_]

Happy New Year to All!

It seems I did not have the best opening post...
In the meantime read more and found some relevant information on SAS expanders, among others here on STH. In the HP SAS expander wiki it is stated that only 1 cable is needed between expander and HBA, unless link aggregation is supported by the HBA.
So my question now is whether RS3GC008 does support link aggregation?
I tried with both ports connected to the expander and could not notice a difference.
Thanks!

 

[Lance Joseph]

Happy New Year to All!

It seems I did not have the best opening post...
In the meantime read more and found some relevant information on SAS expanders, among others here on STH. In the HP SAS expander wiki it is stated that only 1 cable is needed between expander and HBA, unless link aggregation is supported by the HBA.
So my question now is whether RS3GC008 does support link aggregation?
I tried with both ports connected to the expander and could not notice a difference.
Thanks!

Hey Lleo_,
1) Depending on your workload, you may see higher IOPS if you connect both mini-SAS3 connectors to the expander. Either way, it can't hurt.

2) You can attach the drive where you see fit. If the internal controller isn't being used then I would probably use it just to save IOPs for my HBA.

3) I'm not sure what you mean by sync rates and channels. 'sas3ircu 0 DISPLAY' should provide a wealth of information. You can also look at individual drive SMART information with 'smartctl' - smartmontools by the package manager.
You can also check out Brendan Gregg who wrote a nice book called Systems Performance. (Incidentally, he helped to write the code for ZFS's L2ARC, created Dtrace, etc.). In addition to charts on performance analysis, he also has a wealth of documentation on his website: Linux Performance

Could you please link to that wiki article on HP SAS expanders?

Thanks

 

[Lleo_]

Thanks for your reply!
The is no hidden wiki on STH, rather I was referring to a post on this forum titled 'wiki' from where I quoted.
https://forums.servethehome.com/index.php?threads/hp-sas-expander-wiki.146/