SAS3 Backplaine and HBA Upgrade

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

avsion

New Member
Sep 2, 2018
17
2
3
Hi Guys,

It's time to upgrade my FN, i have Purchased a used SM CS846 Chassis that came with SAS846TQ backplane. at the moment i'm using Avago/Broadcom SAS3 9300-i8 HBA that connects straight to my 4TB x 8 HDD and have 6 x SSD plugged directly to the SATA ports on the X10SRi-F MOBO, see full description of my FN in my signature. The Goal is high performance, future proof, reliability and neath solution by using SAS3 throughout to get 12Gbps. i would like your advice of the best option available in my case. here is the two i was thinking about

Option 1 -
Backplane - BPN-SAS-846A + HBA - LSI 9305-24i + 6 x mini SAS HD cables
with this option the downfall is the backplane is 6Gbps and the HBA is 12Gbps and i would need to purchase a new HBA. The pros i will have a direct connection to drives i.e every 4 drives connects to one port on the HBA.

Option 2 -
Backplane - BPN-SAS3-846EL1 + HBA - LSI 9300-8i + 1 x mini SAS HD cable
This is my favourite option i can use my HBA, SAS3 throughout to get 12Gbps, very neath solution only one cable better airflow, however as i'm using one uplink for all drives (MAX 24 when i add more) which will cause a bottleneck and limiting my bandwidth.

please let me know which option i should go with or if there are better options?

Thank you

Edit Typo: BPN-SAS3-846ELI to BPN-SAS3-846EL1 thank you @ari2asem

Hypervisor: ESXi 6.7 U2
SAN: FreeNAS-11.2 U5
Case: Corsair Graphite Series 780T White Full-Tower x3 included AF140L 140mm fans and x6 3.5" Bays
Extra Bays: Custom fit x2 Silverstone 5.25" to 3.5" Drive bay bracket and add Silverstone FP58 Black 5.25"
Motherboard: SuperMicro X10SRi-F
CPU: Intel Xeon E5-2683 V3 MHz 2 GHz LGA 2011
Cooler: Noctua NH-U9DX i4
PSU: Corsair RMx1000W
RAM: Crucial 2 x 16GB + 2 x 32GB DDR4 ECC 2400MHz Total 96GB
HBA: LSI SAS 9300-i8 (Flashed to LSI 9300-i8 P16 IT)
Pool: 8 x HGST Deskstar 4TB
ESXi Boot device: Samsung 860 EVO 250GB SSD
SLOG/L2ARC/Swap: Intel® Optane™ SSD 900P series 280GB NVMe PCIe
Datastores: x2 Samsung 860 PRO 256GB SSD , x2 Samsung 860 EVO 250GB SSD, Intel SSD 540 120GB
USBs: 2 x 32GB Sandisk Cruiser Fit 3.0
Blu Ray: LG BH16NS55
 
Last edited:

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
with this option the downfall is the backplane is 6Gbps and the HBA is 12Gbps and i would need to purchase a new HBA.
That's not correct. SAS adapters are backwards compatible, they also work with sas 2 & 1.
 

avsion

New Member
Sep 2, 2018
17
2
3
Thank you @i386 for your reply, i meant the other way around that i will not get 12Gbps if i use 6Gbps backplane, correct?
 

ari2asem

Active Member
Dec 26, 2018
745
128
43
The Netherlands, Groningen
the bottleneck of 1 uplink for 24 drives is noticable when you want to access all 24 drives at the same time.
so, for option 2 and bottleneck is your use-case important.

but your sm-backplane is EL1, not ELI as you typed. el1 has 2 ports for connecting to hba and 1 expander chip. so, for your 24 drives you only need 1 cable between backplane and hba.

but you can connect it with 2 cables between hba and backplane, for faster speed. i think your hba supprots port multiplexing
 
  • Like
Reactions: avsion

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
the BPN-SAS-846A backplane is officially SAS-2 era item, but it is direct connect and so with SAS-3 HBA and SAS-3 storage, it will link at SAS-3 speeds/12Gbps. I've verified this myself using SAS3008, SAS3408, and even HP H240 HBAs and HGST SAS-3 HDDs; link speed showed 12Gbps in all cases. I've also verified this with a 216A backplane using HGST HUSMM series SAS-3 SSDs.
 

avsion

New Member
Sep 2, 2018
17
2
3
the bottleneck of 1 uplink for 24 drives is noticable when you want to access all 24 drives at the same time.
so, for option 2 and bottleneck is your use-case important.

but your sm-backplane is EL1, not ELI as you typed. el1 has 2 ports for connecting to hba and 1 expander chip. so, for your 24 drives you only need 1 cable between backplane and hba.

but you can connect it with 2 cables between hba and backplane, for faster speed. i think your hba supprots port multiplexing
Thank it was a typo fixed.

regarding connecting 2 cables for faster speed, i have read the SM EL1 PDF and the second port on the EL1 is only for expansion to other backplane, can you confirm that?

Thank you
 
Last edited:

avsion

New Member
Sep 2, 2018
17
2
3
the BPN-SAS-846A backplane is officially SAS-2 era item, but it is direct connect and so with SAS-3 HBA and SAS-3 storage, it will link at SAS-3 speeds/12Gbps. I've verified this myself using SAS3008, SAS3408, and even HP H240 HBAs and HGST SAS-3 HDDs; link speed showed 12Gbps in all cases. I've also verified this with a 216A backplane using HGST HUSMM series SAS-3 SSDs.
@BLinux Interesting good to know, if speed is no issue the 846A is a better solution then the EL1 as i have 6 uplinks (4 drive each) compare to the EL1 1 uplink for 24 drives. correct? however its much more expensive $700 for 9305 i24 HBA
 
Last edited:

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Interesting good to know, if speed is no issue the 846A is a better solution then the EL1 as i have 6 uplinks (4 drive each) compare to the EL1 1 uplink for 24 drives. correct? however its much more expensive $700 for 9305 i24 HBA
it's more a question of what type of storage media you plan to use. HDD? the 9305 would be a total waste. SAS-3 SSDs? then you are definitely looking for 3008 or 3408 LSI controller... but then, the 846 is not really the best choice for 2.5" SFF SSDs... if using HDDs, I would just do something SAS2008 or SAS2308 based LSI controller... i don't know of any HDDs that can even exceed 3Gbps...
 
  • Like
Reactions: avsion

avsion

New Member
Sep 2, 2018
17
2
3
@BLinux agree, i'm planing to use both HDD for my storage pool and SSD for VMs running applications like BIND DNS server, UniFi controller Linux Distro..etc. as SSD is expensive for storage i don't see the value in investing as storage in the near future, but diffidently for the VMs. as describe in my OP i already have SAS3008 controller so i think option 2 will be best at this point. i won't need to purchase a new HBA it will support 12Gbps for the VMs, 1 cable connection neath solution and will be good balance HDD with SSD at the same time for one uplink. Just to clarify my aim to achieve SAS3 speed is for future proof i.e. SSDs not for HDD :D
 
Last edited:

ari2asem

Active Member
Dec 26, 2018
745
128
43
The Netherlands, Groningen
E(xpander) backplanes support dual linking with two multiport sas ports.
this is what i meant when i said 2 cables between backplane and hba.

but i cann't confirm that EL1 will work in this modus, it is NOT mentioned in the manual of el1.
but this feature (dual linking or port multiplexing) is mostly depeandable of hba card
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
"dual linking" a backplane and HBA is technically dependent on BOTH, the HBA and the backplane. However, in general, almost all Supermicro expander based backplanes support it, and almost all LSI (and it's variants) and Adaptec HBAs support it.

EL1 vs EL2 has nothing to do with it. That has to do with single vs dual expanders, not links.

Yes, EL1 works with dual links.
 

avsion

New Member
Sep 2, 2018
17
2
3
i just realised there is an option 3

use my existing BPN-SAS-846TQ REV 3.1 that came with the CS846 Chassis, its a direct attached BPN with 24 dedicated SATA ports. i can use it with HBA - LSI 9305-24i + 6 x mini SAS HD cables to 4 x SATA. if SATA supports 12Gbps i can get full SAS3 bandwidth per port. cabling is not an issue as i already using 2 of the micro cables in a cable sleeve with my 9300-8i HBA, it will be the same for the EL1 cabling. if this is correct it means i'm already future proof for SAS3 SSDs by just upgrading my HBA. Price of the EL1 and 9305-24i is the same but with the HBA i don't have to worry about expender chip limitation as its a passthrough connection. what is your thoughts regarding this option? does someone already using and tested this setup and getting SAS3 Bandwidth?
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
Yes, the TQ backplane supports SAS3 12gb/s speeds even though Supermicro will deny it. The cables have a breakout connector that supports both SATA and SAS, depending on what device is attached to it.

The only difference between the "A" backplane (6x 8087 connectors) vs "TQ" backplane (24x individual SATA connectors and 6x sideband connectors) is cabling. It is cleaner with the A backplane, since the sideband is integrated into the same cable, vs individual sideband connectors on the TQ backplane. If you don't connect the sideband cables on the TQ backplane, the drive "identify" (will not work) and "failure" LEDs may or may not work.

Now, keep in mind, your chassis holds 3.5" drives, and SSDs are typically...2.5". So, you're going to need converters from 3.5" to 2.5" as well. That adds cost.

That all being said, the 9305-24i is a pci-e 3.0 x8 card, that can do ~8GB/s (bus top speed).
  • In an A or TQ backplane, just 8x of enterprise SSDs (like HGST or others) will completely saturate the pci-e bus, leaving no headroom for the HDDs that are still there.
  • In an expander based backplane, with dual linking, you'll top out at ~4GB/s, so roughly half the bandwidth, and just 4x SSDs can potentially saturate that link.
Your call, as to how much bandwidth you need. Keep in mind, even 4GB/s is enough to saturate a 40gb/s network link...
 
  • Like
Reactions: avsion

avsion

New Member
Sep 2, 2018
17
2
3
Thank you for the detail information. regarding the sideband i have 6 molex connectors coming from the PWS and connected to the TQ BPN is that the individual sideband you referring to? i also saw SFF-8087 to SAS/SATA With Sideband Cable see below link, can i use this type of cable instead of the molex or its better power to the backplane to comes directly from the PWS?
SFF-8087 to SAS/SATA With Sideband Cable

I have already include 2.5" trays in my budget, thank you

i think i will go for option 3 as beside the pros i mention above, there is one more important point we didn't talk is redundancy. with the EL1 end i have one cable connection meaning one point of failure, if this link goes down all the 24 drives goes with it. with the direct attached backplane A or TQ end i have 6 links i.e 5 can still be working (cheaper way instead of using EL2:)).
 

ari2asem

Active Member
Dec 26, 2018
745
128
43
The Netherlands, Groningen
sideband has nothing to do with molex or sata powers.

sideband is for LED-activity of the disks. if you use sideband, you can see hdd/sdd LED blinking or flashing when the disk is busy. or if the disk has some errors. sideband is for LED-light
 
  • Like
Reactions: avsion

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
i think i will go for option 3 as beside the pros i mention above, there is one more important point we didn't talk is redundancy. with the EL1 end i have one cable connection meaning one point of failure, if this link goes down all the 24 drives goes with it. with the direct attached backplane A or TQ end i have 6 links i.e 5 can still be working (cheaper way instead of using EL2:)).
I'm confused and had to read that post a few times...

What do you mean with redundancy? What does it have to do with EL2 backplanes?
EL2 enables dual path with sas devices that have two sas ports, so that each device can be access by two hbas.