SAS3 Backplaine and HBA Upgrade

Discussion in 'RAID Controllers and Host Bus Adapters' started by avsion, Aug 6, 2019.

  1. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    Hi Guys,

    It's time to upgrade my FN, i have Purchased a used SM CS846 Chassis that came with SAS846TQ backplane. at the moment i'm using Avago/Broadcom SAS3 9300-i8 HBA that connects straight to my 4TB x 8 HDD and have 6 x SSD plugged directly to the SATA ports on the X10SRi-F MOBO, see full description of my FN in my signature. The Goal is high performance, future proof, reliability and neath solution by using SAS3 throughout to get 12Gbps. i would like your advice of the best option available in my case. here is the two i was thinking about

    Option 1 -
    Backplane - BPN-SAS-846A + HBA - LSI 9305-24i + 6 x mini SAS HD cables
    with this option the downfall is the backplane is 6Gbps and the HBA is 12Gbps and i would need to purchase a new HBA. The pros i will have a direct connection to drives i.e every 4 drives connects to one port on the HBA.

    Option 2 -
    Backplane - BPN-SAS3-846EL1 + HBA - LSI 9300-8i + 1 x mini SAS HD cable
    This is my favourite option i can use my HBA, SAS3 throughout to get 12Gbps, very neath solution only one cable better airflow, however as i'm using one uplink for all drives (MAX 24 when i add more) which will cause a bottleneck and limiting my bandwidth.

    please let me know which option i should go with or if there are better options?

    Thank you

    Edit Typo: BPN-SAS3-846ELI to BPN-SAS3-846EL1 thank you @ari2asem

    Hypervisor: ESXi 6.7 U2
    SAN: FreeNAS-11.2 U5
    Case: Corsair Graphite Series 780T White Full-Tower x3 included AF140L 140mm fans and x6 3.5" Bays
    Extra Bays: Custom fit x2 Silverstone 5.25" to 3.5" Drive bay bracket and add Silverstone FP58 Black 5.25"
    Motherboard: SuperMicro X10SRi-F
    CPU: Intel Xeon E5-2683 V3 MHz 2 GHz LGA 2011
    Cooler: Noctua NH-U9DX i4
    PSU: Corsair RMx1000W
    RAM: Crucial 2 x 16GB + 2 x 32GB DDR4 ECC 2400MHz Total 96GB
    HBA: LSI SAS 9300-i8 (Flashed to LSI 9300-i8 P16 IT)
    Pool: 8 x HGST Deskstar 4TB
    ESXi Boot device: Samsung 860 EVO 250GB SSD
    SLOG/L2ARC/Swap: IntelĀ® Optaneā„¢ SSD 900P series 280GB NVMe PCIe
    Datastores: x2 Samsung 860 PRO 256GB SSD , x2 Samsung 860 EVO 250GB SSD, Intel SSD 540 120GB
    USBs: 2 x 32GB Sandisk Cruiser Fit 3.0
    Blu Ray: LG BH16NS55
     
    #1
    Last edited: Aug 6, 2019
  2. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,649
    Likes Received:
    394
    That's not correct. SAS adapters are backwards compatible, they also work with sas 2 & 1.
     
    #2
  3. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    Thank you @i386 for your reply, i meant the other way around that i will not get 12Gbps if i use 6Gbps backplane, correct?
     
    #3
  4. ari2asem

    ari2asem Member

    Joined:
    Dec 26, 2018
    Messages:
    154
    Likes Received:
    13
    the bottleneck of 1 uplink for 24 drives is noticable when you want to access all 24 drives at the same time.
    so, for option 2 and bottleneck is your use-case important.

    but your sm-backplane is EL1, not ELI as you typed. el1 has 2 ports for connecting to hba and 1 expander chip. so, for your 24 drives you only need 1 cable between backplane and hba.

    but you can connect it with 2 cables between hba and backplane, for faster speed. i think your hba supprots port multiplexing
     
    #4
    avsion likes this.
  5. BLinux

    BLinux cat lover server enthusiast

    Joined:
    Jul 7, 2016
    Messages:
    2,293
    Likes Received:
    761
    the BPN-SAS-846A backplane is officially SAS-2 era item, but it is direct connect and so with SAS-3 HBA and SAS-3 storage, it will link at SAS-3 speeds/12Gbps. I've verified this myself using SAS3008, SAS3408, and even HP H240 HBAs and HGST SAS-3 HDDs; link speed showed 12Gbps in all cases. I've also verified this with a 216A backplane using HGST HUSMM series SAS-3 SSDs.
     
    #5
    nikalai and avsion like this.
  6. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    Thank it was a typo fixed.

    regarding connecting 2 cables for faster speed, i have read the SM EL1 PDF and the second port on the EL1 is only for expansion to other backplane, can you confirm that?

    Thank you
     
    #6
    Last edited: Aug 6, 2019
  7. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    @BLinux Interesting good to know, if speed is no issue the 846A is a better solution then the EL1 as i have 6 uplinks (4 drive each) compare to the EL1 1 uplink for 24 drives. correct? however its much more expensive $700 for 9305 i24 HBA
     
    #7
    Last edited: Aug 6, 2019
  8. BLinux

    BLinux cat lover server enthusiast

    Joined:
    Jul 7, 2016
    Messages:
    2,293
    Likes Received:
    761
    it's more a question of what type of storage media you plan to use. HDD? the 9305 would be a total waste. SAS-3 SSDs? then you are definitely looking for 3008 or 3408 LSI controller... but then, the 846 is not really the best choice for 2.5" SFF SSDs... if using HDDs, I would just do something SAS2008 or SAS2308 based LSI controller... i don't know of any HDDs that can even exceed 3Gbps...
     
    #8
    avsion likes this.
  9. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,649
    Likes Received:
    394
    #9
  10. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    @BLinux agree, i'm planing to use both HDD for my storage pool and SSD for VMs running applications like BIND DNS server, UniFi controller Linux Distro..etc. as SSD is expensive for storage i don't see the value in investing as storage in the near future, but diffidently for the VMs. as describe in my OP i already have SAS3008 controller so i think option 2 will be best at this point. i won't need to purchase a new HBA it will support 12Gbps for the VMs, 1 cable connection neath solution and will be good balance HDD with SSD at the same time for one uplink. Just to clarify my aim to achieve SAS3 speed is for future proof i.e. SSDs not for HDD :D
     
    #10
    Last edited: Aug 6, 2019
  11. ClaudAndrus

    ClaudAndrus New Member

    Joined:
    Aug 6, 2019
    Messages:
    5
    Likes Received:
    1
    E backplane only needs a single cable for 24 drives. I think it's clear.
     
    #11
    nikalai likes this.
  12. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,649
    Likes Received:
    394
    E(xpander) backplanes support dual linking with two multiport sas ports.
     
    #12
  13. ClaudAndrus

    ClaudAndrus New Member

    Joined:
    Aug 6, 2019
    Messages:
    5
    Likes Received:
    1
    This is a point too.
     
    #13
  14. ari2asem

    ari2asem Member

    Joined:
    Dec 26, 2018
    Messages:
    154
    Likes Received:
    13
    this is what i meant when i said 2 cables between backplane and hba.

    but i cann't confirm that EL1 will work in this modus, it is NOT mentioned in the manual of el1.
    but this feature (dual linking or port multiplexing) is mostly depeandable of hba card
     
    #14
  15. kapone

    kapone Active Member

    Joined:
    May 23, 2015
    Messages:
    613
    Likes Received:
    243
    "dual linking" a backplane and HBA is technically dependent on BOTH, the HBA and the backplane. However, in general, almost all Supermicro expander based backplanes support it, and almost all LSI (and it's variants) and Adaptec HBAs support it.

    EL1 vs EL2 has nothing to do with it. That has to do with single vs dual expanders, not links.

    Yes, EL1 works with dual links.
     
    #15
  16. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    i just realised there is an option 3

    use my existing BPN-SAS-846TQ REV 3.1 that came with the CS846 Chassis, its a direct attached BPN with 24 dedicated SATA ports. i can use it with HBA - LSI 9305-24i + 6 x mini SAS HD cables to 4 x SATA. if SATA supports 12Gbps i can get full SAS3 bandwidth per port. cabling is not an issue as i already using 2 of the micro cables in a cable sleeve with my 9300-8i HBA, it will be the same for the EL1 cabling. if this is correct it means i'm already future proof for SAS3 SSDs by just upgrading my HBA. Price of the EL1 and 9305-24i is the same but with the HBA i don't have to worry about expender chip limitation as its a passthrough connection. what is your thoughts regarding this option? does someone already using and tested this setup and getting SAS3 Bandwidth?
     
    #16
  17. kapone

    kapone Active Member

    Joined:
    May 23, 2015
    Messages:
    613
    Likes Received:
    243
    Yes, the TQ backplane supports SAS3 12gb/s speeds even though Supermicro will deny it. The cables have a breakout connector that supports both SATA and SAS, depending on what device is attached to it.

    The only difference between the "A" backplane (6x 8087 connectors) vs "TQ" backplane (24x individual SATA connectors and 6x sideband connectors) is cabling. It is cleaner with the A backplane, since the sideband is integrated into the same cable, vs individual sideband connectors on the TQ backplane. If you don't connect the sideband cables on the TQ backplane, the drive "identify" (will not work) and "failure" LEDs may or may not work.

    Now, keep in mind, your chassis holds 3.5" drives, and SSDs are typically...2.5". So, you're going to need converters from 3.5" to 2.5" as well. That adds cost.

    That all being said, the 9305-24i is a pci-e 3.0 x8 card, that can do ~8GB/s (bus top speed).
    • In an A or TQ backplane, just 8x of enterprise SSDs (like HGST or others) will completely saturate the pci-e bus, leaving no headroom for the HDDs that are still there.
    • In an expander based backplane, with dual linking, you'll top out at ~4GB/s, so roughly half the bandwidth, and just 4x SSDs can potentially saturate that link.
    Your call, as to how much bandwidth you need. Keep in mind, even 4GB/s is enough to saturate a 40gb/s network link...
     
    #17
    avsion likes this.
  18. avsion

    avsion New Member

    Joined:
    Sep 2, 2018
    Messages:
    17
    Likes Received:
    2
    Thank you for the detail information. regarding the sideband i have 6 molex connectors coming from the PWS and connected to the TQ BPN is that the individual sideband you referring to? i also saw SFF-8087 to SAS/SATA With Sideband Cable see below link, can i use this type of cable instead of the molex or its better power to the backplane to comes directly from the PWS?
    SFF-8087 to SAS/SATA With Sideband Cable

    I have already include 2.5" trays in my budget, thank you

    i think i will go for option 3 as beside the pros i mention above, there is one more important point we didn't talk is redundancy. with the EL1 end i have one cable connection meaning one point of failure, if this link goes down all the 24 drives goes with it. with the direct attached backplane A or TQ end i have 6 links i.e 5 can still be working (cheaper way instead of using EL2:)).
     
    #18
  19. ari2asem

    ari2asem Member

    Joined:
    Dec 26, 2018
    Messages:
    154
    Likes Received:
    13
    sideband has nothing to do with molex or sata powers.

    sideband is for LED-activity of the disks. if you use sideband, you can see hdd/sdd LED blinking or flashing when the disk is busy. or if the disk has some errors. sideband is for LED-light
     
    #19
    avsion likes this.
  20. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,649
    Likes Received:
    394
    I'm confused and had to read that post a few times...

    What do you mean with redundancy? What does it have to do with EL2 backplanes?
    EL2 enables dual path with sas devices that have two sas ports, so that each device can be access by two hbas.
     
    #20
Similar Threads: SAS3 Backplaine
Forum Title Date
RAID Controllers and Host Bus Adapters any info on this HP SAS3 expander? Mar 29, 2019
RAID Controllers and Host Bus Adapters LSI card differences? (and a sas3 raid5/6 lsi card) Jan 23, 2019
RAID Controllers and Host Bus Adapters SOLVED: SAS3008 under Freenas detecting SATA drives as 1.5 GBps only Dec 22, 2018
RAID Controllers and Host Bus Adapters BPN-SAS3-826EL1 (sas3 expander) questions.. (really a sas3-826el1-n4 , nvme-4) Nov 30, 2018
RAID Controllers and Host Bus Adapters SAS2208 vs. SAS3108 Nov 9, 2018

Share This Page