SAS2 expanders $60 (IBM, LSI chip, Intel alternative)

Discussion in 'Great Deals' started by sfbayzfs, Sep 20, 2016.

  1. Davidtt485

    Davidtt485 New Member

    Joined:
    Jan 6, 2019
    Messages:
    3
    Likes Received:
    0
    Bought one of these units, had it come with the 602a Firmware and had issues detecting different SATA drives in all of the ports, flashed it and now i have no issues it detects all drives. I am even using one of the top ports for two Intel SSDs

    Setup is 9207-4e4i connect internally to the top first port of the sas expander
    Drives connected to all other ports, using a Norco 4220
     
    #121
  2. ari2asem

    ari2asem New Member

    Joined:
    Dec 26, 2018
    Messages:
    21
    Likes Received:
    1
    in short: keep away from this expander. buy intel res2sv240. or chenbro sk22-serie expander card
     
    #122
  3. Davidtt485

    Davidtt485 New Member

    Joined:
    Jan 6, 2019
    Messages:
    3
    Likes Received:
    0
    I wouldn’t say that in all in fact I plan on buying another one, with a relatively easy firmware upgrade this performs excellent and can support twenty drives. Price vs performance is awesome with this it cost me like 21 bucks
     
    #123
  4. MichalPL

    MichalPL New Member

    Joined:
    Feb 10, 2019
    Messages:
    3
    Likes Received:
    0
    Hi, I need your help - I am not 100% if it's an Expander issue, but looks like something between LSI card and expander.
    Short background: I am trying to build (I hope :) ) fast NAS server. Final configuration: Ubuntu ZFS, 16x Toshiba N300 8TB, LSI 9217-8i+IBM expander, 3x NVME Evo970 for cache, Mellanox 100Gb/s, 2x Xeon E5 8core 4.0GHz, 512 DDR3 1.8GHz. In theory it should go ~3.9GBytes/s from HDD and around 6.4Gbytes/s from cache (CPU too slow, need ~6GHz overclocking single core performance for max speed) and 11Gbytes/s from RAM.

    Temporary I have connected old 16x WD 0.5TB 7200rpm (~110MBytes/s) drives or 14x SSD (~520MBytes/s) for testing an the problem is here:
    [ 1826.294940] print_req_error: I/O error, dev sdm, sector 63248
    [ 1826.294968] sd 0:0:12:0: [sdm] tag#62 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK
    [ 1826.294971] sd 0:0:12:0: [sdm] tag#62 CDB: Write(10) 2a 00 00 01 00 10 00 01 00 00
    [ 1826.294973] print_req_error: I/O error, dev sdm, sector 65552
    [ 1826.294996] sd 0:0:12:0: [sdm] tag#5 FAILED Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

    When I am using HDD very intense IO errors arriving (if using only 8 HDD from array everything is fine, when more I have IO errors)
    card is LSI9217-8i connected to IBM expander 46M0997 (510A) via 2 cables (same issue with 1 cable, but works slower).

    No issues when make 2 ZFS volumes 8xHDD in first and 8x in second when using one volume at the time (when using two at same time only at full speed - same issues).

    I bought LSI 9201-16e, and everything works fine, but due to PCI-e 2.0 x8 the real speed is 2.9Gbytes/s (too slow :/)
    with LSI 9217-8i on 8x ssd the real speed is 4.4Gbytes/s.
    with LSI 9217-8i Expander it is 3.85Gbytes/s but IO errors are slowing down ZFS or make errors.

    I have same issue with HP 487738-001 Expander (IO errors), and same issue when connecting (any) expanders to LSI 9201-16e :/ but here everything works fine up to 15HDD, when using 16 same type of IO errors.

    I am using standard firmware in LSI cards, is the IT mode faster or will fix issues? (I see I have reach max speeds of interfaces on LSI 9201-16e, IO/s not tested yet) is the IT mode also support expanders ?

    I don't know what is wrong :/ The only solution I have at this moment is 16 port 6gbps card with 8 pcie 3.0 lanes or 16 pcie 2.0, means LSI 9206-16e or LSI 9202-16e but it means cables outside PC (RGB lol !!!) case. Thank you !
     
    #124
  5. ari2asem

    ari2asem New Member

    Joined:
    Dec 26, 2018
    Messages:
    21
    Likes Received:
    1
    i/o error could be more than 1 reason, or maybe combination of more reasons.

    1). check SMART info of the hdd/ssd
    2). use another cables (another brand?)
    3). try it with another expander (intel, adaptec, astek, areca) or try with another hba-chipset (like adaptec/microsemi)
    4). enough airflow for heating-"problems" of sas-chipset. maybe passive cooling of sas-chipset isn't enough and then u get weird errors because too high temperature (sometimes 85-90° C) of sas-chipset (on both cards hba and expander)
    5). maybe some problems with pci-e slot of mainboard? or weird and undocumented mainboard incompatibility with lsi-hba's.

    https://forums.servethehome.com/index.php?threads/boot-issue-with-lsi-9201-16i.21255/

    as a good example of point 5
     
    #125
  6. MichalPL

    MichalPL New Member

    Joined:
    Feb 10, 2019
    Messages:
    3
    Likes Received:
    0
    Thank you for a quick answer :)

    all disks are healthy, ale quite cold (50°C when fully loaded)

    I have 2 different brands :/ tested already,
    also works fine when half of the disks are fully loaded, even if it was like this:
    cable1: loaded, laded, loaded, loaded
    cable2: loaded, laded, loaded, loaded
    cable3: not working,not working,not working,not working
    cable4: not working,not working,not working,not working

    also works fine when:
    cable1: not working,not working,not working,not working
    cable2: not working,not working,not working,not working
    cable3: loaded, laded, loaded, loaded
    cable4: loaded, laded, loaded, loaded


    hmmm sounds like good idea.

    Just tested IBM and HP, both at the same time, and same issues :/

    LSI 9217-8i
    *cable1: IBM expander
    *cable A: disk1, disk2, disk3, disk4
    *cable B: disk5, disk6, disk7, disk8
    *cable2: HP expander
    *cable A: disk9, disk10, disk11, disk12
    *cable B: disk13, disk14, disk15, disk16

    yes I have big and slow 14cm fan on all cards (currently setup is on a table) you are able to touch it I think they are not more than 45°C.

    hmmm, I will read it.

    At this moment tested on 2 main boards:

    One is quite nice chinese something huanan "x79" (C600 chipset) orange one but it supports only one CPU and only 64GB RRD3. Tested with Xeon E5 1650 v2 @ 4.1GHz.

    the issue here is lack of memory (don't support 32GB modules, and only 4 slots) and pci-e lanes used in 100%:
    16x for Mellanox Connect-X 4 100gb/s
    16x for asus 4 nvme board for 3x NVME (maybe in final design will be 4x nvme, but CPU is too slow to make any sense)
    and only 8x left for HBA - this is why I choose 9217-8i

    and second (probably final choice) will be mainboard from Fujitsu Celcius R920, because CPUs are too slow to decode ZFS (Linux implementation is crappy :/) and only method to speedup is lot of RAM.
    2.8GB/s per single thread read/write (on overclocked 1650v2 and overclocked Threadripper)
    ~6.4GB/s max on 6+ threads (when reads 10.5GB/s from 3x nvme while no file system).

    dual core, it's support 32GB modules and have 16 ports.
    cpu0 pcie: x16 x16 x4
    cpu1 pcie: x16 x16 x8
    but I am not able to select pci-e slots configuration in bios so each nvme will consume full slot.

    already tested with 1x E5 1650 v2 and 2x E5 2640, (waiting for E5 2667 v2 :) )- same issues :/

    maybe I can use 2 cards, but I am worry about sending data between CPUs, what I read is LSI 9206-16e
    is using 2 chipsets from 9217 but connected 4x pcie 3.0 each (card is 8x pcie 3.0) = 7.2GB/s real speed limit = more than I need (243MB/s for each Toshiba HDD + 128/256 MB fast 550MB/s HDD buffer). but still better to use 9217+expander somehow - it's fast enough and I have it already :)

    thank you
     
    #126
  7. TrevInCarlton

    TrevInCarlton New Member

    Joined:
    Sep 19, 2018
    Messages:
    16
    Likes Received:
    4
    You need to update the firmware on the IBM expander 46M0997 to 634a. I had weeks of problems until I did the firmware update. I am running Freenas without any problems now after updating the firmware. This is not that easy but is well covered in this thread, particularly all the input by "The Bloke". I could not have sorted out my problems without his input.
     
    #127
    Last edited: Feb 11, 2019 at 2:03 AM
  8. RedX1

    RedX1 New Member

    Joined:
    Aug 11, 2017
    Messages:
    10
    Likes Received:
    3
    Hi

    A good video tutorial of how to update the IBM 46M0997 Expander to 634a Firmware can be found here.






    It worked well for me, you might find this very useful too.





    RedX1
     
    #128
    epicurean likes this.
  9. Davidtt485

    Davidtt485 New Member

    Joined:
    Jan 6, 2019
    Messages:
    3
    Likes Received:
    0
    The fact that you are having the same issues with two different expanders means its either the HBA or the Drives, Because this problem appears when you are adding drives I vote for HBA issue, possibly due to over heating, I have had good luck with re applying thermal paste.
     
    #129
  10. MichalPL

    MichalPL New Member

    Joined:
    Feb 10, 2019
    Messages:
    3
    Likes Received:
    0
    Gentleman's it's working ! updating to 634a fixed the problem ! ... kind of.

    ... Meanwhile I have received the 4.0GHz 8 core "old" CPU (Xeon E5 2667 v2) and tested performance and the 634a firmware not looks good :/

    LSI 9217 -> IBM 46M0997 -> 16x old WD HDD 7200 0.5TB its fully working now (HP 487738-001 with updated firmware to 2.10 still have same issues). but the performance is very bad :/ (also tested with 10x ssd).

    in case of HDD instead of 1.9GB/s now I have about 1.4GB/s, and with SSD instead of 3.8GB/s it's 2.5GB/s and the maximum speed per one hdd is 263MB/s instead of 520MB/s. (they are still connected as SATA3 - 6Gbit/s), Expander was connected via 2 cables. The LSI card still working 4.4GB/s with 8x ssd without Expander.

    maybe... it will be ok for Toshiba N300 where it will be 243MB/s but still slows down an older HDD a lot with ~115MB/s bandwidth a lot = need faster solution.
     
    #130
Similar Threads: SAS2 expanders
Forum Title Date
Great Deals CSE-826 w SAS2-826EL1 , no PSU/Trays - €60/BO Aug 18, 2018
Great Deals BPN-SAS2-846EL1 $175 incl shipping (7 avail) Jul 11, 2018
Great Deals BPN-SAS2-826EL1 - $36 May 12, 2018
Great Deals Supermicro BPN-SAS2-846EL1 $200 + $13 shipping May 9, 2018
Great Deals OCZ Talos 2 960gb SAS2 SSDs @ $180/ea Mar 26, 2018

Share This Page