Enclosure keeps going offline

Discussion in 'RAID Controllers and Host Bus Adapters' started by ccoager, Aug 14, 2018.

  1. ccoager

    ccoager New Member

    Joined:
    Aug 14, 2018
    Messages:
    2
    Likes Received:
    0
    I have a server running Ubuntu 16.04.5 (up-to-date). I recently purchased a LSI MegaRAID SAS 9286-8e (firmware 23.34.0-0019) attached to a HP SAS Expander (firmware 2.06, 2.10) with 15 drives attached. The RAID card, the HBA and 8 of the drives are brand new.

    The drives are configured as single, independent RAID0 from the MegaRAID, for lack of better knowledge. I didn't figure out till months later how to configure them as JBOD from the CLI. I will probably reconfigure them all as JBOD in the future. The storage is configured as zfs RAID-Z with the 8 new drives.

    The problem is, intermittently and often, the alarm goes off on the LSI MegaRAID and all drives go offline simultaneously. Seconds later the enclosure reattaches again. Minutes later the enclosure goes offline again, then reattaches. This happens over and over with no set period of time.

    Troubleshooting I've tried...I've replaced the LSI MegaRAID SAS 9286-8e, tried two different HP SAS expander cards, tried HP SAS expander firmware 2.06 and 2.10, tried a Monoprice SAS SFF-8088 and a Tripp Lite SAS SFF-8088 cable. I'm pretty much out of ideas at this point. Can anyone assist?

    I've uploaded the dmesg, storcli output and event logs from storcli.
     

    Attached Files:

    #1
  2. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,418
    Likes Received:
    200
    Sounds like something in the SAS path rather than power. The HP SAS expander is really old now. I'd look there but I might be totally wrong.
     
    #2
  3. gregsachs

    gregsachs Member

    Joined:
    Aug 14, 2018
    Messages:
    52
    Likes Received:
    7
    I had very similar issue, albeit with windows, where my raid card was set up with pcie link state power management enabled. It would sleep, and the external jbod would go offline. ubuntu look for pci express active-state power management
     
    #3
  4. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,735
    Likes Received:
    847
    If you end-goal is to use ZFS you are doing something fundamentally wrong by first off using a RAID ctrl and not a HBA, definitely do NOT want to have any abstraction/raid in between ZFS and disks, JBOD is back on the right track but I'd just get a HBA and be done with it. If I misunderstood and only your new 8 disks and HBA are connected and you have a raid-z that is fine although I would have gone for raidz2. Not sure what to tell ya on RAID ctrl, not my cup o' tea.
     
    #4
  5. ccoager

    ccoager New Member

    Joined:
    Aug 14, 2018
    Messages:
    2
    Likes Received:
    0
    @whitey
    I checked out the Highpoint HBA's, none of their products had enough SATA channels to meet my requirements. This is typically why you would want a RAID card.
     
    #5
  6. Stefan75

    Stefan75 Member

    Joined:
    Jan 22, 2018
    Messages:
    71
    Likes Received:
    7
    I think whitey meant that there are LSI adapters that are raid only, and others that are pure (dumb) HBA by going into IT mode. I even deleted/removed the BIOS of my 9201-16e because I don't need it ;)
     
    #6
  7. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,735
    Likes Received:
    847
    @Stefan75 ...EXACTLY, LSI HBA's in the 8-16 port variants are what @ccoager should be looking at and YES for faster boot times leave the BIOS off if you don't need to boot off the HBA.
     
    #7
  8. XeonSam

    XeonSam New Member

    Joined:
    Aug 23, 2018
    Messages:
    25
    Likes Received:
    11
    I had this happen to me. But it was for OSNEXUS (Quantastor) and not FreeNas, but using ZFS.
    The problem was the SAS expander. These expanders are made and tested with SAS drives but not SATA drives.
    Entire disk groups would dissapear and when they reappeared they would show up as degraded
     
    #8
  9. Terry Kennedy

    Terry Kennedy Well-Known Member

    Joined:
    Jun 25, 2015
    Messages:
    960
    Likes Received:
    427
    This the "SATA behind SAS expander 'resets for EVERYBODY'" problem, which I described here.
     
    #9
    dawsonkm likes this.
Similar Threads: Enclosure keeps
Forum Title Date
RAID Controllers and Host Bus Adapters Trouble with Enclosure Detection - 9690SA-8i Aug 2, 2018
RAID Controllers and Host Bus Adapters Important lessons learned about SAS3 drives and enclosures. Aug 30, 2017
RAID Controllers and Host Bus Adapters 9207-8i cant find enclosure/backplanes Aug 25, 2016
RAID Controllers and Host Bus Adapters SAS HBA interoperability with backplanes & enclosures (SGPIO,SFF-8485,IBPI etc.) Mar 9, 2016
RAID Controllers and Host Bus Adapters Will and MSA60 enclosure work with LSI 9260-8i Feb 21, 2016

Share This Page