Maximum # harddisk a motherboard can handle with 2 SAS expanders

Discussion in 'RAID Controllers and Host Bus Adapters' started by denywinarto, Oct 7, 2016.

  1. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    I'm new to this JBOD thing so bear with me..
    Planning to build gigantic Media server running Ubuntu.
    I just bought 1 HP SAS expander and an IBM M1015 SAS Card which has 2 SFF8087 ports, meaning it can hold up to 2 SAS expanders...
    I've measured the mobo and it looks like 2 SAS expander could fit in it.. and because of certain scripts i want to run, i'm limiting my library to 1 computer.

    Here's the specs :

    Motherboad : Asrock Z87
    Memory : 16GB 1600MhZ DDR3 G-skill
    Processor : Intel® Core™ i3-4130 Processor (3M Cache, 3.40 GHz)
    PSU: Plan to use 2-3 PSUs connected with Add2Psu..


    So technically that's 64 HDD from SAS expander + 10 from motherboard internal SATA ports.

    Questions:

    1. Is there any known limitation from motherboard or other factors that i'm not aware of? Is there anything that could prevent 74 HDD from working properly with this setup? (It will be filled with 8TB and 6TB of HDDs btw)..
    2. Is it Ok to run SATA from SAS expander and from internal SATA port together?
    3. Is non-ECC memory viable enough?

    I don't mind a little performace drop as long as i can stream 1080p content
    I'm asking this because i want to build acrylic custom casing, and of course i need to know the max capacity to detemine the size of the casing

    thanks
     
    #1
  2. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,681
    Likes Received:
    411
    #2
    KioskAdmin and denywinarto like this.
  3. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    #3
  4. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,681
    Likes Received:
    411
    I forgot to ask what os you want to use.
    If you want to go the zfs route you could flash the controller with lsi it/ir firmware, it should then work as an hba and support more drives.
     
    #4
  5. Sleyk

    Sleyk Active Member

    Joined:
    Mar 25, 2016
    Messages:
    793
    Likes Received:
    218
    SAS expanders allow for a butt load of connected drives. You would pretty much run out of room to fit drives before you approach the limit of what a hba card can handle/connect to. (At least in most single chassis setups) I think the number runs in the hundreds I believe when I last read it awhile ago.

    Bandwidth, on the other hand will take a hit as you keep adding drives as that, as you know, has a smaller, fixed limit based on the pcie bus/slots.

    Not too sure, but doesn't Ubuntu have zfs capability? Don't quote me! Heheh.

    Should be fine my peep or peepette!
     
    #5
  6. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    I'm using ubuntu 16.04 with ext4 filesystem.. already have over 40 TB of data in 10 HDDs..
    Are there any limitations for ext4?

    Any links for the thread? perhaps i could PM him..
    Ubuntu does have zfs.. problem is i already have over 40TB of data in ext4
     
    #6
  7. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,764
    Likes Received:
    858
    Pretty sure LSI 2008 HBA ctrls in IT mode (which a M1015 is in the family of) support 256 devices max.
     
    #7
    denywinarto likes this.
  8. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    I see.. i bought it from ebay and i think its chinese made.. do i need to flash it to get past 32 drives limit?
    I tested it yesterday with hp sas expander and 1 sata drive and it works...

    But then again 256 kinda hard to achieve.. i havent seen any expanders that is capable of 16 sff 8087 slots to achieve 64 hdd per expanders...

    Got another question about psu..
    Is it safe to use 4 splitter on 6x6 sata slots like this to achieve to 24 hdd per psu ?
    PRIME 850 W Titanium | Seasonic PRIME

    I will then pair the PSu's with add2psu
     
    #8
  9. Jb boin

    Jb boin Member

    Joined:
    May 26, 2016
    Messages:
    49
    Likes Received:
    16
    If you want to make it simple, you can use a SAS storage array such as a HP MDS600 : HP MDS600 Storage Array (Configure-To-Order)
    It directly provides PSUs, cooling and you have hotswap trays and LEDs on the front panel for every disks.

    We bought one of these with 70*2Tb, plugged to a P812, it has been working fine for the last 2 or 3 months.
     
    #9
    KioskAdmin and denywinarto like this.
  10. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    Now thats interesting !
    Unfortunately i dont live in US so the import cost is gonna be quite alot, since it's 160 lbs.. probably could double the original cost... i'll have to figure out a way to get this to a sea freight maybe..
    A question..
    Can we use our own motherboard + sas card + sas expander on this case?
     
    #10
  11. KioskAdmin

    KioskAdmin Active Member

    Joined:
    Jan 20, 2015
    Messages:
    156
    Likes Received:
    32
    I'd get a sas expander chassis with everything built-in. You're building something with many failure points. SAS cards don't always play well with consumer mobos.
     
    #11
  12. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    336
    The limit someone posted earlier of 256 devices for the M1015 HBA sounds accurate, assuming the card is flashed to IT-mode. In IR-mode (raid mode), that limit will be much much lower (32 is probably right). I've only taken my M1015 as high as 28 drives, though I also have a LSI-9201-16e here (same generation of SAS controller) with 34 drives connected, which at least pushes over that 32-drive limit.

    As for physically connecting them all, you can daisy-chain expanders (that is what that HP MDS600 linked above does), which should easily get you to as many drives as you could possibly want. As far as I know the biggest expander chips out are the LSI 36-port ones, so anything with more than around 30 drives (you need ports to uplink to HBAs, and more ports to link between expanders if there are multiple) is going to be based on multiple expanders..
     
    #12
  13. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    Thanks for the inputs.. are there any alternatives for MDS600? At least a lighter alternative?
    Importing that chassis will triple the cost (bc of the weight) from my calculations :(

    Hmm also after some googling i learnt that this thing isn't as flexible as norco in terms of spareparts.. seems like norco is more universal... meaning i can use mobo sas expander etc from another brand.. But the max norco chassis is 24 HDD
    Is there any other universal storage rack like this?
     
    #13
    Last edited: Oct 14, 2016
  14. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    336
    There are lots of high density disk expansion shelves on the market, I think pretty much every brand has their own version. And they're probably all fairly heavy - weight is not one of the factors I consider when looking at gear so I don't have any advice there.

    That is the trade-off with having the SAS expander chip be integrated into the drive backplane. You lose flexibility in part selection - you have to use the backplane that fits your chassis, but you have fewer parts (don't need to fit an expander card elsewhere, potentially using expansion slots for power, etc.), and get better integration (eg. being able to tell the backplane to turn on the locate LED on the bay associated with port 5). My preference is to have the expander be part of the backplane.

    As for a "universal" storage chassis - with a few little parts you can turn any chassis into an add-on disk chassis. All you really need is a way to turn the PSU(s) on to power drives/expander(s), and a SFF8087/8088 bracket to be able to connect an external SAS cable from your compute node to the expander inside the storage shelf. It could be as simple as an old desktop case with a paperclip to short green to black in the ATX connector to force the PSU to always-on, or as fancy as a 4U supermicro case with up to 90 hotswap drive bays, and a CSE-PTJBOD-CB3 board in it to control power, fans, remote-management, etc.
     
    #14
    denywinarto likes this.
  15. denywinarto

    denywinarto Member

    Joined:
    Aug 11, 2016
    Messages:
    129
    Likes Received:
    7
    I'm about to buy a used MDS600 with 2x I/O SAS module 455972-001/AP763A...
    My main concern is the PSUs.. they age faster than other components so i'll have to replace them sooner..

    So is there absolutely no way to use regular PSU on MDS600?

    I could 've made a custom casing with acrylic but i kinda worried that the vibration will kill the HDDs
     
    #15
  16. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    336
    Nope - only the ones specifically built for it.
     
    #16
Similar Threads: Maximum harddisk
Forum Title Date
RAID Controllers and Host Bus Adapters Maximum SATA disk size for use with Dell XX2X2 RAID controller? Oct 10, 2017
RAID Controllers and Host Bus Adapters Areca - Volume Cannot Be Extended - Exceed Maximum Clusters Feb 10, 2016
RAID Controllers and Host Bus Adapters Maximum Distance Between Workstation and RAID Server Jan 2, 2016
RAID Controllers and Host Bus Adapters M1015 Maximum physical HDD capacity? Oct 16, 2015
RAID Controllers and Host Bus Adapters IBM ServeRAID M1015/LSI MegaRAID 9220-8i - Maximum HDD Size Oct 30, 2013

Share This Page