New storage box for mixed workload

Discussion in 'NAS Systems and Networked Home and SMB Software' started by vrod, Jan 26, 2018.

  1. vrod

    vrod Active Member

    Joined:
    Jan 18, 2015
    Messages:
    205
    Likes Received:
    29
    Hi all,

    I was lucky and got my hands on a 4U supermicro case with 36 bays. Gonna pop a s2600cp in that case with a couple of e5 v2 cpu’s to make a new storage box.

    I have been wondering how to setup it up, been using both freenas and ubuntu to serve nfs and iscsi for some time to my current vsphere env.

    I plan to use 36x3TB HGST Ultrastars and a P3700 ssd (cache) but I’m not sure what would be best. A lot of people recommend going for mirroring, so I think I would do that too.

    Workload will be UHD movies and VM’s (vSphere).

    What would be the best config for the hdds? 3x12,1x24 + 1x12 or 1x36 pool design?

    What OS is the better one to serve either iscsi or nfs? I’m a little tipped on ubuntu but also had some good experiences with freenas.

    In that case, iSCSI or NFS? What do people recommend?

    If I had several pools, it probably wouldn’t be a problem to split the P3700 into several partitions and use it for slog for all the pools, or?
     
    #1
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,679
    Likes Received:
    745
    Raid 10 and p3700 for VMs.

    He's for movies in z2

    If you do use the p3700 for l2arc on movie storage you'll need to set fill rate so you can have a chance to cache movies.
     
    #2
  3. vrod

    vrod Active Member

    Joined:
    Jan 18, 2015
    Messages:
    205
    Likes Received:
    29
    Thanks for your input! How many vdevs would I make in the raidz2? just 1 or several? The box will have 256gb of memory so hopefully I won't need l2arc. I stream 4K UHD movies which can be up to about 70Gb, so I guess that would also fit entirely in memory. I never heard about "fill rate" for l2arc, would you be able to elaborate? :)
     
    #3
  4. Connorise

    Connorise Member

    Joined:
    Mar 2, 2017
    Messages:
    43
    Likes Received:
    10
    Hey, there I agree with this recommendation fully.

    Raid 10 and p3700 for VMs.

    As the option, I see the configuration with ZFS on FreeBSD quite good for this case.

    This is definitely the right way to go here.
     
    #4
  5. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    For ESXi and a mixed workload I would use multi-mirrors. For pure video, a raid-z2 would be ok unless you have many users.
    I would use an Intel Optane 900P with a 20GB vdisk for Slog (for VMs) and the rest for the storage VM and L2Arc with read ahead enabled.

    The fastest ZFS is a genuine ZFS and Oracle Solaris. For Open-ZFS FreeBSD and Illumos (free Solaris fork with distributions like OmniOS or OpenIndiana) are the best options. I prefer Solarish based systems with best ZFS integration and NFS and SMB as kernelbased ZFS properties and Comstar, an enterprise FC/iSCSI stack

    more
    Configuring Storage Devices With COMSTAR - Oracle Solaris Administration: Devices and File Systems
    http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf
     
    #5
  6. vrod

    vrod Active Member

    Joined:
    Jan 18, 2015
    Messages:
    205
    Likes Received:
    29
    I bought a 280gb optane 900p after hearing how well it performs as a slog and because it also has the 10DWPD.

    That means I will probably be getting rid of my P3700, in case the HDD array performs to my liking. :) thanks for the suggestion!

    I have about 15TB of media content so I gotta figure out what to do. I’d honestly really like to have everything in one raid10 pool for the spindle performance. With 18x3 tb I would still get about 50TB of usable space or so.
     
    #6
  7. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    I would probably prefer a large pool from disks and a second fast ssd only pool for performance sensitive data. But with an Slog like the Optane 900p and a multiple raid-10 setup performance should not be a concern even when secure sync write is enabled (see my Optane benchmarks with a 4 x HGST HE 8 Pool and the Optane 900P as slog)
     
    #7
Similar Threads: storage mixed
Forum Title Date
NAS Systems and Networked Home and SMB Software NAS and Storage Options Apr 12, 2018
NAS Systems and Networked Home and SMB Software Oracle ZFS Storage Appliance available - What to test? Apr 3, 2018
NAS Systems and Networked Home and SMB Software NVMe Storage Solution Apr 1, 2018
NAS Systems and Networked Home and SMB Software All flash storage server Aug 25, 2017
NAS Systems and Networked Home and SMB Software iXsystems FreeNAS Certified Flash storage array Feb 24, 2017

Share This Page