ZFS vCluster in a Box

Discussion in 'Solaris, Nexenta, OpenIndiana, and napp-it' started by gea, Nov 9, 2018.

  1. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    Nearly eight years ago, I introduced All-In-One virtualized storage, first at STH
    Time for a new idea:

    napp-it Z-Raid HA vCluster
    easy and cheap enough

    I am on the way to finish my Z-Raid Cluster in a Box solution.
    In the past a Cluster in a Box consists of two servers with a common pool of mpio SAS disks. One of the servers builds a pool from the disks and offers services like NFS or SMB. On a failure or for maintenance you can switch over to the second server within a few seconds. Management is done for example with RSF-1 from high-availability. SuperMicro offers special cases that can hold two mainboards

    Such solutions are expensive and management is quite complex but offers high performance and high availability. To reduce costs, my solution is based on ESXi (any license) to virtualize the two nodes and a control instance. It uses the shared controller/ shared raw disk options of ESXi so my solution does not need multipath SAS but can work with any disks.

    Setup see http://www.napp-it.org/doc/downloads/z-raid.pdf

    If you want to try the preview state that allows a manual failover within 20s, you can update napp-it to the free 18.02 preview (or 18.12dev).

    [​IMG]
     
    #1
    Last edited: Nov 9, 2018
    Marsh, SlickNetAaron, Rand__ and 2 others like this.
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,878
    Likes Received:
    833
    I think you should try selling these to pay for napp-it development.

    There are some Supermicro people on here that I have seen and even @Patrick knows people there. Maybe they could get a part made for you with this software installed?
     
    #2
  3. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    Very nice, looking forward to see more info on the „Twin vCluster“ :)
     
    #3
  4. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    A „Twin vCluster“ is already inside (a howto and some tests are missing) .
    You can set a second cluster control node in settings to manage a second vCluster. Each vCluster has a head and a control VM. Storage in this case is provided by iSCSI, either with dedicated iSCSI servers or a virtual SAN VM on each vCluster for network mirroring between them. This would not only allow a failover between heads but a full vCluster/storage failure.

    @MiniKnight
    No special hardware needed for a vCluster in a Box. A basic ESXi server with an LSI HBA (the typical AiO setup) is enough to setup a vCluster with shared raw disk mapping of SAS or Sata disks. Maybe you want a little more RAM.
     
    #4
  5. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    Yes I read the doc was referring to the howto;)
     
    #5
  6. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    [​IMG]
     
    #6
    Rand__ likes this.
  7. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    So now the only question remains how to get more than 10G sync speed out of my hw, but thats probably for another thread;)
     
    #7
    T_Minus likes this.
  8. tjk

    tjk Active Member

    Joined:
    Mar 3, 2013
    Messages:
    233
    Likes Received:
    32
    Would be great to see this work on bare metal servers! Unless I'm not understanding, your HA functionality requires VMware VM's today, right?
     
    #8
  9. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    Each head needs shared disk/Lun access. The cheapest and easiest way is ESXi. But as the Cluster Control software does not care about, you can basically use barebone server with mpio SAS or FC/iSCSI as well. Shared disk access can and must be configured outside my Cluster management.

    At the moment I do all my tests with an ESXi vCluster as I see this as the typical use case (simple, fast and cheap HA)
     
    #9
  10. azev

    azev Active Member

    Joined:
    Jan 18, 2013
    Messages:
    596
    Likes Received:
    147
    Is it even possible to get 10G sync speed on non specialized hardware ?? I am getting no where near that with optane zil and bunch of enterprise class ssd.
     
    #10
  11. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    I dont know, hence the question. But after @gea has found a solution for the HA problem this will be the next major hurdle I think (at least for my search for the ultimate high speed shared VM storage ;))
     
    #11
  12. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,594
    Likes Received:
    379
    The optane (900/905p) can do ~1.2 gbyte/s @ 1t/1qd with large io (1mb+) and sequential writes.

    The question is what workload; how many io requests, which io sizes, sequential or random writes? :D
     
    #12
  13. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    #13
    T_Minus likes this.
  14. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    If I would have a wish free from SuperMicro
    I would ask for something like Zstor GS-Cube8 Mini Cube Storage Server | Zstor Storage | Produkte-Zstor GmbH - Open Storage

    but
    - Flex ATX (not ITX) with the option to use uATX maybe with only two Slots out of three to outside
    - Case with integrated PSU with one or two silent 120 mm fans for the whole case incl PSU.

    Optionally with 3,5". 2,5" or U.2 , full or shared
     
    #14
    Last edited: Nov 10, 2018
    T_Minus and realtomatoes like this.
  15. minhneo

    minhneo New Member

    Joined:
    Oct 3, 2018
    Messages:
    7
    Likes Received:
    0
    Anyone have idea about Super Storage Bridge Bay (SBB) ?
     
    #15
  16. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    These are the classic hardware solutions for "Cluster in a Box". SM combines two mainboards with shared disk access ex over redundant multipath SAS for both nodes in a single case.

    Napp-it would be basically able to use such a hardare setup to manage the node and service failover with an additional control node. My vCluster concept follows the same idea but intends to reduce costs and complexity but maintain most of its performance and options.

    My vCluster use virtualized nodes instead of two mainboards and the shared disk options of esxi. This allows the same but supports SAS, Sata and NVMe out of the box on a single server (the usual AiO server).
     
    #16
    Last edited: Nov 15, 2018
  17. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    vCluster Beta2 is available (napp-it 18.12g dev)

    Current state:
    Manual Failover between nodes working for NFS and SMB (failover in around 20s)
    For SMB it supports failover for local user and AD users connected to the share during failover.

    Todo
    auto-failover (on tests)

    Expect vCluster to next Pro (Jan 2019)
     
    #17
    Last edited: Nov 17, 2018
    Rand__ likes this.
  18. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
  19. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,273
    Likes Received:
    447
    If only I could get my network going properly ;)
     
    #19
  20. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    #20

Share This Page