Alternatives to vmWare vSan for hyperconverged environment (Home)

Discussion in 'Software Stuff' started by Rand__, May 13, 2017.

  1. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Hi,
    I am currently running a vmWare environment at home using vSan for shared storage and Horizon for VDI. I also have a bunch of VMs running (<20).

    This is licensed via VMUG so 200 bucks a year, my vsan is based on All-Flash (P3700 & P320h as cache, 750's as storage), currently 3, soon to be 4 nodes.

    Now I know that hyperconverged is primarily targeting enterprise users where its more important to have a consistent and equal distribution of performance to a larger user base opposed to peak performance of a few users.


    I am aware that given the low node &disk amount (1 cache+1 capacity/node) the most I can get is 1 disk's worth of performance, but at least for write I am not even getting that.


    I have run a few tests (I know CDM is not really the best tool for this, its just for illustrating the difference between a local ssd and the vsan).

    Its the same VM moved to different hosts (with different cpus and cache drives)

    The local drive on a 26667 (2 cores):
    38_local_s3510.PNG

    Vsan based vm on the same box, with a P320h:
    38_vsan_p320.PNG

    Vsan based vm on a 2630L with a P3700
    37_vsan_p3700.PNG

    And lastly, to test whether this really might be due to the multi user design, 8 concurrent CDM runs on the 2667/P320h (with 8 cores). Note that CDM did not run all in parallel, it grouped as marked with a delay of maybe 15 secs total. That means the last CDM run finished 15 secs after the first one, so some tests were not running in parallel, at some point there was mixed read/write going on.
    Yes I know, "run a proper test using fio" or such :p
    38_vsan_concurrent.PNG



    I can live with the current setup, but I wonder whether one of the alternative options would provide more individual performance than vmWare/vsan. O/C its totally possible that my (unsupported HW) setup is just not well built and this is just a config issue.
     
    #1
  2. cheezehead

    cheezehead Active Member

    Joined:
    Sep 23, 2012
    Messages:
    635
    Likes Received:
    140
    VMware/VSAN is one model

    FOSS would be a Proxmox VE/Ceph build

    Windows-centric would be Hyper-V/Storage Spaces

    FreeNAS might be there at some point by Coral got axed and the current Bhyve implementation is more a POC currently IMO.

    Also with VMware, there are other storage platforms available...if it's Proliant gear then you should have the free licensing for their StoreVirtual (aka Lefthand rebanded product).
     
    #2
    Evan likes this.
  3. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Yes - is any of these alternatives configurable for "few users / high performance" opposed to enterprise "many users/throw more hardware at me" ? :)
     
    #3
    Last edited: May 13, 2017
    dicecca112 likes this.
  4. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    1,994
    Likes Received:
    284
    Other vSAN's or similar commercial products such as scaleio, starwind, nutanix, simplivity, storpool, hedvig, whatever IBM call gpfs these days.

    I am disappointed that FOSS options (ceph) are so limited at the small end, at the high end number of nodes it's more suited.

    Of course there is many "failover" options available as well which for home use situations maybe easier to manage as well.
     
    #4
    NISMO1968 likes this.
  5. dicecca112

    dicecca112 Active Member

    Joined:
    Feb 10, 2016
    Messages:
    300
    Likes Received:
    35
    Bingo, been hunting that too. I'll be watching this thread carefully.
     
    #5
  6. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,634
    Likes Received:
    722
    Scale-io, nutanix, proxmox
     
    #6
    T_Minus likes this.
  7. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Yes, I am aware there a lot of similar solutions :)
    But which one of these caters for the requirement of "great performance for the few" ?
    I can't believe that all except vmware/vsan are top performers for small work-groups :D
     
    #7
  8. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    1,994
    Likes Received:
    284
    I would say none of them.
    Some of them can perform better if you put lots of disks on a small number of nodes but generally nothing works that well at 2 or 3 nodes from what I see, of course that's not to say it's terrible but simple fact is 2 or 3 nodes will never perform better than 1
     
    #8
  9. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    And thats perfectly fine - what I am looking for is a solution that offers vSan like capabilities (real time synchronisation of vm data, HA/DRS, dedup) with at least 70% of the performance of a single node with a 3 or 4 node setup.

    I totally understand that with the low number of nodes only resilience will be improved and not speed because data cannot be striped and mirrored at the same time with these few nodes/disk groups.

    But I do expect to get the better part of that single node's performance - ie I'd hope to get (70%) Cache disk write speed and Capacity disk read speed

    This is a 750 CDM from another box (on a 400GB model) - thats significantly better than I get from the vSan datastore (especially 4K)
    750_2_CDM.JPG

    So either my vsan config is crap and I should get better speeds with what I have
    or vSan is not cut for "great performance for the few" and limits the resources to a single client to be able to serve many

    Edit:
    Hm it might be my box after all - its difficult to find some small setup comparison values but this guy here gets ~ similar results with much cheaper hardware (1 ssd, 2 spinners)

    Edit2:

    Ran a Storage test and that looks fine... cant copy/export it for whatever reason, so partial screenshot only:
    vSan7030.PNG

    30 vmdks total (10 per host), so conservatively its 30x50 MB/s which is 1500MB which is totally ok
     
    #9
    Last edited: May 14, 2017
  10. NISMO1968

    NISMO1968 [ ... ]

    Joined:
    Oct 19, 2013
    Messages:
    61
    Likes Received:
    10
    ScaleIO is extremely slow unless you use 10+ nodes, and it isn't free for production use.

    https://www.emc.com/content/terms/eula-scaleio.htm

    "E. “Internal Business Purposes” means an internal (non-commercial) Use for the purpose(s) of testing and demonstrating the features of the Software, and not for Customer product development, product testing, or other Customer research and development or commercial purposes."

    I know it's a home lab but why would you want to learn anything you can't use @ long run?

     
    #10
  11. NISMO1968

    NISMO1968 [ ... ]

    Joined:
    Oct 19, 2013
    Messages:
    61
    Likes Received:
    10
    Did you get a chance to get hands dirty with SimpliVity VSA? I only heard it exists, but it was never ever made public AFAIK.

    P.S. Nutanix software-only comes with their own KVM fork-out, so it isn't a SDS either. FYI.

     
    #11
  12. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    1,994
    Likes Received:
    284
    Only seen simplivity with demo done by their staff, could tell it did not really bring anything for the company I work for as it would mean more products and processes rather than every site and install being same.
     
    #12
  13. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    So from the other thread it shows Nutanix won't cut it (for my specific use case).
    So how about Promoxx? How is the performance with few fast hosts?
     
    #13
  14. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,255
    Likes Received:
    1,254
    You can't do HA on ProxMox without shared storage :(

    If it let you do it with localized it would be very fast even with ZFS since it's baked in, and no network sharing.

    I hope they offer HA with local storage soon, I would configure for that! For now I'm just using ZFS+Proxmox and doing backups.
     
    #14
  15. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Ah thats why its always ProxMox+Ceph
    So how is Ceph then in terms of performance?
     
    #15
  16. NISMO1968

    NISMO1968 [ ... ]

    Joined:
    Oct 19, 2013
    Messages:
    61
    Likes Received:
    10
    You need quite many nodes to assemble a reasonable I/O bandwidth. Replication is OK-ish but you need erasure coding to gain proper fault tolerance and space savings and E/C is... slow :( I'd say Ceph is OK for a big data dump (CERN and NSA use it for exactly this purpose) but for VM storage it misses quite a lot. IMHO.

     
    #16
  17. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    1,994
    Likes Received:
    284
    Hyper converged sounds like the perfect solution for small installs but in reality it seems to fall short unless you actually have bigger installs of at least 4+ nodes which for most people means license costs that don't make all that much sense.

    2 nodes solutions (or 3 if the 3rd node can run on a really skinny config without storage) that actually work seem to be few any far between.
     
    #17
  18. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Ok, so looks like I better try to coax some more performance out of my vSan setup then ...
    Thanks
     
    #18
  19. NashBrydges

    NashBrydges Member

    Joined:
    Apr 30, 2015
    Messages:
    62
    Likes Received:
    14
    Starwind has recently removed all restrictions on their free version so this might be something worth investigating. You can now scale up or scale out without cost.
     
    #19
  20. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,235
    Likes Received:
    264
    Yes that might be an option, but some kind of gui would be nice to be honest;)
     
    #20
    ecosse likes this.
Similar Threads: Alternatives vmWare
Forum Title Date
Software Stuff Alternatives to DBAN for SSD Secure Erase? May 20, 2016
Software Stuff Reasonably alternatives to Exchange/Server2012/Sharepoint? May 12, 2016

Share This Page