Software Defined Storage

Discussion in 'Software Stuff' started by macrules34, Jul 30, 2018.

  1. macrules34

    macrules34 Member

    Joined:
    Mar 18, 2016
    Messages:
    124
    Likes Received:
    2
    I have 4 dell servers each with 2x 1TB hard drives. I would like to use the extra space from the hard drives to create a storage array like you would with ScaleiO.

    I was wondering if anyone has come across a free Software Defined Storage solution like ScaleiO? I inquired with Dell EMC about ScaleIO software licenses but was told that they no longer sell just the software, its know bundled with their hardware.
     
    #1
  2. sybreeder

    sybreeder New Member

    Joined:
    Oct 8, 2013
    Messages:
    9
    Likes Received:
    1
    #2
    NISMO1968 likes this.
  3. NISMO1968

    NISMO1968 [ ... ]

    Joined:
    Oct 19, 2013
    Messages:
    67
    Likes Received:
    10
    You can get the licenses under the table still, but ScaleIO has mediocre performance on few nodes, so anything below 8 simply doesn't make sense. Did you try Ceph?

     
    #3
  4. fossxplorer

    fossxplorer Active Member

    Joined:
    Mar 17, 2016
    Messages:
    316
    Likes Received:
    25
    Was about to mention Ceph, but @NISMO1968 already hinted to it. In the world of open source storage solutions, Ceph and Gluster are very hot buzzwords.
    Also, another hint would be Home - OpenSDS and opensds/opensds that i recently came over. Looks quite interesting with backing from major entities!
     
    #4
    NISMO1968 likes this.
  5. LaMerk

    LaMerk New Member

    Joined:
    Jun 13, 2017
    Messages:
    14
    Likes Received:
    2
    #5
    NISMO1968 likes this.
  6. macrules34

    macrules34 Member

    Joined:
    Mar 18, 2016
    Messages:
    124
    Likes Received:
    2
    I was planing on having 4 servers as the storage nodes.
     
    #6
  7. NISMO1968

    NISMO1968 [ ... ]

    Joined:
    Oct 19, 2013
    Messages:
    67
    Likes Received:
    10
    You'll be fine with Ceph. Alternatively go GlusterFS (if you don't plan to scale out which is a bit complicated with G/FS). ScaleIO is history...

    P.S. OpenIO looks promising! :)

     
    #7
  8. macrules34

    macrules34 Member

    Joined:
    Mar 18, 2016
    Messages:
    124
    Likes Received:
    2
    I have looked through the Ceph documentation and was wondering if you could use a portion of a disk as storage space? One partition with the OS and the remainder of the disk used as storage. With Scale IO I know you are able to do that.
     
    #8
  9. macrules34

    macrules34 Member

    Joined:
    Mar 18, 2016
    Messages:
    124
    Likes Received:
    2
    I was reading the Ceph manual for hardware recommendations and noticed that it said you need 2 Ethernet ports for each of the following:
    1. OSD
    2. MON
    3. MDS

    So does that mean that I would need six Ethernet ports per node? Each node will be providing storage. I am using some 1U servers and only have one expansion slot, the servers also have 2 onboard Gigabit NICs, plus an IPMI port.

    Will gigabit Ethernet suffice or should I be using 10GB?

    FYI, client server only has 1GB Ethernet and not was to add 10GB.
     
    #9
  10. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,655
    Likes Received:
    1,040
    I wouldn't bother with Ceph in such a small cluster. Almost guaranteed you won't be satisfied with the performance or the resiliency (ability to withstand faults) at such a small scale. Same issue for ScaleIO or really any other fully distributed SDS.

    At the small scale of your lab Gluster is likely ideal.
     
    #10
  11. hhp

    hhp New Member

    Joined:
    Aug 3, 2016
    Messages:
    13
    Likes Received:
    1
    Just as far as OSS SDS software solutions, I would go with ceph if you have IOP performance needs or gluster for media, file sharing, SMB etc. That said you will need a minimum 2 - 10 gbps network ports per server and really should use a switch with enough ports at minimum. To get performance out of these platforms you need a fast control network, decent/nearly current H/W and as many OSD (data) nodes, mon etc. as perf targets dictate.

    If you are planning to reuse 3-4 old servers from the westmere or older era on a 1 gbps network you will be very disappointed (_very_), especially if they are already running something( and will need to continue in this role). The storage nodes should be 100% dedicated to the role and nothing else, they should be reasonably fast with many cores (min 1 osd process per disk, 4 for ssd, nvme - plan for 1 core per disk) and one or more fast flash device for journal, WAL and only enough osd processes/disks assigned that the flash device can service.

    If you use consumer SSD's, old servers with too many disk for the available CPU/RAM to handle, especially running on a 1 gb network your cluster will most likely be slower than a single disk during the time it actually runs before crashing. The thing to remember about Ceph and Gluster is they were designed to run on commodity servers etc. yes - but on a scale most never see, multiple racks of servers per pool etc. To get performance at lower node counts you need very fast, expensive H/W. If you take old servers already running something else and then try to layer Ceph on top of it, that will be a disaster.

    I just went though this learning experience and thanks to experienced Ceph users feedback and suggestions, saved a fair sum of money I was planning to spend on used westmere era servers with lots of disk and slower procs. From those discussions and my research I decided to go with 1U (2x - E5-26xx v2 10C/20T) servers with a maximum of 8 storage devices (10k SAS 1.2 TB) using smaller NVMe addin's for write ahead log etc and a minium of 64GB RAM per SD node. Total of 6 OSD nodes with mon and admin etc. all running on their own lower spec 1U servers of the same class. This is just for a test environment to verify a Ceph implementation will work for our intended purpose, if we go this route the initial prod deployment will most likely be 2-3 racks of EPYC servers using a large amount of NVMe U.2 SSDs and SAS SSD's in a tiered config fronting a few hundred 10k drives.

    My network is 40gbps top of rack so I was already good there, using 1 gbps network is a non-starter for an SDS solution, you would just be wasting your time other than learning a few new things. Also keep in mind, just getting a few servers running in a Ceph cluster is just a start, you will need to master the crushmap and other platform components to really optimize and get the performance out of it. If you do manage to get something running on servers already performing a task , using un-allocated disk space, I doubt the cluster would be able to heal/recover or it would take a few lifetimes of waiting.

    One more thing to keep in mind when sizing, you need to target your specs on your performance minimums during a node failure/recovery not just a clean good state, with a minimum of nodes (3-4) you will lose a large percentage of your capacity, recovery & re-balancing is very resource intensive.

    I am back in the research stage for my test environment double checking everything before making the spend so take what I say with a grain of salt, I realistically don't think this will work for you or deliver what you want out of it.
     
    #11
    jak likes this.
  12. macrules34

    macrules34 Member

    Joined:
    Mar 18, 2016
    Messages:
    124
    Likes Received:
    2
    I am using 4 dedicated servers that are Dell CS24-SC with 16GB RAM each server has two 1TB hard drives and I will run the OS off a USB flash drive. There are two free slots for drive expansion. The server has one expansion slot.

    The client machine has 1GB Ethernet and not possible to upgrade to 10GB, so the client would have to connect to the storage at 1GB, there's no other option that I'm aware of. I could use 10GB for back end duties.
     
    #12
Similar Threads: Software Defined
Forum Title Date
Software Stuff Rendering software supporting windows processor groups >64 and win 10 for workstation Nov 17, 2017
Software Stuff RPD software assistance Jun 15, 2017
Software Stuff Tape Backup Software Apr 21, 2017
Software Stuff Advice needed for Remote Control software Jan 6, 2017
Software Stuff any NFR Software Tipps ? Nov 3, 2016

Share This Page