CEPH: switching from HDD to SSD - HW recommendations

Discussion in 'Linux Admins, Storage and Virtualization' started by charlie, May 23, 2019.

  1. charlie

    charlie Member

    Joined:
    Jan 27, 2016
    Messages:
    58
    Likes Received:
    3
    Hi guys,

    I would like to replace our current HP G6 (64G RAM, 2x L5640 CPU, TGE nic, PCI NVMe) and HDD (1 TB WD Black) CEPH cluster with a used newer system with SSD.

    We have 70 OSD, (10 per node), avg IOPS around 2k, peak ~5k, the cluster used for KVM vms. It working very well, but we would like to switch to SSD because of better latency, lower power usage, and save rack unit space (current cluster hace 3x 1U mon servers and 7x 2U OSD, 17 rack unit total).

    My budget is very limited, i can spend on this project around 25k EUR.

    What i think first (using HP servers, because we shopping from servershop24.de and they mostly selling HP stuffs):

    HP ProLiant DL360p Gen8
    • 10 SFF bay version
    • RAM: 12x 8GB DDR3 REG ECC (beacuse of triple channel 6 modul per cpu)
    • CPU: E5-2670 (8-Core @ 2.60 GHz)
    • FlexibleLOM: Dual SFP+ (649869-001 or similar)
    • NIC: HP NC560SFP+ dual SFP+
    • And for OSD disk: 8x SAMSUNG PM863a (new!)
    • P420i in HBA mode or dedicated HBA, like H220 or similar
    Two NIC because for separate client and cluster network, and both connecting to two switch in active-backup solution.

    I planning to use 7 OSD node and 3 mon, this will be around 50TB gross capacity.

    What do you think about this? Is it can be good, or im have totally wrong vision?
     
    #1
    Last edited: May 23, 2019
  2. Nick85

    Nick85 New Member

    Joined:
    Jul 13, 2018
    Messages:
    6
    Likes Received:
    1
    We've built a ceph cluster with Dell R720XD's (25x2.5, 2680v2 cpus), 40gbit networking, Intel S3710 SSD's. Performance is as you'd expect, astonishing.
     
    #2
  3. charlie

    charlie Member

    Joined:
    Jan 27, 2016
    Messages:
    58
    Likes Received:
    3
    Hm, maybe i choosed wron SSD, because 863a no longer available, PM883 is the successor.
     
    #3
  4. Nick85

    Nick85 New Member

    Joined:
    Jul 13, 2018
    Messages:
    6
    Likes Received:
    1
    Any enterprise/datacentre grade, high endurance SSD's will be fine, which is why we went with the S3710's. PM883 is lower endurance, go for the SM883 if you want Samsung drives.
     
    #4
  5. charlie

    charlie Member

    Joined:
    Jan 27, 2016
    Messages:
    58
    Likes Received:
    3
    Hi,

    As i wrote my budget is limited. Offcourse, S3710 or similar intel will be the best. But it price is very high for my budget (~550 EUR), and my goal is to simply replace spinning disk. I don't need (now) high io, current usage not required it.

    I checked SM883, its the write optimized version from samsung. But as i see, the only - main - different beetwen PM883 and SM883 is the DWPD. Otherwise PM price is half of the SM version.

    (i checked our current cluster graphs, and avg amount of wrote data is 6 TB in the last three month or 25TB in the last 12 month per OSD)
     
    #5
Similar Threads: CEPH switching
Forum Title Date
Linux Admins, Storage and Virtualization New ceph cluster -recommendations? Sep 5, 2019
Linux Admins, Storage and Virtualization Different disk sets for Proxmox Ceph pools? May 29, 2019
Linux Admins, Storage and Virtualization Ceph Benchmark request /NVME Feb 22, 2019
Linux Admins, Storage and Virtualization ceph backfill problem Sep 20, 2018
Linux Admins, Storage and Virtualization Ceph low performance Sep 17, 2018

Share This Page