CEPH: switching from HDD to SSD - HW recommendations

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

charlie

Member
Jan 27, 2016
58
3
8
Budapest, HU
Hi guys,

I would like to replace our current HP G6 (64G RAM, 2x L5640 CPU, TGE nic, PCI NVMe) and HDD (1 TB WD Black) CEPH cluster with a used newer system with SSD.

We have 70 OSD, (10 per node), avg IOPS around 2k, peak ~5k, the cluster used for KVM vms. It working very well, but we would like to switch to SSD because of better latency, lower power usage, and save rack unit space (current cluster hace 3x 1U mon servers and 7x 2U OSD, 17 rack unit total).

My budget is very limited, i can spend on this project around 25k EUR.

What i think first (using HP servers, because we shopping from servershop24.de and they mostly selling HP stuffs):

HP ProLiant DL360p Gen8
  • 10 SFF bay version
  • RAM: 12x 8GB DDR3 REG ECC (beacuse of triple channel 6 modul per cpu)
  • CPU: E5-2670 (8-Core @ 2.60 GHz)
  • FlexibleLOM: Dual SFP+ (649869-001 or similar)
  • NIC: HP NC560SFP+ dual SFP+
  • And for OSD disk: 8x SAMSUNG PM863a (new!)
  • P420i in HBA mode or dedicated HBA, like H220 or similar
Two NIC because for separate client and cluster network, and both connecting to two switch in active-backup solution.

I planning to use 7 OSD node and 3 mon, this will be around 50TB gross capacity.

What do you think about this? Is it can be good, or im have totally wrong vision?
 
Last edited:

Nick85

New Member
Jul 13, 2018
6
1
3
Bournemouth
We've built a ceph cluster with Dell R720XD's (25x2.5, 2680v2 cpus), 40gbit networking, Intel S3710 SSD's. Performance is as you'd expect, astonishing.
 

Nick85

New Member
Jul 13, 2018
6
1
3
Bournemouth
Any enterprise/datacentre grade, high endurance SSD's will be fine, which is why we went with the S3710's. PM883 is lower endurance, go for the SM883 if you want Samsung drives.
 

charlie

Member
Jan 27, 2016
58
3
8
Budapest, HU
Any enterprise/datacentre grade, high endurance SSD's will be fine, which is why we went with the S3710's. PM883 is lower endurance, go for the SM883 if you want Samsung drives.
Hi,

As i wrote my budget is limited. Offcourse, S3710 or similar intel will be the best. But it price is very high for my budget (~550 EUR), and my goal is to simply replace spinning disk. I don't need (now) high io, current usage not required it.

I checked SM883, its the write optimized version from samsung. But as i see, the only - main - different beetwen PM883 and SM883 is the DWPD. Otherwise PM price is half of the SM version.

(i checked our current cluster graphs, and avg amount of wrote data is 6 TB in the last three month or 25TB in the last 12 month per OSD)