Single NVME node Ceph Benchmarking OSD Tips

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

OBasel

Active Member
Dec 28, 2010
494
62
28
I'm trying to benchmark a single Ceph node and have 8 NVMe drives setup as OSD's but my speeds are 700MB / sec. Any tips on why I'm missing the speed of even one drive?
 

vrod

Active Member
Jan 18, 2015
241
43
28
31
First of all I believe the point of ceph is to span a storage across several nodes and not just one. As far as i know though, ceph is really not that good in performance. Unless you are planning to expand to other nodes with ceph, go zfs or something similar.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
@OBasel sorry I missed this. The trick in these types of systems is multiple OSD per NVMe device. That is an extremely common practice for NVMe OSDs.
 

Patriot

Moderator
Apr 18, 2011
1,450
789
113
I'm trying to benchmark a single Ceph node and have 8 NVMe drives setup as OSD's but my speeds are 700MB / sec. Any tips on why I'm missing the speed of even one drive?
Ceph is not a performant solution and you have found the per OSD bottleneck.

@OBasel sorry I missed this. The trick in these types of systems is multiple OSD per NVMe device. That is an extremely common practice for NVMe OSDs.
Just to clarify, this is a test lab solution to get performance, not a production solution.