Analysis of storage performance vs CPU speed/load

[Rand__]

Hi,

does anyone know of an analysis of storage performance vs CPU speed/load?

I see many reports of 'slow' storage speed (usually zfs or network related but sometimes also local storage or raid) and also many many people asking "what CPU do I need for this or that on my storage box" so I wondered whether anyone knows of an analysis in that regard?

O/C @Patrick with his access to dozens of CPUs and probably a ton of different storage (sub)systems would be a prime candidate to create such a piece if it did not exist, but maybe it already does and I just have not been able to find it

Thanks

 

[i386]

Interesting topic.

I think for simple "storage" you won't need much: broadcom/lsi is using mips cpu designs from 1997/98 (with clock increased from 400MHz to ~1.2GHz) for their hbas/raid controllers adn these can do 6+ gbyte/s or 500k+ @ 4k.

Add encryption, checksums or other fancy features and the performance numbers will look totally different ._.

 

[Rand__]

Yes I agree, most Raid based storage will have most of the load offloaded to the controller. There likely other aspects limit (non-local) performance.

 

[Patrick]

Hi,

does anyone know of an analysis of storage performance vs CPU speed/load?

I see many reports of 'slow' storage speed (usually zfs or network related but sometimes also local storage or raid) and also many many people asking "what CPU do I need for this or that on my storage box" so I wondered whether anyone knows of an analysis in that regard?

O/C @Patrick with his access to dozens of CPUs and probably a ton of different storage (sub)systems would be a prime candidate to create such a piece if it did not exist, but maybe it already does and I just have not been able to find it

Thanks

I could probably help out if someone wanted to investigate and do a piece.

Here is the flip side that the industry does not want to talk about too much: basic storage like XOR RAID calculations do not take that much CPU.

Where you do see heavy CPU usage is in things like deduplication, compression at very high speeds, rebuilding scale-out storage (have a vSAN or Ceph cluster node die and you will see this), and 100GbE encryption. With all of the crypto accelerators out there, even adding crypto is not too bad. Take a 12-15W Microsemi Adaptec controller that can do encryption at essentially the same rate as without encryption https://www.servethehome.com/new-mi...ec-smartraid-3162-with-maxcrypto-performance/

To do this, which I agree would be good, you need a few things:

• What kind of storage? HDFS, mdadm, ZFS, Gluster, Lustre, Ceph, vSAN, Microsoft Storage Spaces, other? With distributed storage, we need sets of CPUs.
• What storage features?
• What kind of network speed?
• Which network adapter? (some adapters have heavy offloads)
• What scenarios? Rebuilds, boot storm, analytics?
• Is there anything else you need to run on the nodes?

You quickly get to a domain-specific storage solution.

Again, if someone wants to do this, I am happy to discuss how we can get access to hardware.