CPU considerations for 10Gb networking

[IamSpartacus]

Are there any "general" guidelines out there for minimum CPU requirements needed for 10Gb networking with regard to storage servers? The scope I'm thinking of would be for a simple bulk storage storage server serving Samba/NFS shares and nothing else. I'm talking very few simultaneous (2-3 tops) transfers.

 

[Blinky 42]

10Gb is pretty easy to saturate for file transfer protocols like Samba/CIFS/NFS/HTTP when you are able to use Jumbo frames.
I suspect the metric is more do you have enough disk IO & memory to be able to push 10Gb constantly to/from the filesystem (enough storage headroom for the filesystem operations on top of the 10Gb of net traffic).

 

[T_Minus]

I think it's going to be more of what the filesystem needs... ie: parity, dedup, compression, etc if it's on CPU or HW RAID, etc... more than just saturating the network but at that network speed how the CPU is affected by the storage.

 

[IamSpartacus]

10Gb is pretty easy to saturate for file transfer protocols like Samba/CIFS/NFS/HTTP when you are able to use Jumbo frames.
I suspect the metric is more do you have enough disk IO & memory to be able to push 10Gb constantly to/from the filesystem (enough storage headroom for the filesystem operations on top of the 10Gb of net traffic).

I think it's going to be more of what the filesystem needs... ie: parity, dedup, compression, etc if it's on CPU or HW RAID, etc... more than just saturating the network but at that network speed how the CPU is affected by the storage.

The use case I'm thinking of bulk storage so non-striped parity by the OS. Something like UnRAID or SnapRAID+MergerFS under Linux.

 

[ttabbal]

Without stripes I doubt spinners will be able to saturate 10Gbe anyway... Maybe if every client hits a different physical drive and there are enough drives and clients. That's over 1GByte/Sec... There are few enough SSDs that can sustain that in single drive configurations. Some nvme can do it.

With a remotely modern card, jumbo frames are not needed and can cause all sorts of problems with other gear that can't deal with large MTUs. I don't bother. I tried it with a point-to-point network, it works, but CPU use didn't change more than a fraction of a percent, throughput didn't change much more than that.

CPU for the network isn't a major issue with most gear, at least it doesn't seem to be for me. My old 1366 based server handles it without noticeable load. ZFS uses more CPU than the network does. My cards are old connectX2s, nothing fancy.

 

[IamSpartacus]

Without stripes I doubt spinners will be able to saturate 10Gbe anyway... Maybe if every client hits a different physical drive and there are enough drives and clients. That's over 1GByte/Sec... There are few enough SSDs that can sustain that in single drive configurations. Some nvme can do it.

With a remotely modern card, jumbo frames are not needed and can cause all sorts of problems with other gear that can't deal with large MTUs. I don't bother. I tried it with a point-to-point network, it works, but CPU use didn't change more than a fraction of a percent, throughput didn't change much more than that.

CPU for the network isn't a major issue with most gear, at least it doesn't seem to be for me. My old 1366 based server handles it without noticeable load. ZFS uses more CPU than the network does. My cards are old connectX2s, nothing fancy.

I'll be using a cache pool of SSDs that can sustain 10Gb. But from what you are saying the CPU shouldn't really be much of a consideration.

 

[MiniKnight]

Use a good NIC. It isn't a major issue to sustain the transfers but it does consume a few percent. Good NICs = lower utilization.

There's also a wide delta between small transfers and large transfers.

High bandwidth doesn't mean low latency as a rule. Good NICs and high speed cores help with latency.

 

[fractal]

The use case I'm thinking of bulk storage so non-striped parity by the OS.

Anything made this century should handle that just fine. You won't be filling 1G and the extra burst load of 10G won't have much of an affect on the system. Other issues like power consumption, sufficient ram and enough PCIe channels are more important than wondering whether your CPU will be bothered by the presence of a 10G NIC.

 

[IamSpartacus]

Anything made this century should handle that just fine. You won't be filling 1G and the extra burst load of 10G won't have much of an affect on the system. Other issues like power consumption, sufficient ram and enough PCIe channels are more important than wondering whether your CPU will be bothered by the presence of a 10G NIC.

I'm using SSD's in a RAID0 cache pool so I am able to saturate 10Gb.