Hi everyone,
A few months ago in a discussion thread either here or on one of the many forums I frequent, someone linked me to what I recall looked like a blog post.
I now cant find that blog post, nor the message in which someone linked me to it, so I come to you for assistance.
The topic was NAS performance, and why nothing above 10gig (25gbit, 40gbit, 100gbit) performs so slowly out of the box.
In other words, if you install Gigabit, 2.5gbit, 5 gbit or 10gbit networking on your machine, out of the box you are going to be able to pretty much get full performance out of it less a small amount of overhead.
If you install 25gbit, 40gbit, 100gbit, etc. networking, unless you hit it with 20-100 simultaneous iperf threads, you never wind up actually seeing the full bandwidth.
On my direct connected 40gbit NIC between my workstation and NAS, I rarely see more than 1.6GB/s to 1.8GB/s data transfers, despite the storage on the server performing MUCH above that level.
What is the bottleneck has never been evident. There is no great CPU load, the PCIe lanes provide sufficient speeds, etc. etc.
Someone in a conversation on this topic suggested it had to do with hand tuning your implementation based on its latency, and this could drastically improve single link performance.
I remember reading the linked blog post. It was on a Linux or Unix host. It had something to do with measuring round trip latency (I believe through pings, but can't remember) then doing some light math on those latencies and coming up with very specific performance tuning setting for the Ethernet link (or network storage protocol, can't remember which).
I have been tearing my hair out googling today without much luck.
Does anyone know what I am talking about, and know where to find it?
Much obliged,
Matt
A few months ago in a discussion thread either here or on one of the many forums I frequent, someone linked me to what I recall looked like a blog post.
I now cant find that blog post, nor the message in which someone linked me to it, so I come to you for assistance.
The topic was NAS performance, and why nothing above 10gig (25gbit, 40gbit, 100gbit) performs so slowly out of the box.
In other words, if you install Gigabit, 2.5gbit, 5 gbit or 10gbit networking on your machine, out of the box you are going to be able to pretty much get full performance out of it less a small amount of overhead.
If you install 25gbit, 40gbit, 100gbit, etc. networking, unless you hit it with 20-100 simultaneous iperf threads, you never wind up actually seeing the full bandwidth.
On my direct connected 40gbit NIC between my workstation and NAS, I rarely see more than 1.6GB/s to 1.8GB/s data transfers, despite the storage on the server performing MUCH above that level.
What is the bottleneck has never been evident. There is no great CPU load, the PCIe lanes provide sufficient speeds, etc. etc.
Someone in a conversation on this topic suggested it had to do with hand tuning your implementation based on its latency, and this could drastically improve single link performance.
I remember reading the linked blog post. It was on a Linux or Unix host. It had something to do with measuring round trip latency (I believe through pings, but can't remember) then doing some light math on those latencies and coming up with very specific performance tuning setting for the Ethernet link (or network storage protocol, can't remember which).
I have been tearing my hair out googling today without much luck.
Does anyone know what I am talking about, and know where to find it?
Much obliged,
Matt
