How to acheive SMOKIN' ZFS send/recv transfers

[Rand__]

FreeNas perf issues usually where tuning topics like send/recv buffers - not sure the values still apply but I had a post in the FN forum with recommended 10g config values, was copied here somewhere.

 

[Rand__]

So I have played around a bit and I had a weird effect just now - I was looking at these tuning params
Mellanox ConnectX-3
when I saw

Also, be sure that adaptive-rx is off:
ethtool -C eth0 adaptive-rx off

Found:
esxcli network nic coalesce get




So I looked at how to do it in ESX - fairly simple in 6.5:

esxcli network nic coalesce set -a off -n vmnic2

Did that, ran a test - 8Gb/s. (down from 12 which I am trying to fix up to at least 20 for a 56Gbit connection)
So I thought - not good and reset to <on>

Reran the test to check it - 17Gb/s - with the same setting as before ... The other box even did 23 (from 12) - totally don't understand that. Wonder whether it holds