I was wondering if anyone had experience tuning the Mellanox cards for 10gb performance. I seem to be having an issue where sending is very slow, but receiving is considerably better. I am using the 5.35.12978.0 drivers on Win10 Client and 2016 Server. Firmware has been updated to 2.9.1200 on both devices. Send and receive buffers maxed to 4096. Jumbo frames not enabled.
Sending
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 68.4 MBytes 573 Mbits/sec
[ 4] 1.00-2.00 sec 136 MBytes 1.14 Gbits/sec
[ 4] 2.00-3.00 sec 218 MBytes 1.83 Gbits/sec
[ 4] 3.00-4.00 sec 226 MBytes 1.89 Gbits/sec
[ 4] 4.00-5.00 sec 202 MBytes 1.69 Gbits/sec
[ 4] 5.00-6.00 sec 164 MBytes 1.38 Gbits/sec
[ 4] 6.00-7.00 sec 199 MBytes 1.67 Gbits/sec
[ 4] 7.00-8.00 sec 227 MBytes 1.90 Gbits/sec
[ 4] 8.00-9.00 sec 231 MBytes 1.94 Gbits/sec
[ 4] 9.00-10.00 sec 232 MBytes 1.95 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.86 GBytes 1.60 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.86 GBytes 1.60 Gbits/sec receiver
Receiving
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 680 MBytes 5.70 Gbits/sec
[ 4] 1.00-2.00 sec 789 MBytes 6.62 Gbits/sec
[ 4] 2.00-3.00 sec 788 MBytes 6.61 Gbits/sec
[ 4] 3.00-4.00 sec 792 MBytes 6.64 Gbits/sec
[ 4] 4.00-5.00 sec 792 MBytes 6.64 Gbits/sec
[ 4] 5.00-6.00 sec 791 MBytes 6.64 Gbits/sec
[ 4] 6.00-7.00 sec 800 MBytes 6.71 Gbits/sec
[ 4] 7.00-8.00 sec 798 MBytes 6.69 Gbits/sec
[ 4] 8.00-9.00 sec 800 MBytes 6.71 Gbits/sec
[ 4] 9.00-10.00 sec 804 MBytes 6.75 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 7.65 GBytes 6.57 Gbits/sec sender
[ 4] 0.00-10.00 sec 7.65 GBytes 6.57 Gbits/sec receiver
Sending
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 68.4 MBytes 573 Mbits/sec
[ 4] 1.00-2.00 sec 136 MBytes 1.14 Gbits/sec
[ 4] 2.00-3.00 sec 218 MBytes 1.83 Gbits/sec
[ 4] 3.00-4.00 sec 226 MBytes 1.89 Gbits/sec
[ 4] 4.00-5.00 sec 202 MBytes 1.69 Gbits/sec
[ 4] 5.00-6.00 sec 164 MBytes 1.38 Gbits/sec
[ 4] 6.00-7.00 sec 199 MBytes 1.67 Gbits/sec
[ 4] 7.00-8.00 sec 227 MBytes 1.90 Gbits/sec
[ 4] 8.00-9.00 sec 231 MBytes 1.94 Gbits/sec
[ 4] 9.00-10.00 sec 232 MBytes 1.95 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.86 GBytes 1.60 Gbits/sec sender
[ 4] 0.00-10.00 sec 1.86 GBytes 1.60 Gbits/sec receiver
Receiving
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 680 MBytes 5.70 Gbits/sec
[ 4] 1.00-2.00 sec 789 MBytes 6.62 Gbits/sec
[ 4] 2.00-3.00 sec 788 MBytes 6.61 Gbits/sec
[ 4] 3.00-4.00 sec 792 MBytes 6.64 Gbits/sec
[ 4] 4.00-5.00 sec 792 MBytes 6.64 Gbits/sec
[ 4] 5.00-6.00 sec 791 MBytes 6.64 Gbits/sec
[ 4] 6.00-7.00 sec 800 MBytes 6.71 Gbits/sec
[ 4] 7.00-8.00 sec 798 MBytes 6.69 Gbits/sec
[ 4] 8.00-9.00 sec 800 MBytes 6.71 Gbits/sec
[ 4] 9.00-10.00 sec 804 MBytes 6.75 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 7.65 GBytes 6.57 Gbits/sec sender
[ 4] 0.00-10.00 sec 7.65 GBytes 6.57 Gbits/sec receiver