Connect-X 2 40Gbps IPoIB Windows-Linux performance issue

[jrandria]

Hi everyone,

I need your help !

I'm working with 3 identical boxes embedding Intel Xeon E5-2650 CPUs and connected with Mellanox MT26428 ConnectX-2 40Gbps cards.

Two boxes are running Redhat with OpenFabrics driver whereas the third one is running Windows 10 with the last Mellanox WinOFED drivers (note that i only have the admin privileges on the Windows box).

Running iperf gives me an acceptable bandwidth between the 2 Linux boxes (~19 Gbits/sec) but gives me poor results between Linux and Windows (~1.5 Gbits/sec).

I have tried to tune my Windows settings by following this Windows server tunning guide (HowTo Configure RSS on ConnectX-3 Pro for Windo... | Mellanox Interconnect Community) but i didn't get significant progress.

Do you have any idea concerning this performance issue ?

Due to those IPoIB poor performance between Linux and Windows, i'm considering using RSockets API. Are they supported with Mellanox WinOFED ?

Thank you !

 

[i386]

Running iperf gives me an acceptable bandwidth between the 2 Linux boxes (~19 Gbits/sec) but gives me poor results between Linux and Windows (~1.5 Gbits/sec).

I'm no expert for infiniband but even the linux performance is lower than I would expect (~25gbit/s) from qdr ib (32gbit/s).

Are the windows drivers up to date? Firmware? Did you try to swap the cards or cables?