[SOLVED]Slow speeds between two Connectx-2 machines

Discussion in 'Networking' started by rubylaser, Mar 1, 2016.

  1. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,715
    Likes Received:
    425
    I think -p is parallel streams, not # of threads
     
    #61
  2. arglebargle

    arglebargle H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

    Joined:
    Jul 15, 2018
    Messages:
    636
    Likes Received:
    211
    Ah, yeah, you're right. I assumed -P would spawn one thread per stream but with -P 8 I only see two.
     
    #62
  3. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    I updated the Firmware to 2.32.5100, it was 2.11
    I see no real improvement :(
    argh this is driving me crazy :(
    iperf results new#2.JPG
     
    #63
  4. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,715
    Likes Received:
    425
  5. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    I am unclear on how to run this on the Synology DS1618+.
     
    #65
  6. saivert

    saivert Member

    Joined:
    Nov 2, 2015
    Messages:
    118
    Likes Received:
    12
    @arglebargle
    Yes you are right as long as you pick Linux or FreeBSD as OS but as long as you go with Windows something fishy is going on. Either issues with receive side scaling handling or some bug in the Windows drivers. Seems to be very reliant on having lots of free CPU cycles. This never impacts TX throughput. Only RX.
    Of course this is only my simple conclusions from lots of testing on my old system before I upgraded to Coffee Lake i7 8700k where I can't replicate this issue anymore.

    On the server side i have Dual Xeon E5-2670 and I have tried both ConnectX-2 and Intel X520. I'm currently using the X520 card as it was just easier to enable SR-IOV on and also doesn't seem to generate as much heat and has a bigger heatsink too. Just more suitable for my server.
     
    #66
  7. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Yes, sadly I think I am going to have to go X520 :(
     
    #67
  8. BackupProphet

    BackupProphet Well-Known Member

    Joined:
    Jul 2, 2014
    Messages:
    790
    Likes Received:
    279
    I did some testing tonight on my Connect-X2. Use iperf2, do not use iperf3. Ubuntu/FreeBSD has iperf2 in their repositories, so that is also what I downloaded for Windows.

    The bandwidth numbers from iperf2 on Windows are not correct. Use the task manager to check the actual bandwidth. The numbers are correct on the Linux/FreeBSD side. If you try Windows to Windows you can only rely on the task manager. Good luck.

    I managed to fully utilize 10G on both send and receive with Windows.
     
    #68
  9. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Good points BackupProphet.

    However, I have used the taskmanager, it shows the highest bandwidth I get at about 2.3G.
    ATTO also confirms these numbers.

    So, I saw some one mention cables, could this be an issue ?

    From reading around through google searchs which many times points back to STH. A large number of people have this issue, but for some everything seems to work fine. I am at a complete loss on my issue.
    Maybe its my platform … the ASUS Z10PE-D16 WS.. the BIOS is updated to latest.
    I thought for awhile it was my switch D-Link DXS-1210-12TC … but if I direct connect WS to DS1618+ I see about the same numbers... 2.3G
     
    #69
  10. BackupProphet

    BackupProphet Well-Known Member

    Joined:
    Jul 2, 2014
    Messages:
    790
    Likes Received:
    279
    Try using iperf2
     
    #70
  11. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Its not any better :(
    Actually its worse.

    iperf2 results.JPG
     
    #71
  12. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Here is the Task Manager.
    You can see on ATTO that it caps out on reads at 1G, makes it to about 2.3G on writes..

    As far as using the ConnectX2, I has these in before and they do the exact same thing.

    Task Manager.JPG
     
    #72
  13. BackupProphet

    BackupProphet Well-Known Member

    Joined:
    Jul 2, 2014
    Messages:
    790
    Likes Received:
    279
    And you have replaced the cable?
     
    #73
  14. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Not yet, I have to order a pair.
     
    #74
  15. saivert

    saivert Member

    Joined:
    Nov 2, 2015
    Messages:
    118
    Likes Received:
    12
    For what its worth I always had Task manager up when testing as I was also measuring CPU utilization during the iperf2 runs. And iperf2 and Task Manager seemed to agree.
    I never had as low as 2.3Gbps though. It was usually either full 9.8Gbps or ~6.5Gbps.

    The only reason I got the Mellanox cards was because they were cheap. Certainly not for the quality or ease of use.
     
    #75
  16. William

    William Active Member

    Joined:
    May 7, 2015
    Messages:
    762
    Likes Received:
    236
    Yup, the first two ConnectX2 cards I got was like $15 each with cables !
    The ConnectX3 card I just got was $28.

    So all in all it wasn't a huge investment.

    But sadly nothing seems to be working to get these cards up to 10G speeds. Very strange.
     
    #76
Similar Threads: [SOLVED]Slow speeds
Forum Title Date
Networking Fluctuating speeds on 40gb Mellanox Aug 4, 2019
Networking Different Speeds from opposite ends of a VPN [Diagram]? Oct 2, 2018
Networking Can't get 10Gbit speeds on Intel X550-T1 Jul 3, 2018
Networking ConnectX-3s Work with 10, 40 and 56 speeds? Oct 10, 2017
Networking Not getting 10gbe Speeds Jul 14, 2017

Share This Page