Gigabit LACP and 10Gb

Discussion in 'Networking' started by SIlviu, Sep 8, 2017.

  1. SIlviu

    SIlviu Member

    Joined:
    May 27, 2016
    Messages:
    64
    Likes Received:
    3
    I have some servers with 4 x LACP with 1Gb NICs and some servers with 10Gb NICs no LACP.

    The servers with 4 x LACP over 1Gb work well and I can transfer data at the speed of 4Gb/s between them.

    When a server with 10Gb NIC makes a transfer with a server over 4 x LACP 1Gb it only works at 1Gb/s.

    Is there a setting or smth I must do to make this work or I need 4 x LACP 10Gb so it can do 4Gb/s ?
     
    #1
  2. ttabbal

    ttabbal Active Member

    Joined:
    Mar 10, 2016
    Messages:
    723
    Likes Received:
    193
    That's just how LACP works. A single stream between machines can only use one interface. If you can get the machines to use multiple streams, some switches can break them over the multiple connections. It depends on the exact firmware and such. Usually, it's a hash of the IP and ports. I'm a little surprised you say you are getting 4Gbps between LACP servers. Most people don't.

    I played with it a while back. It works alright if you have a lot of clients hitting a server that has LACP running. But there are no guarantees. Sometimes the client data will hash out the same as another and a few end up on a single link. Other times, they spread out pretty evenly. After experimenting for a while, I decided not to bother with it and just run 10Gbe when I want >1gbit/s.
     
    #2
    Stux likes this.
  3. SIlviu

    SIlviu Member

    Joined:
    May 27, 2016
    Messages:
    64
    Likes Received:
    3
    The switch I use is a Dlink DGS-1510-52 with LACP load balance algorithm by source mac and the servers use windows server 2012 r2 with nic teaming set to lacp dynamic.

    Forgot to mention
    Server 1 = 4 x 1 Gb LACP
    Server 2 = 1 x 10Gb

    If Server 1 downloads from Server 2 I get 1Gb/s, if Server 1 uploads to Server 2 I get 4Gb/s
     
    #3
  4. manfri

    manfri Member

    Joined:
    Nov 19, 2015
    Messages:
    42
    Likes Received:
    5
    This explain everything: because the source mac of 10gb nic is only one, the switch cannot do any type of balancing and the traffic goes to only one of the 4 NIC.

    Windows teaming can redistribute the transmission using the concept of flowlets, and transmit multiple stream of data distributed between the 4 NIC (if the software user multiple TCP connections) even to a single destination server.

    So, with the help of the right software, you can transmit to one destination at more than 1GB of data.
    LBFO Dynamic Teaming mode may drop send packets in Windows Server 2012 R2
     
    #4
    Last edited: Sep 8, 2017
  5. Stux

    Stux Member

    Joined:
    May 29, 2017
    Messages:
    30
    Likes Received:
    10
    If your 10gbe server was running with 4 interfaces on the one NIC... say within an ESXi environment... and each interfrace had a different MAC...

    Or maybe with SR-IOV.

    Then I'd expect the load balancing to work.
     
    #5
Similar Threads: Gigabit LACP
Forum Title Date
Networking Multi-gigabit bandwidth without LACP Sep 11, 2015
Networking Quiet or fanless 24 port Gigabit PoE+ switch Oct 30, 2019
Networking Identifying Counterfeit Intel Gigabit CT Network Adapters Jul 11, 2019
Networking DIY 40Gigabit Switch? Oct 13, 2018
Networking Wifi ac router with 8port gigabit POE/POE+ Oct 8, 2018

Share This Page