Gigabit LACP and 10Gb

SIlviu

Member
May 27, 2016
68
3
8
31
I have some servers with 4 x LACP with 1Gb NICs and some servers with 10Gb NICs no LACP.

The servers with 4 x LACP over 1Gb work well and I can transfer data at the speed of 4Gb/s between them.

When a server with 10Gb NIC makes a transfer with a server over 4 x LACP 1Gb it only works at 1Gb/s.

Is there a setting or smth I must do to make this work or I need 4 x LACP 10Gb so it can do 4Gb/s ?
 

ttabbal

Active Member
Mar 10, 2016
743
199
43
43
That's just how LACP works. A single stream between machines can only use one interface. If you can get the machines to use multiple streams, some switches can break them over the multiple connections. It depends on the exact firmware and such. Usually, it's a hash of the IP and ports. I'm a little surprised you say you are getting 4Gbps between LACP servers. Most people don't.

I played with it a while back. It works alright if you have a lot of clients hitting a server that has LACP running. But there are no guarantees. Sometimes the client data will hash out the same as another and a few end up on a single link. Other times, they spread out pretty evenly. After experimenting for a while, I decided not to bother with it and just run 10Gbe when I want >1gbit/s.
 
  • Like
Reactions: Stux

SIlviu

Member
May 27, 2016
68
3
8
31
The switch I use is a Dlink DGS-1510-52 with LACP load balance algorithm by source mac and the servers use windows server 2012 r2 with nic teaming set to lacp dynamic.

Forgot to mention
Server 1 = 4 x 1 Gb LACP
Server 2 = 1 x 10Gb

If Server 1 downloads from Server 2 I get 1Gb/s, if Server 1 uploads to Server 2 I get 4Gb/s
 

manfri

Member
Nov 19, 2015
44
7
8
53
The switch I use is a Dlink DGS-1510-52 with LACP load balance algorithm by source mac
This explain everything: because the source mac of 10gb nic is only one, the switch cannot do any type of balancing and the traffic goes to only one of the 4 NIC.

Windows teaming can redistribute the transmission using the concept of flowlets, and transmit multiple stream of data distributed between the 4 NIC (if the software user multiple TCP connections) even to a single destination server.

So, with the help of the right software, you can transmit to one destination at more than 1GB of data.
LBFO Dynamic Teaming mode may drop send packets in Windows Server 2012 R2
 
Last edited:

Stux

Member
May 29, 2017
30
10
8
42
The switch I use is a Dlink DGS-1510-52 with LACP load balance algorithm by source mac and the servers use windows server 2012 r2 with nic teaming set to lacp dynamic.

Forgot to mention
Server 1 = 4 x 1 Gb LACP
Server 2 = 1 x 10Gb

If Server 1 downloads from Server 2 I get 1Gb/s, if Server 1 uploads to Server 2 I get 4Gb/s
If your 10gbe server was running with 4 interfaces on the one NIC... say within an ESXi environment... and each interfrace had a different MAC...

Or maybe with SR-IOV.

Then I'd expect the load balancing to work.