Asymmetric ethernet speeds - seems to be a Windows setting issue

[BAR_FOO]

I recently upgraded my LAN from gigabit to 2.5 gig, replacing the switch between my Windows PC (HP Spectre X360 laptop running Win 11 Pro 25H2) and my NAS (Unraid) with a 2.5 gig unmanaged switch and adding a 2.5 gig PCIE ethernet adapter to the NAS. My PC already had a 2.5 gig adapter (in a Thunderbolt dock). After installing everything, I've found that the connection between the PC and the NAS is slower in one direction.

Using iperf3, with the NAS sending a stream to my PC, I'm getting ~1.3 gigs; in the opposite direction it's full speed, a solid 2.37 gigs. At first I was getting barely a gig, but I rolled back to an older driver, did a Windows network reset, and played with some settings in the adapter properties (turned of all the offloading, packet priority, and VLAN) and that got it up to 1.3, sometimes 1.5. The connection is stable, it's just not getting full speeds both ways.

I know it's a software/configuration issue and not a hardware problem because I've tried:

• booting the same PC to a Linux live USB and it gets over 2 gigs in both directions
• using another laptop running Windows 10 on the same Thunderbolt dock and it gets over 2 gigs both ways as well

It's connecting through an HP Thunderbolt Dock G4 with an Intel I225-LMvP ethernet controller. The driver version is 2.1.5.7, the latest available from Intel. I've also tried the version that HP supplies for this dock specifically, 1.1.4.43, and get highly inconsistent speeds, sometimes a bit faster and often slower.

Any ideas of how to troubleshoot this further? Is there something in the TCP/IP settings in Windows that could be causing this?


UPDATE

I've now tried a different USB ethernet adapter, this one just one gigabit, and the same thing is happening, proportionally: about 200 Mbps download speeds and over 900 Mbps upload, whether on the LAN or to the internet (Ookla speed test). The same adapter, same connection, on a different laptop gets 900+ in both directions. So it must be some setting on my PC, not even specific to the adapter. I've tried Winsock and TCP/IP resets, but they haven't helped.

 

[Falloutboy]

How many and what type of PCIe lanes does your Ethernet device plugged into your Thunderbolt dock require?
If it is an older device say PCIe gen 3 and uses four lanes of PCIe gen 3 to communicate you are actually only going to get one lane of PCIe gen 3, as Thunderbolt 4 AFAIK reserves four lanes on your PC for operating but 3 are reserved downstream for Display Port, the remaining one PCIe lane is reserved for downstream PCIe devices so a Gen 3 x 4 device plugged into a Thunderbolt 4 Port will only get one lane of PCIe Gen 3.
Regardless of how fast everything else is this will limit your connection to this slowest speed.

This would be my best guess for why you aren't getting what you expect.

 

[BAR_FOO]

How many and what type of PCIe lanes does your Ethernet device plugged into your Thunderbolt dock require?
If it is an older device say PCIe gen 3 and uses four lanes of PCIe gen 3 to communicate you are actually only going to get one lane of PCIe gen 3, as Thunderbolt 4 AFAIK reserves four lanes on your PC for operating but 3 are reserved downstream for Display Port, the remaining one PCIe lane is reserved for downstream PCIe devices so a Gen 3 x 4 device plugged into a Thunderbolt 4 Port will only get one lane of PCIe Gen 3.
Regardless of how fast everything else is this will limit your connection to this slowest speed.

This would be my best guess for why you aren't getting what you expect.

Thanks for the suggestion. I'm not sure how to determine how many lanes are available for this. but in Windows settings the dock has
a bandwidth (down/up) of 40Gbps/40Gbps (Gen 3, dual lane).

However, I find it unlikely that this is the issue because (a) it's still getting speeds over 1 Gig, just not the full 2.5, (b) with another, older laptop on the same hub, running Win 10, I'm getting the full 2.5 bidirectionally and (c) with the original laptop, same hardware, I get 2.5 in both directions booting to Linux. So the hardware is definitely capable of it.

 

[Falloutboy]

Thanks for the suggestion. I'm not sure how to determine how many lanes are available for this. but in Windows settings the dock has
a bandwidth (down/up) of 40Gbps/40Gbps (Gen 3, dual lane).

However, I find it unlikely that this is the issue because (a) it's still getting speeds over 1 Gig, just not the full 2.5, (b) with another, older laptop on the same hub, running Win 10, I'm getting the full 2.5 bidirectionally and (c) with the original laptop, same hardware, I get 2.5 in both directions booting to Linux. So the hardware is definitely capable of it.

OK lets dissect this... I am assuming this this is a Thunderbolt 4 dock, regardless of what bandwidth your Thunderbolt system has so many of the PCIe lanes are provided down stream I.E ( 1 PCIe pass though lane on the dock ) so 1 lane of PCIe from your PC is available at the dock ( or if there is a switch in the dock ) it may convert one PCIe gen 4 lane to two PCIe gen 3 lanes ).

BUT if your PCIe network card only using one lane of PCIe 4 for its connection then all you will get is one lane of PCIe gen **3** as that is all that can be provided in the dock..

You need to know the exact specifications of everything you are using via the Thunderbolt systems and what is causing bottle necks , you can not magically fit a lane of PCIe gen 4 on the network card for example over one lane of PCIe Gen 3 on the thunderbolt system.

The interesting thing from my perspective is AFAIK Thunderbolt 4 requires four Gen 4 lanes, and leaves one available ( the rest reserved for display port ) so my difficulty here is understanding where the Gen 3 is coming from.

Now I just checked and while I hate doing it, according to AI...
HP Spectre X360 laptop shares PCIe lanes across two USB (Thunderbolt 4 Ports) so only use one of the USB ports and hopefully that gives you all four lanes..
BUT - remember Thunderbolt 4 takes 3 of those 4 lanes and assigns them to display port so now you have 1 lane of PCIe.

So firstly is the model of the dock and are you using both ports on the laptop?


** UPDATED **

Both USB4 ports on the laptop share PCIe lanes
The dock you mentioned has a 2.5GB nic built in.

Best suggestion at this point.. only use one USB4 port on the laptop and ONLY the 2.5GB nic on the dock with nothing else connected to the dock and see if things change....

Please let me know... how you get on.

 

[mattventura]

What is the exact iperf3 command you're using? iperf3 on windows is known to not have great performance.

 

[BAR_FOO]

What is the exact iperf3 command you're using? iperf3 on windows is known to not have great performance.

I've been using "iperf3 -c -V 192.168.1.X", and I've also tried -P with various numbers of parallel streams. On the server just "iperf3 -s".

It's not just iperf, though: the same differential in speed happens with file transfers. Going from the PC to the NAS gets ~1.3 gigs, the other direction is over 2.4 (via SMB).

 

[BAR_FOO]

So firstly is the model of the dock and are you using both ports on the laptop?

The dock is the HP Thunderbolt Dock 120W G4. It's the only USB/Thunderbolt device plugged into the laptop. I've tried it without the monitor and that doesn't change the ethernet speeds. According to the specs, the laptop has two USB4/Thunderbolt ports, "with USB Type-C® 40Gbps signaling rate (USB Power Delivery, DisplayPort™ 1.4, HP Sleep and Charge)."

The older Win 10 laptop that gets full 2.5 speeds with the same dock on the same connection only has Thunderbolt 3.

 

[blunden]

What is the exact iperf3 command you're using? iperf3 on windows is known to not have great performance.

It's true that the official iperf3 builds for Windows are old and outdated, and therefore doesn't perform well.

Newer unofficial builds of iperf3 for Windows perform pretty well though, at least up to 10 Gbit/s. I don't have faster network equipment to test with.

 

[TrevorH]

How many cables have you tried changing? Also, have you tried different switch ports?

 

[BAR_FOO]

It's true that the official iperf3 builds for Windows are old and outdated, and therefore doesn't perform well.

I'm using ar51an's build of 3.20, which should be up to date.

 

[BAR_FOO]

How many cables have you tried changing? Also, have you tried different switch ports?

I've tried two different ports and various cables. And the same PC, same dock, same cable gets the full 2.5 gig Linux. And the older laptop running Windows 10, on the same dock with the same cable, gets 2.5 gig as well.

 

[BAR_FOO]

An update: I took the laptop to my office, which has gigabit ethernet through an entirely different dock (USB-C with a Realtek chipset). There it gets about 300 Mb/s down and 900 Mb/s up on a speed test. An Android phone plugged into the same dock gets 900 Mb/s both ways.

So clearly there is a setting somewhere in Windows that is nerfing the connection speed on the laptop. I've tried the network reset in Windows settings, but that didn't fix it.

 

[tgl]

Maybe this is helpful or maybe not: I have seen similar asymmetric behavior in a totally different context. I have some MoCA adapters that are rated for 2.5G, and they can deliver solid 2.35Gbps according to iperf3 ... but when connected to certain devices, it's 2.35Gbps in one direction and much less (200-400Mbps) in the other. No Windows anywhere in the picture; the connected devices are switches of one brand or another. The best theory I can come up with is that it's a bufferbloat effect somewhere, but I don't have enough visibility into either the MoCA adapters or the switches to be sure about that.

One very interesting detail is that with multiple iperf3 streams (6 or more) the total throughput gets pretty close to the rated speed, even with a setup that exhibits the problem for fewer streams. If your iperf3 version can do multiple streams, what effect does that have for you?

 

[BAR_FOO]

Maybe this is helpful or maybe not: I have seen similar asymmetric behavior in a totally different context. I have some MoCA adapters that are rated for 2.5G, and they can deliver solid 2.35Gbps according to iperf3 ...

One very interesting detail is that with multiple iperf3 streams (6 or more) the total throughput gets pretty close to the rated speed, even with a setup that exhibits the problem for fewer streams. If your iperf3 version can do multiple streams, what effect does that have for you?

Good thought about MoCA, I do indeed use MoCA to connect to the ISP's fiber box. However, between the PC and the NAS is only a single unmanaged switch, and disconnecting from the WAN entirely makes no difference to the transfer speed.

With multiple streams, too, the total transfer rate basically doesn't change much, with numbers of streams from 1 to 128, but interestingly with high numbers it does go up a little on the initial batch and then slows down on subsequent runs.

 

[TrevorH]

This is traffic into the windows machine? Could it be something like anti-virus or firewall inspecting inbound packets but passing outbound ones without checking?

 

[BAR_FOO]

This is traffic into the windows machine? Could it be something like anti-virus or firewall inspecting inbound packets but passing outbound ones without checking?

I tried uninstalling the antivirus package from my employer (Cisco Endpoint Protection) and temporarily disabling Windows Defender, and that made no difference. Cisco Umbrella (an internet threat protection package) was also installed and I removed that. Resetting Windows Firewall and turning it off likewise didn't change things.