>=25G network not performing well with Windows 11
- Thread starter Chriggel
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Sure, it's almost standard. The only thing I've changed is the RTO, which was set to 1000 but I've seen that actually 3000 should be standard, once again I have no explanation why out of the box installations have these differences. Changing it to 3000 had no effect so far, but I also didn't expect it to have one.
Globale TCP-Parameter
----------------------------------------------
Zustand der empfangsseitigen Skalierung : enabled
Autom. Abstimmungsgrad Empfangsfenster : normal
Add-On "Überlastungssteuerungsanbieter" : default
ECN-Funktion : disabled
RFC 1323-Zeitstempel : allowed
RTO (anfänglich) : 3000
Zustand der Empfangssegmentzusammenfügung : enabled
Nicht-SACK-RTT-Widerstandsfähigkeit : disabled
Maximale SYN-Neuübertragungen : 4
Fast Open : enabled
Fast Open-Fallback : enabled
HyStart : enabled
Proportionale Ratenreduzierung : enabled
Schrittsteuerungsprofil : off
** Die oben angezeigte autotuninglevel-Einstellung ist das Ergebnis einer Gruppenrichtlinie, die alle lokalen
Einstellungen außer Kraft setzt.
It's German, the second entry is the auto tuning and the ** at the bottom says that the displayed auto tuning setting is the result of a group policy which overrides all local changes. That's why I can't change or disable it.
Globale TCP-Parameter
----------------------------------------------
Zustand der empfangsseitigen Skalierung : enabled
Autom. Abstimmungsgrad Empfangsfenster : normal
Add-On "Überlastungssteuerungsanbieter" : default
ECN-Funktion : disabled
RFC 1323-Zeitstempel : allowed
RTO (anfänglich) : 3000
Zustand der Empfangssegmentzusammenfügung : enabled
Nicht-SACK-RTT-Widerstandsfähigkeit : disabled
Maximale SYN-Neuübertragungen : 4
Fast Open : enabled
Fast Open-Fallback : enabled
HyStart : enabled
Proportionale Ratenreduzierung : enabled
Schrittsteuerungsprofil : off
** Die oben angezeigte autotuninglevel-Einstellung ist das Ergebnis einer Gruppenrichtlinie, die alle lokalen
Einstellungen außer Kraft setzt.
It's German, the second entry is the auto tuning and the ** at the bottom says that the displayed auto tuning setting is the result of a group policy which overrides all local changes. That's why I can't change or disable it.
The only difference I see with mine is the Initial RTO (mine is the default 1000) and RFC 1323 Timestamps are disabled for me (this appears to be the default on Windows). I wouldn't expect RFC 1323 Timestamps to be the problem, as from what I read, if anything, that should make my setup slower.
The really strange part to me is you have group policy applied despite not being domain joined. Are you logging on using a Microsoft account? I think that can apply some nonsense on Win11. So when you test Win11 Enterprise, make sure you create a local account so no unexpected GPOs get applied.
Code:
PS C:\Users\DAVe3283\Apps\Networking\iperf3> netsh interface tcp show global
Querying active state...
TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State : enabled
Receive Window Auto-Tuning Level : normal
Add-On Congestion Control Provider : default
ECN Capability : disabled
RFC 1323 Timestamps : disabled
Initial RTO : 1000
Receive Segment Coalescing State : enabled
Non Sack Rtt Resiliency : disabled
Max SYN Retransmissions : 4
Fast Open : enabled
Fast Open Fallback : enabled
HyStart : enabled
Proportional Rate Reduction : enabled
Pacing Profile : off
PS C:\Users\DAVe3283\Apps\Networking\iperf3> .\iperf3.exe -c ThreadReaper -P 8 -M 8960
Connecting to host ThreadReaper, port 5201
<snip>
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[SUM] 0.00-10.00 sec 29.9 GBytes 25.7 Gbits/sec sender
[SUM] 0.00-10.00 sec 29.9 GBytes 25.7 Gbits/sec receiver
iperf Done.The really strange part to me is you have group policy applied despite not being domain joined. Are you logging on using a Microsoft account? I think that can apply some nonsense on Win11. So when you test Win11 Enterprise, make sure you create a local account so no unexpected GPOs get applied.
This isnt that surprising, you can happily apply group policy settings to a non domain machine. It is surprising that its happening without you doing it.
Check Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies and Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies
Just wanted to let all of you know that I appreciate all the input of everyone who tried to help, but I've given up. I've decided to not waste any more time with this mess. I'll either use dual boot or maybe get a second machine and will use Linux for my main desktop.
So, even though it appears to be working for many people, I know that I'm not alone with this problem. If anyone should find this in the future, sadly I can't offer you a solution. I was never able to make any sense of this mess. I don't easily give up, but I've given up this time. Sorry.
So, even though it appears to be working for many people, I know that I'm not alone with this problem. If anyone should find this in the future, sadly I can't offer you a solution. I was never able to make any sense of this mess. I don't easily give up, but I've given up this time. Sorry.
I have some further info regarding Windows 11 and 10gb networking. Might shed some further light on the issues OP ran into.
I was working on one of my older test motherboards tonight, an Asrock Z170 Extreme4. Its running a I7-6700K, and 16 gigs of ram. I was testing to see if the Intel X520 , X710 and E810 cards would behave performance wise if you only give it 4x pci-e lanes. Long story short, they did fine with 4x lanes in the first tested motherboard. I was using Iperf3 3.16 for all my tests.
Initially I was using Debian 12 to run all my tests. After I finished up in Debian, I decided to try Windows 10 LTSC 21H2, Windows server 2022, and Windows 11 23H2 pro to see how the various Windows OS's compared. I had previously tested the cards on my more powerful test machine (a I7-9700K on a Z370 motherboard), but the Z370 box has a different pci-e configuration and I wanted to make sure I was only giving the cards 4x.
Windows 10 performed as expected, giving a solid 9.5gb in Iperf with all the cards. Server 2022 also behaved the same. Windows 11 though......
The results with 11 were all over the place, and no card, even the vaunted Mellanox 4x, was able to give consistent network transfers. I even moved the cards to the 16x lane slot to see if that would help, and shockingly it made things even worse!
I wondered if this was some sort of bug with this old board, so I dug out another old test board an Asrock Z170 pro4. The pro4 was even worse! It could barely hold 5gb, regardless of what card I tried, or what slot I plugged the card into, or even what OS was tested.
Conclusion #1 - I think the server grade cards we're all using really can get PICKY about all sorts of things. I don't think the vendors really test or design for consumer grade systems. Different motherboards, different OS's, different pci-e slots, etc. Change one variable, and you can get totally different results.
Conclusion #2 - Windows 11 was the most temperamental OS in terms of network performance. On my Z370 it performs fine, but not on the Z170 boards (either one). Considering how M$ has treated 11 as a non stop beta I'm not terribly surprised though.
Conclusion #3 - The Mellanox 4x's seem to be the most "stable" in terms of network performance. But even they're not immune to buggy motherboards (i.e my Asrock pro 4 ), or certain combinations of things (i.e. Win11 + the Extreme4 Z170).
Conclusion #4 - Keep a Linux box with a modern distro handy(or at least a flash drive with a live OS). It makes ruling out weirdness in Windows kernel or drivers, or the Windows iperf3 client a lot easier.
Conclusion #5 - consumer motherboards has unknown bugs or quirks in them that can cause issues. I never would of expected my pro 4 test board to act the way it did, yet the extreme4 board behaved just fine.
I was working on one of my older test motherboards tonight, an Asrock Z170 Extreme4. Its running a I7-6700K, and 16 gigs of ram. I was testing to see if the Intel X520 , X710 and E810 cards would behave performance wise if you only give it 4x pci-e lanes. Long story short, they did fine with 4x lanes in the first tested motherboard. I was using Iperf3 3.16 for all my tests.
Initially I was using Debian 12 to run all my tests. After I finished up in Debian, I decided to try Windows 10 LTSC 21H2, Windows server 2022, and Windows 11 23H2 pro to see how the various Windows OS's compared. I had previously tested the cards on my more powerful test machine (a I7-9700K on a Z370 motherboard), but the Z370 box has a different pci-e configuration and I wanted to make sure I was only giving the cards 4x.
Windows 10 performed as expected, giving a solid 9.5gb in Iperf with all the cards. Server 2022 also behaved the same. Windows 11 though......
The results with 11 were all over the place, and no card, even the vaunted Mellanox 4x, was able to give consistent network transfers. I even moved the cards to the 16x lane slot to see if that would help, and shockingly it made things even worse!
I wondered if this was some sort of bug with this old board, so I dug out another old test board an Asrock Z170 pro4. The pro4 was even worse! It could barely hold 5gb, regardless of what card I tried, or what slot I plugged the card into, or even what OS was tested.
Conclusion #1 - I think the server grade cards we're all using really can get PICKY about all sorts of things. I don't think the vendors really test or design for consumer grade systems. Different motherboards, different OS's, different pci-e slots, etc. Change one variable, and you can get totally different results.
Conclusion #2 - Windows 11 was the most temperamental OS in terms of network performance. On my Z370 it performs fine, but not on the Z170 boards (either one). Considering how M$ has treated 11 as a non stop beta I'm not terribly surprised though.
Conclusion #3 - The Mellanox 4x's seem to be the most "stable" in terms of network performance. But even they're not immune to buggy motherboards (i.e my Asrock pro 4 ), or certain combinations of things (i.e. Win11 + the Extreme4 Z170).
Conclusion #4 - Keep a Linux box with a modern distro handy(or at least a flash drive with a live OS). It makes ruling out weirdness in Windows kernel or drivers, or the Windows iperf3 client a lot easier.
Conclusion #5 - consumer motherboards has unknown bugs or quirks in them that can cause issues. I never would of expected my pro 4 test board to act the way it did, yet the extreme4 board behaved just fine.
I would "invert" that statement and say the consumer hardware manufacturers don't test their systems enough...
(My most recent experience was an asus board that wouldn't boot with a >x4 pice device (hba or 25GBE nic) in a x8 slot and the gpu in the x16 slot. This was later fixed with a bios update)
Agreed that the consumer vendors do minimal testing on their stuff, and let the end users find all the bugs for them. You just hope that a bios upgrade will fix it later.
Sometimes even the "big" boys don't test very well. I got a new Supermicro server (based on a H13SAE-MF board) a few weeks ago at work that will not boot with either an E810 or Mellanox 4 card. But its perfectly happy with a X710 or X520. I need to send them a nasty gram and hopefully they can address it with a bios update.
Sometimes even the "big" boys don't test very well. I got a new Supermicro server (based on a H13SAE-MF board) a few weeks ago at work that will not boot with either an E810 or Mellanox 4 card. But its perfectly happy with a X710 or X520. I need to send them a nasty gram and hopefully they can address it with a bios update.
I also verified the problems with W11 and Server 2022 on a dual 3647 workstation platform and it was the same thing. I understand the testing argument, but I'm not sure how a potential bug would cause this or at least contribute to the problem
Meanwhile, I've noticed more weird behaviour with W11. It's a mess and broken in many ways.
Meanwhile, I've noticed more weird behaviour with W11. It's a mess and broken in many ways.