10gb network only getting 2.5gb on windows

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
Ok, so I got my new to me Brocade 7250 48 port POE in and been playing around with it.
Got it mostly configured and ready to go.

Tested 1 of my POE ReoLink 4k cameras and works
1gb nic pluged in and works
10gb x4 via fiber working

interestingly enough though.. I cannot get more than 2.5gb out of iperf.
Tried windows 10 to Windows 11...
just tried server 2022 to server 2022 and still no more.

my old connectx-1 cards, I was able to get 1.09gb/s (yes i know caps matter in that but I am not that smart lol)

trying to go from the old connectx cx4 cable to LC fiber and just not getting the same performance.

Am I missing something?

1710115991372.png

1710116017018.png
I do see RDMA Capable is False and on my old connectx-2 it was True


1710116147964.png


1710116303984.png


Hyper-V is installed on this box...


old screenshot on my connectx-2 cards showing i can get 1gb

1710116228416.png
 
acquacow

acquacow

Well-Known Member
Feb 15, 2017
787
439
63
42
start/run gpedit.msc

Local computer policy > Computer configuration > Windows settings > Right click Policy-based QoS > Advanced QoS Settings > Inbound TCP Traffic > Specify inbound TCP throughput level > Level 3

And then adjust all of your 10gig NIC hardware options for max perf on both ends.

You may also want to make sure SMB Multi-channel is enabled so that you can use multiple CPU threads.
 
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
ok did gpedit on both and no difference


From my googling, people are saying if you do linux distro or something and do iperf, everything is fine.. so it is windows.. go figure.. frustrating...
 
I

i386

Well-Known Member
Mar 18, 2016
4,245
1,546
113
34
Germany
Do you have a 1gbe port connected on the source or target host?
Have you tried to set the metrik manually?
 
S

SchirgMa

New Member
Feb 16, 2017
1
1
3
45
Increase send and receive Buffer of your NIC (Control Panel > Devicemanger > Settings of your NIC > advanced) Increase to the maximum possible

activate Jumbo Pakets if needed
 
  • Like
Reactions: TeleFragger
MountainBofh

MountainBofh

Active Member
Mar 9, 2024
123
94
28
An idea to rule out software or driver issues. Boot the system off a usb flash drive with a modern version of Linux ( I'd suggest Parted Magic or Debian 12 ), and see if the Iperf results are any different.
 
  • Like
Reactions: blunden and TeleFragger
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
SchirgMa said:
Increase send and receive Buffer of your NIC (Control Panel > Devicemanger > Settings of your NIC > advanced) Increase to the maximum possible

activate Jumbo Pakets if needed
Click to expand...
did send to 2048, receive to 16384...
jumbo made it really bad
 
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
Tech Junky said:
Well, that changes things. Fiber either works or doesn't.

Swap the SFP then and see if it's just bad or dirty.
Click to expand...
yeah i have 4 machines where i put the cards in... and I have cross iperfed every which way to the same results... lol
i gues i will live with it for now... 2.5gb is better than 1gb
 
T

Tech Junky

Active Member
Oct 26, 2023
351
120
43
@TeleFragger

If you have a spare SFP just go to each machine and swap / run test and figure out which one is bad. Or pull the extras from 2 machines and test. Obviously you have it pinned down to 1 path that's giving you an issue so, it should be on one end or the other causing the slow speed.

The other options could be blame it on windows or slow disk speed if you're using a spinner in one of the machines. The alternative would be boot to Linux from USB on both sides or all 4 and test from there with RAM speeds bypassing any disk/OS issues.

Another thing is double check the slow machine and make sure it's in a good slot with enough lanes. There are some really bad OEM MOBOs that cut lanes significantly just to provide slots. I've come across a few dropping to x1 electrical but an x4/x8 physical.
 
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
Tech Junky said:
@TeleFragger

If you have a spare SFP just go to each machine and swap / run test and figure out which one is bad. Or pull the extras from 2 machines and test. Obviously you have it pinned down to 1 path that's giving you an issue so, it should be on one end or the other causing the slow speed.
Click to expand...
i am getting 2.5gb no matter what machines i try from. I cannot isolate just 1...and different OS..
I went from win11 to win10, server 2022 to server 2022, win11 to server 2022...

The other options could be blame it on windows or slow disk speed if you're using a spinner in one of the machines. The alternative would be boot to Linux from USB on both sides or all 4 and test from there with RAM speeds bypassing any disk/OS issues.
Click to expand...
All NVME and SSD here, so no spinneys involved...
I want to say Windows BUT... my connectx-1 cards on old cx4 cables got 1.09gb/s transfers versus 350-400mb/s transfers... so is it OS?
might be drivers.... or tweaks on card settings.

Another thing is double check the slow machine and make sure it's in a good slot with enough lanes. There are some really bad OEM MOBOs that cut lanes significantly just to provide slots. I've come across a few dropping to x1 electrical but an x4/x8 physical.
Click to expand...
my gaming rig is all pci-e x16 so good there...
my plex box only has 2 pci-e slots... x16 and x8... I will swap video to x8 and nic to x16

I am going to also do the linux live boot test first...
 
T

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
I tried standardizing on Intel x520-da2's but I know on win10 and server, I got a lot faster speeds with mellanox 1 and 2...
so I am going to pickup 2x Mellanox ConnectX-3 EN's...
 
T

Tech Junky

Active Member
Oct 26, 2023
351
120
43
@TeleFragger

Before you go spending more $$$$ have you tried bypassing the switch using a fiber coupler and running P2P? If it's not the optics or the fiber it could be the switch causing the slower speeds. Could be a bad port or a backplane issue.
 
  • Like
Reactions: TeleFragger
You must log in or register to reply here.
Top Bottom