Poor network speed on WS2016

[Markus1989]

hello,

my server has very poor networking speed when copieng files from a SMB share on the server to a client. The operating system is a freshly installed Windows Server 2016. The server uses a I350 T4 NIC and has two I210/I217 LM NICs onboard. The client runs on Windows 10 and has 2 I210 NICs as well. IP adresses were given manually.

Today I tested many setups including NIC teaming and unteamed. The server is about 4 years old and under Windows Server 2012R2 NIC teaming delivered stable and high speeds. But with Windows Server 2016 speeds went down from 200Mb/s to 20Mb/s max.

I think WS2016 is causing the problems since its cutting features of the intel driver. Especially with NIC teaming which is now down in the ServerManager (Software Teaming?). Can someone help on this?

 

[Nizmo]

Generally speaking the speed of SMB share's over a network is disk dependent more than your NIC's. Depending on the disk configuration, you would see slow copy speeds. Copy paste I assume, I think cut/paste is better for a quick speed idea imo.

 

[Markus1989]

Generally speaking the speed of SMB share's over a network is disk dependent more than your NIC's. Depending on the disk configuration, you would see slow copy speeds. Copy paste I assume, I think cut/paste is better for a quick speed idea imo.

Ok. Both directorys are lying on a SSD. When I use just one NIC on the server copying runs with 118 Mb/s

 

[j_h_o]

Run iperf between the 2 machines and see what speed you get.

iPerf - Download iPerf3 and original iPerf pre-compiled binaries

 

[Markus1989]

Run iperf between the 2 machines and see what speed you get.

iPerf - Download iPerf3 and original iPerf pre-compiled binaries

I tested it and it ran with 110 Mb/s with 2 NICs on bot sides. Than i copied a file and again speeds dropped. In the PowerShell i frequently ran the command "Get-SmbMultiChannelConnection" and I noticed that when there are two connections on the client side RSS gets disabled. As far as I understand it RSS is needed for balancing CPU load.

I restarted the client and started copying a file from the server and suddenly the speed decreased. On the client are 16Gb of RAM and about 3Gb were used while copying. In the TaskManager I can see no change in RAM consumption which tells me, that there was almost not buffering to the RAM.

 

[cesmith9999]

Server 2016 Teaming now delivers performance with hardware acceleration.

Windows Server 2016 Feature: A Look at Switch Embedded Teaming

Chris

 

[Markus1989]

I did further testing on that. As soon as there is a second path between both computers transfer speeds break down. I doensent matter if there is no NIC teaming, any kind of NIC teaming (switch configured accordingly) or even if the Intel drivers are installed . As I mentioned in the beginning the setup's hardware was capable of transferring 200MB/s.

 

[cesmith9999]

You should not need to do NIC teaming... as long as you have multiple sources you will get SMB Multi channel.

are you using different subnets or a single subnet?

I have over 1000 servers running multiple 40 GB links and all of them are setup with SMB Multi-Channel.

and most of them (80%) are running 2016

Chris

 

[Markus1989]

I don't doubt your knowledge or your work on the systems. I did some further investigation and installed WS2012R2 and the system worked fine. I teamed 2 NICs and got my 110MB/s. The client is running on Windows 10 1803 and NIC teaming isn't availabe here (Intel might fix ist the next weeks, fingers crossed). Back to WS2016 i "discovered" that my server should only habe one IP adress. So I teamed all 4 ports of the I350 T4. From the server to the client I get about 50MB/s and from client to server 110MB/s. Again I monitored the multi channel status via "Get-SmbMultiChannelConnection". All NIC's IP adresses are in the same net/subnet.

As you mentioned using all NICs unteamed would result in a server with multiple IP-adresses which is needed vor SMB multi channel - at least from what I understood. I can't help, but my system seems to get confused with multiple connections /IPs.

EDIT:

SMB-Status, NO NICs, all static IPs

 

[Markus1989]

I am very sure, that my problem is caused by the number of NICs and/or SMB protocol. I tested all kinds of setups and it showed, that whenever there are multiple sending NICs (teamed/unteamed) on the server transfer speeds show a significant drop - removing all connections but one results in normal speeds. Both, WS2016 and Windows 10 are updated as well as the NIC drivers. Unfortunately I can't compare my results to other hardware because all my computers use either the I210/I211 or the I350

 

[Stefan75]

Here's the way I set up SMB multichannel...

Windows 2012 R2
- 1 main BCM5709 NIC in subnet A with gateway
- 1 secondary BCM5709 NIC in subnet B without gateway

Windows 10
- 1 main Intel PCIx NIC in subnet A with gateway
- 1 secondary onboard Realtek NIC in subnet B without gateway

The ~200MByte/s (SSD to SSD) worked out of the box, no special config needed.
Didn't try it with 2016 server yet.