Windows 10 to Windows Server 2019 Robocopy Speed

[zkrr01]

I have two Dell 8920 Workstations running Windows 10 Pro for Workstations. One system can be dual booted with Windows Server 2019.

Both systems have a separate G: drive consisting of a Samsung 970 Pro 954gb SSD for Hyper-V Linux Systems testing and backup.

I use robocopy to backup the SSD, using the command:

ROBOCOPY /MT:1 /MIR /R:0 /W:0 /LOG:G:\files.log /NP /NDL \\Robin\g\HyperV-Exported-Systems \\Eagle\g\HyperV-Exported-Systems

For several months I have been backing up the Hyper-V systems from the Robin system to the Eagle system and was getting an average speed of 3190 MB/s

Recently I started booting the Eagle system to running Windows Server 2019 and I noticed that I was getting an average speed of 2333 MB/s when I ran the identical backup. That's a big difference!

Confused over the speed difference, I ran CrystalDiskMark 6.0 and got almost identical results of 3500MB/s read and 2700MB/s write on the SSD on both systems.

So why do get 3190 MB/s when I backup to a Windows 10 system and only 2333 MB/s when going to a Windows Server 2019 system? I would have expected identical results, perhaps a little faster on the Windows Server 2019.

 

[Rand__]

Maybe the '19 box is configured differently (out of the box)?
Things coming to mind would be
RX queues of NIC, TCP Window Autoscaling, Error Checking, additional permission checking...

Or different (less optimized) drivers?

 

[i386]

Windows 10 (1809) to Server 2019 here without problems. I upgraded from Server 2016 inplace & reinstalled mellanox drivers.

Was your server 2019 a clean isntall or an upgrade?

 

[zkrr01]

The install was from a current .iso file and the install went without errors.

I do see something strange on the computer that can dual boot between Windows 10 and Windows Server 2019. When I boot that computer with Windows 10 everything works ok. It shows that there are three NICS, a 1Gb, 10Gb and 40Gb. The system selects the 40Gb NIC when performing the file transfer.

When I boot that computer with Windows 2019, it only sees 2 NICS, the 1Gb and the 40Gb. It cannot see the 10Gb NIC.

Then when I run the file transfer it selects the 1Gb NIC instead of the 40Gb NIC. That must be what is causing the slowdown.

When I checked the Device Manager the 10Gb NIC is missing! It has a new section called "Other Devices" which I suspect is garbage from the corrupted 10Gb NIC.

Any thoughts how to fix that?

 

[zkrr01]

I finally fixed the Windows Server 2019 corrupted 10Gb NIC issue, but now I have another problem. I am using ROBOCOPY to backup my data from one host to another using the following command:

ROBOCOPY /MT:1 /MIR /R:0 /W:0 /LOG:G:\files.log /NP /NDL \\Robin\g\HyperV-Exported-Systems \\Eagle\g\HyperV-Exported-Systems

Since I have three network adapter cards, and I cannot tell ROBOCOPY which NIC to use, the OS determined which NIC to use. When I copy from Windows 10 to Windows 10 the system selected the fastest NIC to use, which was the 40GbE NIC. However when I copy from Windows 10 to Windows Server 2019 it selects the 10GbE NIC, thus giving a much slower transfer rate. I suspect some internal rules were modified when the 10GbE NIC was corrupted.

I there any other way to force the systems to do the copy using the 40GbE NICS?

 

[cesmith9999]

Turn the other 2 off

Chris

 

[im2geek4you]

Try change the Interface Metric.

 

[Rand__]

Set vlans or distinct ip ranges and then use IP or name mapped to appropriate ip range

eg
192.168.1.1/2 servera_1g/serverb_1g + servera/server_b (move to default interface)
192.168.2.1/2 servera_10g/serverb_10g
192.168.3.1/2 servera_40g/serverb_40g

 

[zkrr01]

I tried changing the interface metric and it had no effect. Since ROBOCOPY does not take an IP address, how can I use vlans or distinct ip ranges? I don't want to turn off the NIC's because If I turn off the 1GbE, then I lose internet access. How did the Windows 10 to Windows 10 work? It seemed to know what to do without doing any of the others? It's only Windows Server 2019 that does not work like Windows 10 that is causing a problem.

 

[cesmith9999]

Yes it does

robocopy \\10.0.0.1\sharename \\10.0.0.2\sharename /e

Chris

 

[zkrr01]

Here is the log:

-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------

Started : Thursday, October 25, 2018 7:47:02 PM
Source - \\10.0.1.1\Robin\g\HyperV-Exported-Systems\
Dest - \\10.0.1.2\Eagle\g\HyperV-Exported-Systems\

Files : *.*

Options : *.* /NDL /S /E /DCOPYA /COPYAT /PURGE /MIR /NP /MT:1 /R:0 /W:0

------------------------------------------------------------------------------

2018/10/25 19:47:03 ERROR 67 (0x00000043) Accessing Source Directory \\10.0.1.1\Robin\g\HyperV-Exported-Systems\
The network name cannot be found.

 

[Rand__]

ROBOCOPY /MT:1 /MIR /R:0 /W:0 /LOG:G:\files.log /NP /NDL \\10.0.0.1\g\HyperV-Exported-Systems \\10.0.0.2\g\HyperV-Exported-Systems

 

[Evan]

You used to be able in Windows to set a priority of network interfaces in the properties , not on a machine to check if it’s still possible but I can’t imagine why not.

 

[zkrr01]

Now when I use:
ROBOCOPY /MT:1 /MIR /R:0 /W:0 /LOG:G:\files.log /NP /NDL \\10.0.0.1\g\HyperV-Exported-Systems \\10.0.0.2\g\HyperV-Exported-Systems

It works perfect on Windows 10 to Windows 10. However when I try it on Windows 10 to Windows Server 2019, it works but it runs extremely slow. I did some more searches on Windows Server 2019 and a couple of stories say Microsoft was having terrible results with 2019 and had recalled it. I will just run Windows 10 to Windows 10 until Microsoft gets all their problems resolved.

 

[PnoT]

What 10/40Gb NICs are you using and are they RDMA?

Have you look at which NICs are using SMB Multichannel? SMB will use any and all network adapters while making SMB transfers to other machines as long as those adapters can support it. What can happen is it could be using all of your adapters instead of the one you really want. You can use SmbMultichannelConstraint to remedy the issue.

 

[zkrr01]

I reloaded Windows 2019 recently after I heard it had be released again, but now I am unable to access the 2019 system from Windows 10. For some reason it is giving robocopy a access is denied. Once I figure out what is causing that I will see if the original problem is still there.

The 40Gb NIC does support RDMA. I doubt RDMA would be used by robocopy? What makes you think it would?

 

[cesmith9999]

RDMA is under the COPYAPI in the software stack that robocopy uses.

Because of this robocopy down not care if RDMA is turned on or not. it just works eitherway

Chris

 

[zkrr01]

How do I find out if RDMA is being used? Since I am using Windows 10 workstation to initiate the robocopy to copy files from Windows 10 workstation to Windows server 2019, would RDMA still be used?

 

[PnoT]

How do I find out if RDMA is being used? Since I am using Windows 10 workstation to initiate the robocopy to copy files from Windows 10 workstation to Windows server 2019, would RDMA still be used?

I should clarify that I don't think any of your issues have to do with RDMA was just curious as to whether you were using Mellanox cards.

Can you run an iperf test from Win10->2019 and then 2019->Win10 by using just the ip addresses on the 40GbE interfaces?

 

[cesmith9999]

Yes, if it is setup correctly on both machines.

Get-netadapterrdma

Chris