Tips for SMB 3.0 Multichannel Performance Improvements

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

MikhailCompo

Member
Feb 14, 2017
41
0
6
44
I have two servers, each has 4 NIC ports, and dual CPU 12 core. On each 2 ports are intel I350-T2 and the other are some fujitsu crap.

I have not managed above 220 MB/s transfer rates, although i have managed to see traffic on all 4 ports, so SMB is working.

Can anyone suggest things i can try to improve the performance?
 

i386

Well-Known Member
Mar 18, 2016
4,263
1,558
113
34
Germany
Can you post the output of "get-SMBClientConfiguration"?


Other question: do you need so many interfaces? (If I need more than 1gbe bandwidth on a workstation or server then I use cheap mellanox 10gbe cards.)
 

cesmith9999

Well-Known Member
Mar 26, 2013
1,422
478
83
It seems to work better if there are different subnets. (this adds load to your routers though).

Make sure that your RSS settings are optimal for the CPU the cards are connected to. Assuming you have a multi-proc server.

Chris
 

MikhailCompo

Member
Feb 14, 2017
41
0
6
44
Can you post the output of "get-SMBClientConfiguration"?

Other question: do you need so many interfaces? (If I need more than 1gbe bandwidth on a workstation or server then I use cheap mellanox 10gbe cards.)
EnableMultiChannel is true. Are there any other parameters you want to know specifically? [I cannot cut and paste between those machines and internet connected, so i need to type manually.]

This isnt my kit so i cant put other nics in, this is a test with existing kit to improve bandwidth. We've acheived that, but not the theoretical max (and max reported by others, Youtube vids etc.) and want to examine why.

It seems to work better if there are different subnets. (this adds load to your routers though).

Make sure that your RSS settings are optimal for the CPU the cards are connected to. Assuming you have a multi-proc server.

Chris
For different subnets do you mean just setting different IP addresses in the TCPIP settings for each adapter, or settings the ports to different VLANs?

2 CPUs each with 12 cores, but NIC adapter settings increase the queues in multiples, so i have them set to 8 queues (the max setting).

Changing adapter settings increase from about 200 to 220 mb/s but it maxes out at that level. Even tried maxing CPU's (using 7-zip on all cores) on each server (points A and B in the transfer) which doesnt slow down the speed at all, so this is not a cap due to available CPU.
 

cesmith9999

Well-Known Member
Mar 26, 2013
1,422
478
83
the RSS settings are the number of physical cores. Not logical cores

if you have different subnets, those usually require a router to know the different subnets and route appropriately. does not sound like you could do that in this instance.

what happens when you robocopy with the /mt:[2x the number of logical cores]?

Chris
 

Cutha

Member
Sep 24, 2016
75
7
8
50
Canada
if you have different subnets, those usually require a router to know the different subnets and route appropriately. does not sound like you could do that in this instance.
I think for SMB multichannel you don't want a route between the separate subnets.

If for example your normal network is 192.168.0.1/24 and one of each NIC on the server is in that subnet you would then take another NIC in each server and address them in the 192.168.1.1/24 subnet. On the second NIC leave the gateway and DNS entries empty and in Advanced->DNS uncheck "Register this connection's addresses in DNS" to prevent hair pulling.

Below are some notes to self I have for SMB setup that you may find helpful:


DNS issue, a non issue if setup "correctly". The issue is multiple entries in DNS and its possible a client will get a name/IP that it can't access. Solution is that NIC's that you don't want this to happen with, uncheck

Server1:NIC1:1Gbps:192.168.0.1/24
Server1:NIC2:10Gbps:192.168.1.1/24

Server2:NIC1:1Gbps:192.168.0.2/24
Server2:NIC2:10Gbps:192.168.1.2/24

If there are 2 adapters as above, if Server1 does copy from 192.168.0.2, SMB Multichannel will actually move the data over the 10Gbps (1.2/1.1) even without DNS entry for 192.168.1.x entries.

Scale-Out File Servers – DNS Settings

SMB 3.0 Multichannel Auto Configuration In Action With RDMA / SMB Direct - Working Hard In IT