Bad read performance over 10Gbe

bmacklin

Member
Dec 10, 2013
96
5
8
I got two computers directly attached via 540-T2 10GBe cards.

One is a dual E5-2670 workstation running windows 10.
The other is the NAS with E3-1265v3 and 24 GB Ram.
The disks in the NAS are 4x4Tb WD, and 4x4Tb Seagate.

I have it configured as Raidz50 - striped raidz pool.

I am running samba server on linux version 4.1.17-Ubuntu.

When I do iPerf3 between the two machines, I saturate the line if I use 4 threads.

When I set up a ramdisk on the windows box and copy to and from the linux server I get the following:

10 GB file copy:
Server---->Workstation = 368Mb/s - this is capped. If I do two transfers at the same time, I get about 170Mb/s each.
Workstation---->Server = 1000Mb/s in the beginning... hovers around 700 to 800 after that.

So why am I getting such terrible read performance? Write, perhaps, but why read?

I found this discussion and some people said Samba is single threaded and performance is CPU dependent. So I have top running when I do the transfer and the cores are pretty unused during a read. There is far more load writing.

FreeNAS 9.3 (CIFS poor 10GBe performance) Looking for tweaks / hints / whatever


Anybody got any ideas? I would be okay if single thread caps at around 350Mb/s because that's what I saw running iPerf. However, I would expect that to go up to 700Mb/s once I get two copies going on at the same time.
 

coolrunnings82

Active Member
Mar 26, 2012
399
88
28
Maxing out at 3G instead of 10G unless I use 6+ threads. No problem saturating with 1 thread booting from a Linux LiveCD. Tried Server 2012 to Windows 10 or Server 2012 to Ubuntu and either gives similar results.