Synology Low Speed with 40Gbps NIC & Nvme SSD

blackeyed

New Member
Mar 29, 2020
3
0
1
Hello guys, sorry for my poor English.

I just built a server with Xpenology(synology) on C612 motherboard, 40G nic, 2 x samsung PM1725a 6.4T nvme pcie ssd(2 x basic no raid)
just in case some ppl don't know it yet: We now are able to use nvme ssd as storage in synology (by using mdadm in ssh)

I expect that I could get 3GB/s speed from this server, but I failed. It only runs at 1GB/s.
I first thought its NIC or Network problem, but when I try to copy from one ssd to another ssd internal (on the synology webui), it also around 1GB/s, so I think its a internal problem.

Does anyone know any clue for my problem? I checked CPU(E5 2650L V3) usage when copying files, its about 16%, but if copying file can only use one thread of CPU(not sure), then I guess its CPU bottleneck? Is there a way to check each thread usage?
 

i386

Well-Known Member
Mar 18, 2016
2,464
677
113
31
Germany
Have you run iperf/NTttcp to make sure that there is no bottleneck in your network?
Try a tool like robocopy which uses multiple "copy" process for transfering the files.
 

blackeyed

New Member
Mar 29, 2020
3
0
1
Have you run iperf/NTttcp to make sure that there is no bottleneck in your network?
Try a tool like robocopy which uses multiple "copy" process for transfering the files.
its not network problem, I tried copy files internal between two ssd, its still 1GB/s
 

blackeyed

New Member
Mar 29, 2020
3
0
1
after few testings, I figured something

1 nvme disk to another nvme disk copy speed is about 1.5G/s, its normal, the speed 3G/s is, I guess, disk to memory speed
2 40G nic, iperf test between win and linux/win, can't get full 40G speed report unless you open more iperf instance once
3 if using win for network speed test, use ntttcp, for 40G nic, it report 20G speed for 1 thread, and 40G speed for 2 threads
4 nvme raid0 is helpless compare to no raid, seq speed raise 15% but 4K speed drops

if somebody know how to achive full 40g speed for one thread, please let me know
 

TLN

Active Member
Feb 26, 2016
425
56
28
31
I assume you're using Add-in cards, right? I was playing with two of those in single system (C612 Also, Dual Xeon 2683v3). I tried to run mdadm raid0 and reached 9500MBps seq read.
Just tested using crystaldisk mark and getting 5600MBPS seq read off single drive. Interestingly, that similar drive show worse performance when using multiple namespace.
I'm not sure if you can get 40G off single thread. You'll need to do lots of OS tuning to get it though.

I can run some tests (in fact, I have xrenology VM) if you want. I don't have 40g nic available though. Is it possible to tune vmware nic to 40G?