Recommended NAS for 800MB/Sec Sequential on 10Gig

Ryan McClelland

New Member
Oct 16, 2018
2
0
1
Hello,

Does anyone have a NAS they can recommend that can do 800MB/Sec sequential throughput with spinning disk over 10Gig ethernet? Was looking at the 12 bay systems from Synology, but it looks like the tests they publish are all done with SSD.

This would be the central storage system for an HPC cluster that would read off chunks of data via NFS, process it, then write back the results.

Thank you.

-Ryan
 

BlueFox

Well-Known Member
Oct 26, 2015
1,275
679
113
You should easily be able to achieve that with hard drives too on a Synology NAS. Modern hard drives are at ~200mb/s sequential, so it's not exactly tough to achieve.
 

kapone

Well-Known Member
May 23, 2015
1,032
595
113
One windows server + one raid card + drives = problem solved? or..
One _Fill in OS_ + one RAID/HBA + your choice of hardware/software RAID + drives = problem solved?

The phrase "NAS" is overrated. It's a file server...well..to be precise...a file server with a network card...but..you get my drift. :)

That said...I'm oversimplifying things. Is your data load:
- few big files
- lots of small files
- 800MB/s to each client or...total throughput??
- Redundancy/HA/backup?
- etc etc.

You need to put atleast some thought towards these things.
 

Ryan McClelland

New Member
Oct 16, 2018
2
0
1
It would be few big files. Looking for ~800MB/Sec total throughput to the NAS.

Redundancy/HA/Backup aren't necessary in this application, as they would just reload their data sets and re-crunch the data; or so I'm told.

So with something like the RS3617RPxs (RackStation RS3617RPxs | Synology Inc.) loaded with 12 10TB "enterprise" SATA drives in raid6 (shr2) and a couple of LACP'd 10Gig ports, you think ~800MB/sec is achievable?
 

gea

Well-Known Member
Dec 31, 2010
2,655
909
113
DE
A disk can have a sequential read/write performance of up to 250MB/s. But this can be achieved only when a single user reads/writes a large file like a video to the outer tracks of an empty disk. Real world performance even if they are from larger files are a mix of random and sequential loads so you should count 100-150 MB/s per disk. This means that you need around 6-8 disks in a Raid-0 setup to achieve a real world 800 MB/s.

If you do not care about iops, you can add 2 disks for a raid-6 or raid-Z2 config. If you expect concurrent read/writes you must care about iops so a setup from 12-16 disks in a multiple raid-10 would be the best option. Count around 100 iops from a Raid-6/Z2 (like a single disk as every disk must be positioned in every io). A pool from 6 x mirror would give you around 600 write iops and 1200 read iops in case of ZFS.

Next question is data security in general and in case of a crash during a write. In such a case the content of the write cache is lost. Without a powerloss protection (hardware raid-6 with battery/flash protection or ZFS with a ZIL/Slog) even the raid itself or the filesystem is in danger. If you care about it is very hard to achieve 800 MB)s writes.

What I would use
A fileserver with fast and modern CPU, at least 32 GB RAM, 12-16 disks (HGST HE, SAS) in a multiple Raid-10, a high quality nic (Chelsea or Intel), no hardware raid and no expander but LSI HBAs like 9305-16i with 16 ports or LSI 9300 HBAs with enough ports.

Use software raid, best regarding data security and performance due its advanced rambased read/write cachings is ZFS. To secure writes, you must enable sync write. ZFS with an Optane up from 800P can achieve up to 600-800 MB/s sync write.

Fastest ZFS server OS regarding ZFS, NFS and SMB is Oracle Solaris with a genuine ZFS but this is not free (around 800 USD per year). Next fastest are the free Solaris forks around Illumos, example OmniOS. Next are Free-BSD based solutions. Linux LVM raid over btrfs like Synology is using is far below ZFS in nearly all aspects that affects data security or performance.

Nic aggregation is not needed as a single 10G nic can give more than 1 GB/s. You may enable Jumbo Frames but you this can give more performance but can also give troubles.

see my benchmarks
https://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf

A good professional class server would be an
https://www.supermicro.nl/products/system/2U/5029/SSG-5029P-E1CTR12L.cfm

more hardware suggestions
https://napp-it.org/doc/downloads/napp-it_build_examples.pdf
 
Last edited:

maze

Active Member
Apr 27, 2013
565
84
28
I run the RS3617xs with a 10g nic setup. Put at around 1100MB/S. Did so from a 4 samsung evo drive raid0 to 12x ironwolf drives in raid0 (just to test).
 

DedoBOT

Member
Dec 24, 2018
44
13
8
Raid 10 from 8 HDDs @250MB/s must be enough for single client/file. Usually , the NAS performance's headache starts with the network and transport protocols . In a scenario single client working with single file the LAG barely helps .