10g Home NAS Setup

Sleuthman

New Member
Sep 14, 2020
6
0
1
Hi all, I'm new here and have searched to try to find what I'm after with no avail, and you lot seem to know what you're on about!

I would like to see if you can find an issue with my proposed setup, I work in 3D using a lot of data and am moving into a new home in a few weeks, a perfect time to upgrade my ageing synology 1 bay nas drive which is practically full now, and slow.

I would like to be able to work off the NAS using a 10g connection, but I am not very knowledgeable with networks, so what I am thinking about is:

Build a cheap AMD system with a 10g add in-card, comes to about £380, buy a Netgear GS110MX switch that has 2 10g ports, plug my pc into one and the nas into another. I will have a 4tb WD Red SSD in there, that backs up to a 4TB HDD overnight, and/or an external off site internet backup thing like Polar Backup for cold storage. I would also like a couple of IP cameras , maybe a wifi extender, ps5/xbox etc..

What do you guys think, will this work as a 10g lan? I was also curious about managed and unmanaged switches, will I need a managed one to avoid my router dumbing the network down to gigabit speed, unsure of how this works with a mix of 10g and 1g devices and routers, i'm in the UK btw.

Thanks
 

cw823

Active Member
Jan 14, 2014
276
82
28
you don't need a 10gb switch, just run a cable between your PC and NAS on a different subnet.
 

Sleuthman

New Member
Sep 14, 2020
6
0
1
you don't need a 10gb switch, just run a cable between your PC and NAS on a different subnet.
That's interesting, could you break that down for me in simpleton's terms?

So would I have 2 ethernet's plugged into each pc, one going to the router, one going to the nas?

So the other devices would still be able to access the NAS, just at 1gb speeds then?

Cheers
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,289
442
83
As an example:
Server A has a 1Gb interface on 10.0.0.1. This interface holds the default gateway.
Server A has a 10Gb interface on 192.168.0.1.

Server B has a 1Gb interface on 10.0.0.2. This interface holds the default gateway.
Server B has a 10Gb interface on 192.168.0.2.

You set up custom routes and/or hosts file entries on each of these servers so that when A tries to talk to B or vice versa, it uses the 192.168.0.x network and thus goes over the 10Gb interface. Whenever they want to talk to anything that isn't A or B, they'll go out via the default 10.0.0.x network that your router is on.

Whilst technically possible, personally I'd avoid this approach if you can afford a 10G switch - messing about with multiple routes is never fun IMHO and can cause "interesting" problems. I'm assuming here that as a simple unmanaged switch the GS110MX can have any of its 1Gb ports act as an uplink to your router.
 
  • Like
Reactions: MBastian

Sleuthman

New Member
Sep 14, 2020
6
0
1
All I can find is that it says

-2 x 10 Gigabit Uplinks
-No Network Bottlenecks thanks to the 2 10-Gigabit/Multi-Gigabit Uplinks
-The two uplink ports automatically detects which speed is needed by the connected device and provides the adequate speed"

But I don't know if that means the router can go into say port 1, and it all still works as above?
 

ttabbal

Active Member
Mar 10, 2016
758
203
43
43
It does. Switches will adjust the speed for each port to what it can handle. That, combined with flow control at the software level, means you will get the best speed that the two ports can provide. So 10g->1g will get 1g speeds, while 10g->10g will get 10g. More importantly for me, the server can send multiple 1g streams simultaneously. You don't need a managed switch for this. Those are more for features like VLANs.

Note, these are low level network speeds. How fast file servers and such work depends on other factors like the speed of the drive array and issues with the software. Sustaining 10g file access is difficult, even for SSD setups. Many work well until they hit the end of their cache, then they fall down. This is one reason enterprise drives cost more. Test with iperf or similar to make sure the network is working well, before testing with file copy speed as it will help you see what level the bottlenecks are at.

With remotely modern gear, you don't need to fuss with jumbo frames / MTU, and it can cause other problems that can be tricky to diagnose.

As for a separate point to point network... well, it works and I've done it. But a switch is really the way to go if you can. It can also create tricky problems, though most software is a bit better about it these days. The rest of your network can also benefit by having the server have enough bandwidth that it's not easily capped out by one client.
 

acquacow

Well-Known Member
Feb 15, 2017
595
311
63
39
I ran point to point with a 2nd 10-gig nic for many years, there are some pains with it, but they can be resolved via custom windows firewall rules to block file sharing protocol broadcasts on your gig-e network. The problem with that is if you want to connect to any if your windows PC shares from something else on wifi or your gig-e network.

And if you block the packets at your 10gig nas, then other things on your gig-e network can't access it...

You have to write some very complex rules for each use-case to make it all work correctly.

I eventually just bought a 10gige switch and haven't looked back.
 

BlueFox

Well-Known Member
Oct 26, 2015
1,105
538
113
I ran point to point with a 2nd 10-gig nic for many years, there are some pains with it, but they can be resolved via custom windows firewall rules to block file sharing protocol broadcasts on your gig-e network. The problem with that is if you want to connect to any if your windows PC shares from something else on wifi or your gig-e network.

And if you block the packets at your 10gig nas, then other things on your gig-e network can't access it...

You have to write some very complex rules for each use-case to make it all work correctly.

I eventually just bought a 10gige switch and haven't looked back.
Did you try running everything on the same network/subnet? If so, no wonder you had issues. The point-to-point connection needs to be on it's own subnet. No firewall rules are required. It's very straightforward and takes no effort to configure.
 

BoredSysadmin

Active Member
Mar 2, 2019
468
129
43
I think the networking question was mostly answered (Instead of a new cheap 10gig switch, you may want to check eBay for used Brocade/Arista switches. I got my ICX6610 very cheaply if you can put it in the basement or some soundproof room :)
Another question is your new NAS specs - Did you already figured it out what exactly you need - high throughput or low latency?

At the bare minimum, I'd suggest 4 disks Raid 10 and NVMe SSD caching, like QNAP tvs-472xt
 
Last edited:

Sleuthman

New Member
Sep 14, 2020
6
0
1
Ok, thanks guys

When it comes to cache in the above propsed setup how do I increase the cache?

The specs I had planned out were:

AMD 3200g
ASRock B550M
8GB Corsair 2x4gb DDR4
ASUS XG-C100C Network Add-in-Card
256GB Intel SSD 760p Series
300W Be Quiet! PURE POWER 11, 80 PLUS Bronze
4TB Western Digital Red SSD

Basically the cheapest branded stuff I could find, let me know if you see any issues.

I don't necessarily want to saturate the 10g at this point, not really sure I can afford multiple SSDs for raid in the size I need, so would a WD Red sata 3 ssd be ok?

Cheers
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,289
442
83
Personally, if I was buying a ~4TB I'd go for an enterprise-lite Intel D3-S4510 or Micron 5300 Pro rather than a desktop-esque Red. They're not any faster in absolute terms than any other SATA SSD (although they will almost certainly be more expensive), but they should have far superior steady-state performance and endurance (in layman's terms, much better for server use but the specifics depend very much on your use-case).

10G can be hard to saturate over a single connection, especially with a single client - even with SSDs. For example, windows explorer will often run out of CPU well before it hits the limits of 10GbE. There's a RAID10 of six SAS SSDs at work that'll easily fill >2GB/s locally, but most of the windows workstations that talk to it can't push more than 300MB/s due to them only using explorer to access it. But again this depends on use-case.

What OS are you planning to run on this box? You're on a forum where many people, myself included, are going to tell you to make sure you get a CPU + mobo that has IPMI and ECC support as well ;)
 

Sleuthman

New Member
Sep 14, 2020
6
0
1
Yeah like I said I don't need to saturate 10g, but just better than an HDD speed of ~100MB/s was the hope

The kind of thing I'll be doing is running 3D scenes, saving gigs of data every few minutes, accessing lots of large image files and PSDs and autosaving them every few minutes, all of this at the same time, would I run into issues with my proposed setup? I really don't want to spend more than £1,000 on everything unless I have to.

I haven't really thought about the OS, what would you suggest, I'd like simple and effective, if that exists?? :)
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,289
442
83
"Simple and effective" totally exists... but with as with everything else on this forum, the straight answer is "it depends" :)

That isn't a bad thing - there's use-cases for hundreds of different scenarios - but it'll need some input from you on regards to details, especially with regards to prior experience. Personally I'm a debian linux dude and have built and set up all my home servers myself, but other people prefer the "do all the details stuff for me please" distros like FreeNAS preferable. Others might prefer windows, some other flavour of linux or BSDs, or run the whole thing as a hypervisor or <descend in to decision paralysis>

If you're chucking in a 10GbE card and an SSD in, and you're not too concerned about e-peen maxing out the bandwidth, any combination of OSes should be able to get you 250MB/s over the wire (and IMHO the Ryzen chips in combination with a board - like the ASRock ones that support ECC - make a great budget home server; I use a 3700X on an X470D4U myself) but the OS and the stuff it can/can't support well should also be taken in to consideration before you start spending moolah.
 

BlueFox

Well-Known Member
Oct 26, 2015
1,105
538
113
Why do you need this data on a NAS anyway? You will get considerably better performance if you're working with local disks. You can back it up to the NAS as required.
 

acquacow

Well-Known Member
Feb 15, 2017
595
311
63
39
Did you try running everything on the same network/subnet? If so, no wonder you had issues. The point-to-point connection needs to be on it's own subnet. No firewall rules are required. It's very straightforward and takes no effort to configure.
I did both, there were drawbacks to each method. One was that FreeNAS won't really let you put two subnets on a single adapter... I didn't have another adapter at the time, so that was a PITA to deal with. I tried to mod the configs to alias an adapter/etc, but it didn't like that either.
 

mervincm

Active Member
Jun 18, 2014
152
38
28
For folks that want simple, a switch makes everything much simpler. if you don't need ful 10 gig then 2.5gig cards are cheap and the 99$ 5 port qnap switch makes everything fast and easy.

 

mervincm

Active Member
Jun 18, 2014
152
38
28
Also a good home free NAS OS is openmediavault. it read/writes 400-500MB/sec with a 10gig network, to a 4 DISK RAID 5, and my windows 10 desktop. To / from the NVME drive in the NAS I can get 1GB/sec (all on large files)
 

Sleuthman

New Member
Sep 14, 2020
6
0
1
Why do you need this data on a NAS anyway? You will get considerably better performance if you're working with local disks. You can back it up to the NAS as required.
Yeah I keep coming back to this point. What software would you use to manage backups? I'm not one to leave my computer on all the time, and wouldn't want to be overwriting data that hasn't changed...
 

BlueFox

Well-Known Member
Oct 26, 2015
1,105
538
113
I did both, there were drawbacks to each method. One was that FreeNAS won't really let you put two subnets on a single adapter... I didn't have another adapter at the time, so that was a PITA to deal with. I tried to mod the configs to alias an adapter/etc, but it didn't like that either.
You shouldn't be putting two subnets on a NIC used for a point-to-point connection. I think you may have a misunderstanding about how TCP/IP works as there is really only one correct way of setting up a point-to-point connection (or any isolated network with no routing) and there are zero drawbacks to it.
Yeah I keep coming back to this point. What software would you use to manage backups? I'm not one to leave my computer on all the time, and wouldn't want to be overwriting data that hasn't changed...
Doing incremental vs regular full backups is pretty common. This is what I use: Free Windows Backup Solution for PCs and Endpoints
 

EngChiSTH

Member
Jun 27, 2018
50
18
8
Chicago
Yeah I keep coming back to this point. What software would you use to manage backups? I'm not one to leave my computer on all the time, and wouldn't want to be overwriting data that hasn't changed...
I use Veeam software, the free edition version for all of my backups. Also creates 'usb key' for whatever you are backing up so you can do full restore if you lose entire system (MB fries ,etc). the software will automatically manage the basic things - backing up changes, etc. i think it can also turn off your PC after the backup job run if that is what you want.