Help with home 10GbE network (10Gbase-T and SFP+)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

ERDrPC

Member
Aug 14, 2012
36
3
8
I currently have a Mikrotik CCR1036 -8G-2S+ connected to FTTH 1 Gig internet thru the ethernet port. My main switch is a cisco SG350XG-48T and connected to one of the CCRs SFP+ ports. All ethernet cables are CAT7. I have 10Gbase-T cards in my windows computers. This switch connects to my Control4 home automation system, 3 x Ruckus R710 APs and my synology server and couple windows computers. The other CCR SFP+ port is connected to a SG500X-48P. This provides ports for my IP cameras and Hikvision DVR. Alot of my IOT such as doorbird video intercom and various boxes such as roku, apple TV, printers are connected to either one of the switches. There are no VLANs. CCR is running a custom build of Router OS. I have a second SG500X-48P for just-add-power video over IP. I would like to connect this via SFP+ once I figure out below.

I also have a server that connects thru a VPN to a different country and it connects directly to the CCR ethernet port. VPN is setup at the router level. See topology below. I keep traffic on this VPN isolated from my home network. The server itself has dual 1 GbE NICs. Currently if I have to move data off of the server onto my synology, I have use teamviewer or trek to the basement with a USB drive. I'm not a fan of teamviewer for this as the files are typically 30 - 50 Gb in size and my understanding is that the data leaves my intranet, goes over the TV servers and then back to my other TV computer. This takes a long time and half the time the file transfer fails. I did have both NICs setup where one was on the home network with intranet access only and the other to the VPN connection.(VPN NIC1 192.168.x.x range with subnet mask 255.255.0.0, and intranet NIC 2 on the 10.0.x.x range with subnet mask 255.255.0.0. and blank default gateway) However I was concerns that "someone" could gain access to my home network thru the VPN connection into the switch or server and learn my true IP location. Teamviewer has a intranet only mode so this was used to transfer files from the server by connecting to NIC 2. It was much faster.

Questions
1. Am I right to be concerned about the dual NIC and potentially exposing my IP address
2. I'm planning on upgrading my hosting server (currently a synology) to an unraid build (new box) and keep the function of the downloading function of the old server. I was thinking about using two Mellanox MCX311A-XCAT ConnectX-3 cards in a DAC to provide 10GbE speed for transfers. Would this open up my home network to IP address detection or put it at risk with network sniffing malware?
3. Is there a way to tunnel or static route the VPN data from a shared CCR SFP+ port to my download server if I decided to attach the download server via SFP+ to the spare port on the SG350XG or SG500X-48P? I would still need to be able to transfer files from the download server to the unraid server which would be connected by SFP+ to the other spare port on the SG350XG or SG500X-48P?? Or is it better to use a dual SFP+ card on the unraid - one connection to home network and other DAC to download server (separate IP addresses)? Note the cisco switches currently only function as basic switches and all of the routing is done thru the CCR (no cisco layer 3 functionality enabled). If the CCR had a third SFP+ port then this would be much easier I presume.
4. if the answer to 3 is no then what if I add the CRS326-24S+2Q+RM (24 SFP+ ports) above the SG350XG-48T. I could either bond both CCR SFP+ to the CRS for better bandwidth and I would have plenty of SFP+ ports. Would I still have the issue of the getting VPN connection out from the CCR as per question 3? I could use one CCR SFP+ port for my home network (would I lose out on bandwidth??) and the other SFP+ port dedicated to the VPN connection for the download server?

Thanks for digesting all of that and I look forward to responses

Network edit.png
 
Last edited:

ERDrPC

Member
Aug 14, 2012
36
3
8
anyone care to take a stab at this? Basically my VPN server is separated from the rest of my network. I need to move files off of it to my main storage server. What is the safest/fastest way to accomplish this? DAC connection between two SFP+ ports? Does this open the rest of my network up to detection? Teamviewer is too slow.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
This may not be the most elegant or secure solution, BUT What I would do If I wanted to do keep my download server of large amount LINUX ISOs 30 to 50gb each secure, I'd have a dual-homed NAS which would have one port in a regular network and one in "vpn" network.
connect the same NFS or SMB share to both your home client and your LINUX ISOs download server. For extra security, you could block internet access from the NAS. This way you won't be able to jump from download server (if compromised) to NAS.
 

ERDrPC

Member
Aug 14, 2012
36
3
8
This may not be the most elegant or secure solution, BUT What I would do If I wanted to do keep my download server of large amount LINUX ISOs 30 to 50gb each secure, I'd have a dual-homed NAS which would have one port in a regular network and one in "vpn" network.
connect the same NFS or SMB share to both your home client and your LINUX ISOs download server. For extra security, you could block internet access from the NAS. This way you won't be able to jump from download server (if compromised) to NAS.
So you are suggesting a dual card on the storage server. One on the home network and the other on the VPN network. This would allow the storage server access to home network (to allow roku et al access to plex media server) but also allow transfer of files from VPN to storage. I could setup the storage server connection to the VPN network as intranet only with a blank gateway.

What if I downloaded ransomware, or someone gained access to the VPN server. If they did a map of the network then they would find the storage server on a different subnet. Could they gain access? Could they then jump into the main network? The VPN and home networks are both protected by the Tik router firewall.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
a) don't download executables (including scripts) files from places you don't fully trust. This way you will unlikely to download malware.
b) have a firewall between vpn and storage server (or a software ACL rule) on a storage device to only allow smb or nfs traffic.
It's not 100% bulletproof, but these two, especially NFS would unlikely to compromise or discover the second IP. Again have storage device prohibit to access the internet at all for extra tin foil effect.
 

ERDrPC

Member
Aug 14, 2012
36
3
8
a) don't download executables (including scripts) files from places you don't fully trust. This way you will unlikely to download malware.
b) have a firewall between vpn and storage server (or a software ACL rule) on a storage device to only allow smb or nfs traffic.
It's not 100% bulletproof, but these two, especially NFS would unlikely to compromise or discover the second IP. Again have storage device prohibit to access the internet at all for extra tin foil effect.
I never DL exe files
I guess this would prevent me from using plex pass and watching my movies at my in-laws on their Roku?
 

ChuckMountain

Member
Nov 6, 2019
30
4
8
To clarify your VPN server is segregated from your network at the moment by virtue of firewall rules on the CCR? If the VPN was to drop what would happen?

If you have got dual nic why not plug the other into the CCR and set up some firewall rules that allow you to access a network share on the VPN server from a host on your internal network but not the other way round?
 

ERDrPC

Member
Aug 14, 2012
36
3
8
Yes - it is segregated via the CCR.

I'm looking to speed up transfers by using Mellanox SFP+ card on the storage server and VPN server. If I plug the other GbE port from the VPN server into the CCR then I'm stuck with GbE which defeats the purpose of this whole venture

I think what makes sense is to install a dual port SFP+ on the storage server. Connect one to the Cisco SG350 (for network and internet access) and the other to the single port card on the VPN server. This would be in DAC mode bypassing the network itself. I would need to restrict internet access on this connection. I need to be able to move files one way - from VPN to storage. Is there a way of restricting this in unraid (storage server OS) or windows 7 (VPN server OS)? To access the the VPN in headless mode - I guess I would still have to use teamviewer in normal mode (ie not LAN mode only as the firewall isolates it) to manage usenet and qbittorrent running on it?? I should be able to setup a share on the unraid to the VPN thru Krusader docker and this can initiate the file transfer.

Crazy idea - Would it make sense to combine the two boxes (just learning unraid and never used a VM before)? If I combined both boxes with a windows 10 VM, I could isolate a GbE ethernet connection just for the VM and this would connect to my VPN. The question becomes to use a physical network connection or virtual for the large file transfers between unraid array and the VM. VM would have an unassigned 2 TB SSD forwarded to it.
Physical connection thru to the VM - I understand this would mean the VM will communicate with the host as if it was a separate physical machine, going out the one NIC, down to router/switching infrastructure, and then back in. Or do I do it virtually - "When VMs utilize VirtIO, their is another distinct advantage in that networking between the host and guest can take place without traversing the copper wire. This allows for much faster throughput than the physical NIC hardware even supports at the port level. As an example, in mounting an SMB share to SSD-based cache pool from inside Windows VM, able to see IO throughput to the share exceed 250MB/s (that's megabytes, not bits)". I need to maintain anonymity with respect to the VPN connection and can't risk the download police tracing to get my true IP address and location. If the virtual connection puts me at risk for this then physical it is. If it is physical then I would need to add 1 dual port and 1 single port mellanox SFP+ cards to my server - dual port for unraid (one to switch and one to DAC with the other card) and one for VM (other end of the DAC). The VM would also need a second port for the ethernet connection to do the router VPN port. Virtual seems better but this is why I ask you guys
 

ChuckMountain

Member
Nov 6, 2019
30
4
8
I think you are approaching this from the wrong angle. The previous post seems a bit over-engineering and relying on some software functions to provide a firewall like service which is never quite as good.

For getting stuff on to your VPN box, your limit is your 1Gbps Internet connection and your VPN solution will be slowed, so if you can keep up with this then there is no real benefit of going faster and you have the equipment already in place. I would look at a solution that can see a share via the CCR to the VPN server from your storage server and run a sync on the folder(s). So when new files arrive they are automatically copied\moved across. This removes a need for TV (how do you know they don't monitor stuff)

If you wanted to up the bandwidth why not plug your middle switch into the top one and then use the "spare" SFP+ port on the CCR to connect to a new SFP+ card on your VPN.
 

ERDrPC

Member
Aug 14, 2012
36
3
8
Hey Chuck

Thanks for the ideas. I don't move the files over in real-time. I keep them on the VPN till I can ensure no malware/ viruses. So when I'm ready to move it, doing it at 10GbE would be much faster - usually 3 to 4 files of 50-80gb each. With direct attached cable I can move the files without the limitations for going thru switch/router.

If I went about it as per your suggestion - connect VPN to ccr sfp+ port. Connect sg500x to sg350 via sfp+ instead of ccr. Connect storage server to open sfp+ port on sg500x. Use something like rsync and firewall rules to move files from VPN to storage. Would still need Tv to connect and manage vpn server but not for file transfer. Maybe I could setup a teamviewer firewall rule to allow LAN only connection from my workstation?
 

ChuckMountain

Member
Nov 6, 2019
30
4
8
Is there any reason why you use TeamViewer and not something else?

Presumably, that route of control is out from your workstation via the Internet and back to your VPN server?

Could you not use Remote Desktop and have a rule on CCR that only allows traffic from your desktop to the RDP port on your VPN server?

Just seems hopping via the Net to be odd.

I assume your have firewall rules locking down the VPN server from the rest of the net as well as no VLANs etc. If the VPN connection failed what would the server see?
 

ERDrPC

Member
Aug 14, 2012
36
3
8
I use teamviewer for the ease. I tried setting up RDP before with a simple one - computer to another and never got it to work. Does it matter which version of Windows 7 my VPN server is running for RDP to work ?

If the VPN goes down then the server cannot access the internet and I cannot access the server. Knock-on-wood been up for 11 months straight.

The challenge will be setting up the firewall to allow the unraid access to the VPN via LAN without exposing it to the VPN traffic. Maybe it's easy....this is all new to me. If the firewall gets setup correctly then I should be able to use the unraid krusader docker to mount a share of the VPN data drive??

No VLANs. That's just another layer of complexity.
 

ChuckMountain

Member
Nov 6, 2019
30
4
8
I mainly use W10 and Server variants which are all nicely behaved with RDP.

Can the VPN be bound to one NIC and the other allow traffic to the LAN, which then you can block with FW rules.