help re-organizing homelab after long tinkering stint

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

digity

Member
Jun 3, 2017
53
1
8
54
I've been tinkering, on a budget, with various technologies over the last year (mainly FreeNAS, Docker and pfSense) and I just looked up and realized how messy my home network is now. My home network also serves as my home lab and network for my small IT support business so that adds to the mess (network is flat/basic single subnet, no VLAN). I've got techie's block right now... how would you re-organize this mess of a network? Any improvements that can be had very cheaply?

  • NAS - Synology DS415+, 16.3 TB (4 x 6 TB, Intel ATOM C2538, 2 GB)
    • stores Plex media, ESXi datastore (NFS), some backup and archive (full metal and file & folder)
  • NAS - FreeNAS 11.0 U2, 9 TB (5 x 2x2 TB mirrored pool, 1 x X5650 6c/12t, 24 GB, Dual CPU capable mobo, 16 x HDD capable ATX case)
    • mainly built to store CrashPlan backups for home, family and business. has backup and archive too (full metal and file & folder)
  • VM host server - ESXi 6.5 (X5550 4c/8t, 64 GB, Dell Precision T7500, Dual CPU capable mobo)
    • VM (Ubuntu) for Docker host for Plex (and related apps), CrashPlan and ThousandEyes
    • Business Windows Server VM - line of business, QuickBooks
    • Business Windows client (Win 10) VM
    • Personal/home Windows client (Win 10) VM
  • VM host server - ESXi 6.5 (Core i5 2400, 4c/4t, 14 GB, Dell OptiPlex 790)
    • only set up because VT-d/DirectPath I/O kept crashing the above ESXi build
    • Windows 10 VM for VT-d use with tuner (for recording HDTV OTA) and NVR software (Blue Iris), which won't perform well anyway (Intel's Quick Sync doesn't pass through to VM apparently)
  • Firewall/Router - pfSense (AMD Athlon 64 X2 4400+, 2c/2t, 2 GB, dual NIC PCI-e)
  • Docker host - Raspberry Pi 3
    • Docker for Uni-Fi controller (Ubiquiti access points)
      • This container/service is isolated because I read somewhere that "edge" services should be hosted separately from other services to avoid a single point of failure in your network...???
  • 16 port smart switch (VLAN capable)

Separation between personal & business and consolidation would be nice, but any kind of help would be greatly appreciated.


P.S. - I do have spare mobos, builds, laptops and parts lying about, but unfortunately they're as old as the components listed above (if not older).
 

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
For really cheap I would probably combine the FreeNAS and Xeon ESXi server (if you can get VT-d working for the HBA) and slap in a pair of L5640's and combine your ram. The pfSense box depending on your throughput needs you may be able to get away with something much lower powered and save yourself some electricity. Oh and I would probably run that Windows NVR on bare metal to take advantage of hardware decode. You can always use hyper-v/VMware Workstation/Virtual Box if you need something else on it.

If you are willing to spend more then the 2011 v2 Xeon cpus look good right now and you can reuse your DDR3.
 
  • Like
Reactions: MiniKnight

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
I don't want to sound discouraging, but you're not at a point where its TOO big. It's also too small to do OpenStack.

On the VLANs it sounds like you want everything to talk to everything else. If that's the case, you're flat network makes some sense. Perhaps take a few baby steps. Add a VLAN for storage, then reconfigure everything to use that VLAN. That'll teach you what you need and get you going.
 

digity

Member
Jun 3, 2017
53
1
8
54
For really cheap I would probably combine the FreeNAS and Xeon ESXi server (if you can get VT-d working for the HBA) and slap in a pair of L5640's and combine your ram. The pfSense box depending on your throughput needs you may be able to get away with something much lower powered and save yourself some electricity. Oh and I would probably run that Windows NVR on bare metal to take advantage of hardware decode. You can always use hyper-v/VMware Workstation/Virtual Box if you need something else on it.

If you are willing to spend more then the 2011 v2 Xeon cpus look good right now and you can reuse your DDR3.
  1. I thought about combining them too, but I thought it was best practice to avoid having a single point of failure...?
  2. What type of devices, capable of at least dual NICs, show I be on the lookout for that uses much less power?
  3. My current new-to-me X5500 and X5600 series CPUs won't be supported in the next ESXi release, so this f*ck up, on my behalf, made me reluctant to upgrade to the only 2 generations newer 2011 v2's as I don't want to be down ish creek again... unless I'm being paranoid and the 2011 v2's will remain supported for at least another 4 years...?

I don't want to sound discouraging, but you're not at a point where its TOO big. It's also too small to do OpenStack.

On the VLANs it sounds like you want everything to talk to everything else. If that's the case, you're flat network makes some sense. Perhaps take a few baby steps. Add a VLAN for storage, then reconfigure everything to use that VLAN. That'll teach you what you need and get you going.
Is the benefit of putting storage on it's own VLAN simply to reduce subnet noise/talking and increase transfer speeds? Or am I getting that confused with network cameras and voice VLANs?
 

Unfadingpyro

New Member
Sep 17, 2016
18
12
3
30
Is the benefit of putting storage on it's own VLAN simply to reduce subnet noise/talking and increase transfer speeds? Or am I getting that confused with network cameras and voice VLANs?
Same thing applies to storage. Especially backup traffic as that can have a lot impact on network speed.

You could also separate out Wifi on a VLAN and use Pfsense to route/allow certain traffic to your servers if needed. (ie. Plex, etc)

I have an overkill home network with 4 or 5 VLANs. Not that i need them, but i like play around and test different things out. As it sits now i have a Guest Wifi VLAN, a Prod VLAN that all my servers go on, a MGMT vlan for Switches, APs, IPMI, etc, and Camera VLAN. Camera VLAN is locked down so it does not have internet access or access to any other VLAN. Just to my Blue Iris server.
 

digity

Member
Jun 3, 2017
53
1
8
54
Same thing applies to storage. Especially backup traffic as that can have a lot impact on network speed.

You could also separate out Wifi on a VLAN and use Pfsense to route/allow certain traffic to your servers if needed. (ie. Plex, etc)

I have an overkill home network with 4 or 5 VLANs. Not that i need them, but i like play around and test different things out. As it sits now i have a Guest Wifi VLAN, a Prod VLAN that all my servers go on, a MGMT vlan for Switches, APs, IPMI, etc, and Camera VLAN. Camera VLAN is locked down so it does not have internet access or access to any other VLAN. Just to my Blue Iris server.
This is exactly my setup (pfsense, plex, blue iris, etc.)... can you quickly jot down how you configured your pfsense and major networking devices to achieve this?

Sent from my SM-G955U using Tapatalk
 

Diavuno

Active Member
This is exactly my setup (pfsense, plex, blue iris, etc.)... can you quickly jot down how you configured your pfsense and major networking devices to achieve this?

Sent from my SM-G955U using Tapatalk
Digity, you seem to have the same issue I have/had, Only I moved from ESXi 5.5 to 2012 hyper V (now 2016)
also my ubnt stack is 3x windows 10 VMs (Unifi/NVR/UNMS+Aircontrol)

I was on a big flat network. Then I decided to add a single AP for guest WiFi...
Then I watched a friend (keep in mind I work from home) access a shady looking website trying to stream a ball game.

I then split my network up with PFSense and untagged port based VLANS.
I upgraded my PFSense from an old opteron to pair of A1SRi-C2758 based superservers.
During the upgrade to the C2758's I setup HA and did real VLANs...

Now I have a few VLANs
Physical business hosts and storage network.
business VMs
External facing VMs
MY network (personal)
Guest stuff (includes my Echos and HTPC)

I know its way overboard, but its very clean.
I do have a lot of rules denying access between all networks
With some very specific exceptions:
HTPC is allowed to my personal network... but limited to the PLEX box.
Personal network also has a loopback rule to allow access to the external services, mostly for the ubnt stack)
Business machines can also access the printer in my personal vlan.

I have 2 of the 4 ports in PFSense LAGG to the switch (as it does all the routing between subnets)
the main switch is a 48 port Powerconnect, it handles all the servers and storage network, it also feeds a few smaller 8 ports around the house:
One in my office for the personal network.
Two I keep on guest for when I work on client stuff. (possibly infected)

I did all of that in early 2017, learned a ton about PFSense
 
  • Like
Reactions: StammesOpfer