Earlier this year I bought some rural property that will become my home closer towards the end of the year. I'm upgrading from a small city center townhome on a postage stamp lot to a 35 acre ranch in the country with three different buildings I will be wiring up with networking, servers, and WiFi. I've got a long list of decisions to make about infrastructure and I'm just now diving into the deployment. I thought I should get in front of the work so I can get some feedback and critique about my plans while it's still easy to make changes. I hope this thread can generate some good discussion. I'm excited to set up a TIA-606 Class 3 HomeLab!
TL;DR I'm deploying a UniFi campus network in my new home, including buried fiber runs to four different locations on the property. This will be an upgrade/replacement to my existing home network which consists of a pair of Proxmox VE servers, a FreeNAS Mini XL+, and a variety of other sundry boxes and RPis. This thread will document my progress and any related shenanigans.
The Current Situation
My new home had an existing "server" closet which was home to a junky 2-post round-hole rack that held A/V gear (media room is on the other side of one wall) and an old-and-busted security camera system. I'm keeping the room, but throwing everything else away. There's also enough existing cat5 in the walls and attic to qualify the home as a copper mine. I've already ripped all of that out, as well as the shag carpet that was in the server closet. New tile is down, but still waiting for an electrician to come in and tidy everything up and then for the walls to be cleaned up and new drywall.
This is how things looked when I bought the place (the UDM Pro is mine):
The closet is empty now, all the cabling worth keeping has been pulled up through the walls into the attic, and I set up the ISP modem, UDM Pro, and an access point up there as a temporary measure while construction takes place in the closet and main house. Thankfully the attic is insulated and everything seems happy enough up there for the short term even in the crazy Texas heat.
Rough Plans (layer 2)
I've already bought a bunch of UniFi access points, switches, and the UDM Pro to host layer 2. I have no interest in anything UniFi beyond the Network app. No plans for UniFi security or access control or whatever other nonsense they're cooking up. I've been happy enough with the switching and very happy with their WiFi since switching from Ruckus a few years back. I will be doing 10gbit fiber from the main house to a workshop which is about 400 meters from the main house. I'll also be doing fiber from the main house to a powered front gate. There's an existing RF point-to-point network link from the workshop to a guest home which I plan to re-use if I can get it working. Previous owner was using it and claims they got "decent" bandwidth from it. They were not very technical, but did manage to install it themselves. The fiber trenching will start next Monday with a rental rock saw and ambitious optimism. If the two necessary fiber runs go well it might be possible to trench fiber to the guest house also, but the routing for that will be challenging. Should know more next week, we've got the rock saw rented for two days.
Rough Plans (software)
I currently run a two node Proxmox VE cluster (with RPi quorum device) and rely on automated zfs snapshots to support VM live migration. I've been very happy with this approach, it got me enough resiliency with not too much complexity in my current home. I haven't decided if I'm going to keep going forward with this approach or if I'm going to instead try a different approach. I've been using FreeBSD since 2.2.5 and zfs since it was in -STABLE. I've been happy with FreeNAS and I'm leaning towards a TrueNAS Core or Scale solution at the new place. I'm open to a TrueNAS Scale cluster and shared storage for VM live migrations, but I don't have a very clear vision of how this would look. Still researching, I don't really have to have a plan for this until the server closet construction is complete and that's at least a month out right now. It feels like TrueNAS Scale and Proxmox VE are really starting to overlap, which gives me some pause.
My goals are to support a bunch of VMs that will keep running even if one of the servers croaks. I also will have some form of Kubernetes running, but my needs there are simple and I've been well-served so far with just a k3s VM.
How and how much I can support Arm containers is an open question. My primary workstations are all Apple Silicon, so this is on my radar, but I'm strictly amd64 for homelab infrastructure currently.
Rough Plans (gear and everything else)
There will be a 42U four post rack in the new server closet. A wall-mounted small rack in an air conditioned room in the workshop, and a small switch in an interior closet in the guest house. I've got the UniFi switching needed to connect all that together. My current home servers are Supermicro Xeon D-1541 and I've been really happy with them, so I'm naturally leaning towards the new Xeon D-2700 platform (rack mount instead of mini tower this time), but haven't made any selections yet. Looks like they're finally available now, which is encouraging. I figure once I've fully moved I can repurpose my old FreeNAS hardware by putting it in the workshop for an offsite backup/zfs snapshot target.
I'll keep this thread updated as I make progress and I'd love to hear any advice or snark (as appropriate), both now and as I progress.
I wasn't quite sure where to post this. Mods please move if I missed the target.
TL;DR I'm deploying a UniFi campus network in my new home, including buried fiber runs to four different locations on the property. This will be an upgrade/replacement to my existing home network which consists of a pair of Proxmox VE servers, a FreeNAS Mini XL+, and a variety of other sundry boxes and RPis. This thread will document my progress and any related shenanigans.
The Current Situation
My new home had an existing "server" closet which was home to a junky 2-post round-hole rack that held A/V gear (media room is on the other side of one wall) and an old-and-busted security camera system. I'm keeping the room, but throwing everything else away. There's also enough existing cat5 in the walls and attic to qualify the home as a copper mine. I've already ripped all of that out, as well as the shag carpet that was in the server closet. New tile is down, but still waiting for an electrician to come in and tidy everything up and then for the walls to be cleaned up and new drywall.
This is how things looked when I bought the place (the UDM Pro is mine):
The closet is empty now, all the cabling worth keeping has been pulled up through the walls into the attic, and I set up the ISP modem, UDM Pro, and an access point up there as a temporary measure while construction takes place in the closet and main house. Thankfully the attic is insulated and everything seems happy enough up there for the short term even in the crazy Texas heat.
Rough Plans (layer 2)
I've already bought a bunch of UniFi access points, switches, and the UDM Pro to host layer 2. I have no interest in anything UniFi beyond the Network app. No plans for UniFi security or access control or whatever other nonsense they're cooking up. I've been happy enough with the switching and very happy with their WiFi since switching from Ruckus a few years back. I will be doing 10gbit fiber from the main house to a workshop which is about 400 meters from the main house. I'll also be doing fiber from the main house to a powered front gate. There's an existing RF point-to-point network link from the workshop to a guest home which I plan to re-use if I can get it working. Previous owner was using it and claims they got "decent" bandwidth from it. They were not very technical, but did manage to install it themselves. The fiber trenching will start next Monday with a rental rock saw and ambitious optimism. If the two necessary fiber runs go well it might be possible to trench fiber to the guest house also, but the routing for that will be challenging. Should know more next week, we've got the rock saw rented for two days.
Rough Plans (software)
I currently run a two node Proxmox VE cluster (with RPi quorum device) and rely on automated zfs snapshots to support VM live migration. I've been very happy with this approach, it got me enough resiliency with not too much complexity in my current home. I haven't decided if I'm going to keep going forward with this approach or if I'm going to instead try a different approach. I've been using FreeBSD since 2.2.5 and zfs since it was in -STABLE. I've been happy with FreeNAS and I'm leaning towards a TrueNAS Core or Scale solution at the new place. I'm open to a TrueNAS Scale cluster and shared storage for VM live migrations, but I don't have a very clear vision of how this would look. Still researching, I don't really have to have a plan for this until the server closet construction is complete and that's at least a month out right now. It feels like TrueNAS Scale and Proxmox VE are really starting to overlap, which gives me some pause.
My goals are to support a bunch of VMs that will keep running even if one of the servers croaks. I also will have some form of Kubernetes running, but my needs there are simple and I've been well-served so far with just a k3s VM.
How and how much I can support Arm containers is an open question. My primary workstations are all Apple Silicon, so this is on my radar, but I'm strictly amd64 for homelab infrastructure currently.
Rough Plans (gear and everything else)
There will be a 42U four post rack in the new server closet. A wall-mounted small rack in an air conditioned room in the workshop, and a small switch in an interior closet in the guest house. I've got the UniFi switching needed to connect all that together. My current home servers are Supermicro Xeon D-1541 and I've been really happy with them, so I'm naturally leaning towards the new Xeon D-2700 platform (rack mount instead of mini tower this time), but haven't made any selections yet. Looks like they're finally available now, which is encouraging. I figure once I've fully moved I can repurpose my old FreeNAS hardware by putting it in the workshop for an offsite backup/zfs snapshot target.
I'll keep this thread updated as I make progress and I'd love to hear any advice or snark (as appropriate), both now and as I progress.
I wasn't quite sure where to post this. Mods please move if I missed the target.
