I am in the Southeast US and starting over with a new home lab. I am limited in space, so I am looking at the Tiny-Mini-Micro as my solution.
Have an RPi 4 8GB running Docker & Portainer to host ARM containers and three NetGear 8-port 1000/100/10 switches. I want to add the following:
- pfSense/OPNSense router that can handle 1 Gbit VPN;
- x86-64 device to host x86 containers; and
- NAS to host at least 8TB of storage with redundancy (two 8TB drives mirrored or four 4TB drives in RAID5).
My "production" environment will need to provide Firewall, VPN (Gigabit Fiber internet connection), Ad Blocking, Plex, and Gaming (Minecraft, Rust, etc.) Servers. I want the additional capacity to run a "test" environment to spin up and tear down containers that interest me.
I have experience with OMV, TrueNAS, and ProxMox, but I was just burned by my two NFS drives dying on my TrueNAS server. My draft new design is a commercial NAS and a Tiny-Mini-Micro with a four-port Intel NIC installed. The hosting would be:
- T-M-M device: pfSense; gaming server; test containers;
- RPi 4: PiHole; test containers;
- NAS: Plex
The "client-side" of the network includes three Win10 gaming PCs connected via CAT6 and several devices connected via WiFi (printer, iPhones, iPad).
I have ~$1,200 to spend right now but will have another $500-600 to spend on February 1st and another ~$1,000 to spend on March 1st.
I am open to advice on hardware, where to source the hardware, which hosting OS to use, and configuration. My current questions are:
- Which T-M-M device will give me the best bang for the buck in 2023?
- Should I purchase a NAS (QNAP TS-464?), or do you still get more bang for the buck building your own?
- Does it make sense to consider potential 2.5Gb networking in devices as a future upgrade?
Thank you in advance for your advice and assistance!
Well, if you want a NAS, that's pretty much a quadbay to start with. Eh, I personally like the HP Microserver Gen10 plus, but that was back maybe 1-2 years ago when they had a G5420/BYOD SKU for around 600 or so, and then you stuff it with coffee lake components from workbench leftovers for a cheap upgrade. That's probably not around anymore, and the newer v2s are rather expensive - I dunno, did
@Patrick review the Gen10 Plus v2s yet? As for QNap, eeeeeh. I like the TVS-473 (either the RX421 or the V1500B series), but the newer TVS474s pack more horsepower but are more expensive. Unless you push the NAS hard (like run a ton of transcoding on Plex), or have high I/O requirements, you probably don't need anything with too much horsepower, to the point where even a used SuperMicro 5028D-TN4T will do the job (hell, I use an ancient Microserver Gen 7 N40L on my rack running TrueNAS Core 13 - I just offload the transcoding to something else and mount the disk remotely as part of the conversion job).
So yeah, look for a decent quadbay chassis with good rep, and see if you can get a decent pricing - maybe also a Synology?
As for TMM node, eh...if you want a quadport PCIe NIC to do routing, well, that leaves you with 4 main families of machines to consider:
a) Intel NUC Extreme (Skull Canyon, Ghost Canyon, Dragon Canyon and etc)
b) Dell Wyse 5070 Extended "fat" thin client
c) HP t740 "big" thin client (t730 is a bit too old for 2023, but if you are desperate for cash, it's still okay but only as a dedicated edge router for something like Opnsense, which it'll excel at)
d) Lenovo ThinkCentre Tiny P320-P340
The NUC extremes will be expensive - they are meant as premium gamer boxes, so even the retired ones will be more expensive than it should. It is definitely the "power move" option here.
The Wyse 5070 Extended? Well, that one is okay, but it uses a Gemini lake Atom, which is discontinued. You are also dealing with 32GB of DDR4 max in Linux, 16 in Windows. So that one is long in the tooth but likely the cheapest option - probably less than 140 for a machine, and it should handle most of your needs just fine. The t740 is a fun one and you can probably get it for about/less than 2 hundred with some shrill negotiations on fleabay, but the t750 is likely around the corner. It can do up to 64GB of RAM (what I have running ESXi 6.5) and I personally had it wired up with 40GbE to the NAS, and it performs fairly well. The ThinkCentre p series is a mini workstation, and whether it comes with the bracket and slot needed for a quadport card - that's usually the question, but it's a known quantity here on STH, as in, people actually messed with them. Pricing is consummate with the usual fleabay supply-and-demand shenanigans. The P320-340s should also work well here. The p-series should also have fully working SR-IOVwith ACS, so you can conceivably slap an Intel i350 quadport in there and do multiple virtual firewalls VMs at line speeds (the t740 doesn't have ACS so it can do it up to 7 VFs, only in Linux, and only for certain cards).
If you don't need a fullsize PCIe card, well, most normal TMMs have M2 A+E ports, and they are usually PCIe 3.0x2. There are M2 to MiniPCIe cards, and
MiniPCIe versions of the i350-AM2 does exist. That would also allow you to do virtualized firewalls on a secondary port setup...if you wish (and assuming their BIOS have a complete SRIOV implementation). For those setups the headache is usually to try and find a place to fit it inside the chassis...neatly.
As for whether 2.5/5GbE is worth it? Let's put it this way...until vendors put together cheap 2.5/5/10GbE unmanaged switches and media converters or your Wifi5/6 router has multi-gig LAN ports, the entire thing just seem...pointless. I mean, if I get 2.5 GbE on a point to point link between 2 Intel i225LM devices, great. Otherwise I would not lose sleep over it.
