[WTB] New Tiny-Mini-Micro Hardware - Starting Over From Scratch

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Chewy0645

New Member
Jan 25, 2023
7
9
3
I am in the Southeast US and starting over with a new home lab. I am limited in space, so I am looking at the Tiny-Mini-Micro as my solution.

Have an RPi 4 8GB running Docker & Portainer to host ARM containers and three NetGear 8-port 1000/100/10 switches. I want to add the following:
  • pfSense/OPNSense router that can handle 1 Gbit VPN;
  • x86-64 device to host x86 containers; and
  • NAS to host at least 8TB of storage with redundancy (two 8TB drives mirrored or four 4TB drives in RAID5).
My "production" environment will need to provide Firewall, VPN (Gigabit Fiber internet connection), Ad Blocking, Plex, and Gaming (Minecraft, Rust, etc.) Servers. I want the additional capacity to run a "test" environment to spin up and tear down containers that interest me.

I have experience with OMV, TrueNAS, and ProxMox, but I was just burned by my two NFS drives dying on my TrueNAS server. My draft new design is a commercial NAS and a Tiny-Mini-Micro with a four-port Intel NIC installed. The hosting would be:
  • T-M-M device: pfSense; gaming server; test containers;
  • RPi 4: PiHole; test containers;
  • NAS: Plex
The "client-side" of the network includes three Win10 gaming PCs connected via CAT6 and several devices connected via WiFi (printer, iPhones, iPad).

I have ~$1,200 to spend right now but will have another $500-600 to spend on February 1st and another ~$1,000 to spend on March 1st.

I am open to advice on hardware, where to source the hardware, which hosting OS to use, and configuration. My current questions are:

  • Which T-M-M device will give me the best bang for the buck in 2023?
  • Should I purchase a NAS (QNAP TS-464?), or do you still get more bang for the buck building your own?
  • Does it make sense to consider potential 2.5Gb networking in devices as a future upgrade?

Thank you in advance for your advice and assistance!
 
  • Like
Reactions: zac1 and Samir

Kwincy

New Member
Nov 29, 2016
20
9
3
44
  • Like
Reactions: Samir

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
I'd recommend moving PiHole to NAS (as a container) in addition to plex - make sure it has a recent intel CPU with iGPU for quicksync - otherwise, transcoding would be crap. Possibly illuminate RasPI or keep it dedicated to HaSS if needed.
Run your Pf/OpnSense on a dedicated box for the wan router/firewall.

If buying a commercial NAS with the idea of using m.2 slots for caching, read the details and find out how many PCIe v3 lanes NAS has for m.2 slots.
If you assume it is 4x for each m.2 slot, you'd be mistaken in most cases. Only high-end Xeon-based NAS boxes give 4 lanes to m.2 slots.
I have a mid-range TVS-872XT that gives 2x lanes for each m2 slot. Also, qnap is conservative with max memory spec. My 872 should only support 32 GB, but it works fine with the installed 64 GB (2x32gb).
 
Last edited:
  • Like
Reactions: Samir

Chewy0645

New Member
Jan 25, 2023
7
9
3
I'd recommend moving PiHole to NAS (as a container) in addition to plex - make sure it has a recent intel CPU with iGPU for quicksync - otherwise, transcoding would be crap. Possibly illuminate RasPI or keep it dedicated to HaaS if needed.
Run your Pf/OpnSense on a dedicated box for the wan router/firewall.
I have an AT&T Gigabit Fiber. From what I am reading it looks like I will need a processor at 2.0 Ghz or faster to handle the VPN. Any hardware recommendation for the WAN router/firewall?

If buying a commercial NAS with the idea of using m.2 slots for caching, read the details and find out how many PCIe v3 lanes NAS has for m.2 slots.
If you assume it is 4x for each m.2 slot, you'd be mistaken in most cases. Only high-end Xeon-based NAS boxes give 4 lanes to m.2 slots.
I have a mid-range TVS-872XT that gives 2x lanes for each m2 slot. Also, qnap is conservative with max memory spec. My 872 should only support 32 GB, but it works fine with the installed 64 GB (2x32gb).
If I had $2,500 - $3,000 then I could just go with something like the QNAP TVS-h874 and be done with it. I don't even have half of that right now but could come up with it over the next 6 weeks. Is it better just to wait, save, and go for one big capable NAS?
 
  • Like
Reactions: Samir

Kwincy

New Member
Nov 29, 2016
20
9
3
44
That's a beast of a NAS. If you need 8 bays, sure go for that one. It looks very powerful and looks to be the last NAS you'd ever buy. It's also worth mentioning that most if not all of the QNAP NAS devices can be expanded with something like a TR-004 later on down the road if you need more than 4 or 6 bays.
 
  • Like
Reactions: Samir

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
I have an AT&T Gigabit Fiber. From what I am reading it looks like I will need a processor at 2.0 Ghz or faster to handle the VPN. Any hardware recommendation for the WAN router/firewall?
Depending how cheap you could get it, Dell Wyse 5070 Extended (this one has a PCIe slot for additional NICs) could do near 1gig speed WireGuard VPN - The WYSE 5070, a perfect little VyOS device – blog.kroy.io
Alternatively, the regular 5070, you might find a "wifi" m2 adapter to the ethernet. like this one. We have a whole large thread here on 5070 with tons of information.

If I had $2,500 - $3,000 then I could just go with something like the QNAP TVS-h874 and be done with it. I don't even have half of that right now but could come up with it over the next 6 weeks. Is it better just to wait, save, and go for one big capable NAS?
I have had a mixed experiences with my Qnap. I bought it about 1 year ago as used on eBay for about 1k, which is about 1/2 the price of a new one.
Then one day, after doing a regular update, it just died on me. Like completely DEAD. Nothing but red light inside. This is a known and larger issue. Qnap refused to do a recall. The only solution provided was to extend the warranty by 1 year.
Sources:

Qnap originally quoted me $750 for repair service - which involves replacing the entire motherboard. Long story short, I got away only with paying shipping of my unit to qnap, California, and a month later got it back repaired.
Knowing now what I had to go through, I would never trust Qnap again. I should also add that Qnap's software is half-baked at best, and OFTEN new updates absolutely WILL break existing, previously running fine configurations.

You could and probably should roll your own DIY NAS, based on Truenas Scale maybe? Many folks here like beefier (all-in-one) boxes, and install proxmox on it. Pass-thru the hard drives to Truenas (or similar ZFS-based VM) and share the storage back to other running VMs .
 
  • Like
Reactions: Samir

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
I am in the Southeast US and starting over with a new home lab. I am limited in space, so I am looking at the Tiny-Mini-Micro as my solution.

Have an RPi 4 8GB running Docker & Portainer to host ARM containers and three NetGear 8-port 1000/100/10 switches. I want to add the following:
  • pfSense/OPNSense router that can handle 1 Gbit VPN;
  • x86-64 device to host x86 containers; and
  • NAS to host at least 8TB of storage with redundancy (two 8TB drives mirrored or four 4TB drives in RAID5).
My "production" environment will need to provide Firewall, VPN (Gigabit Fiber internet connection), Ad Blocking, Plex, and Gaming (Minecraft, Rust, etc.) Servers. I want the additional capacity to run a "test" environment to spin up and tear down containers that interest me.

I have experience with OMV, TrueNAS, and ProxMox, but I was just burned by my two NFS drives dying on my TrueNAS server. My draft new design is a commercial NAS and a Tiny-Mini-Micro with a four-port Intel NIC installed. The hosting would be:
  • T-M-M device: pfSense; gaming server; test containers;
  • RPi 4: PiHole; test containers;
  • NAS: Plex
The "client-side" of the network includes three Win10 gaming PCs connected via CAT6 and several devices connected via WiFi (printer, iPhones, iPad).

I have ~$1,200 to spend right now but will have another $500-600 to spend on February 1st and another ~$1,000 to spend on March 1st.

I am open to advice on hardware, where to source the hardware, which hosting OS to use, and configuration. My current questions are:

  • Which T-M-M device will give me the best bang for the buck in 2023?
  • Should I purchase a NAS (QNAP TS-464?), or do you still get more bang for the buck building your own?
  • Does it make sense to consider potential 2.5Gb networking in devices as a future upgrade?

Thank you in advance for your advice and assistance!
Well, if you want a NAS, that's pretty much a quadbay to start with. Eh, I personally like the HP Microserver Gen10 plus, but that was back maybe 1-2 years ago when they had a G5420/BYOD SKU for around 600 or so, and then you stuff it with coffee lake components from workbench leftovers for a cheap upgrade. That's probably not around anymore, and the newer v2s are rather expensive - I dunno, did @Patrick review the Gen10 Plus v2s yet? As for QNap, eeeeeh. I like the TVS-473 (either the RX421 or the V1500B series), but the newer TVS474s pack more horsepower but are more expensive. Unless you push the NAS hard (like run a ton of transcoding on Plex), or have high I/O requirements, you probably don't need anything with too much horsepower, to the point where even a used SuperMicro 5028D-TN4T will do the job (hell, I use an ancient Microserver Gen 7 N40L on my rack running TrueNAS Core 13 - I just offload the transcoding to something else and mount the disk remotely as part of the conversion job).
So yeah, look for a decent quadbay chassis with good rep, and see if you can get a decent pricing - maybe also a Synology?

As for TMM node, eh...if you want a quadport PCIe NIC to do routing, well, that leaves you with 4 main families of machines to consider:
a) Intel NUC Extreme (Skull Canyon, Ghost Canyon, Dragon Canyon and etc)
b) Dell Wyse 5070 Extended "fat" thin client
c) HP t740 "big" thin client (t730 is a bit too old for 2023, but if you are desperate for cash, it's still okay but only as a dedicated edge router for something like Opnsense, which it'll excel at)
d) Lenovo ThinkCentre Tiny P320-P340

The NUC extremes will be expensive - they are meant as premium gamer boxes, so even the retired ones will be more expensive than it should. It is definitely the "power move" option here.
The Wyse 5070 Extended? Well, that one is okay, but it uses a Gemini lake Atom, which is discontinued. You are also dealing with 32GB of DDR4 max in Linux, 16 in Windows. So that one is long in the tooth but likely the cheapest option - probably less than 140 for a machine, and it should handle most of your needs just fine. The t740 is a fun one and you can probably get it for about/less than 2 hundred with some shrill negotiations on fleabay, but the t750 is likely around the corner. It can do up to 64GB of RAM (what I have running ESXi 6.5) and I personally had it wired up with 40GbE to the NAS, and it performs fairly well. The ThinkCentre p series is a mini workstation, and whether it comes with the bracket and slot needed for a quadport card - that's usually the question, but it's a known quantity here on STH, as in, people actually messed with them. Pricing is consummate with the usual fleabay supply-and-demand shenanigans. The P320-340s should also work well here. The p-series should also have fully working SR-IOVwith ACS, so you can conceivably slap an Intel i350 quadport in there and do multiple virtual firewalls VMs at line speeds (the t740 doesn't have ACS so it can do it up to 7 VFs, only in Linux, and only for certain cards).

If you don't need a fullsize PCIe card, well, most normal TMMs have M2 A+E ports, and they are usually PCIe 3.0x2. There are M2 to MiniPCIe cards, and MiniPCIe versions of the i350-AM2 does exist. That would also allow you to do virtualized firewalls on a secondary port setup...if you wish (and assuming their BIOS have a complete SRIOV implementation). For those setups the headache is usually to try and find a place to fit it inside the chassis...neatly.

As for whether 2.5/5GbE is worth it? Let's put it this way...until vendors put together cheap 2.5/5/10GbE unmanaged switches and media converters or your Wifi5/6 router has multi-gig LAN ports, the entire thing just seem...pointless. I mean, if I get 2.5 GbE on a point to point link between 2 Intel i225LM devices, great. Otherwise I would not lose sleep over it.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
We have not done the MSG10+V2 yet. Genoa + Sapphire has us very behind right now.
 
  • Like
Reactions: Samir

thesunstarecontest

New Member
Jun 8, 2018
3
8
3
Depending how cheap you could get it, Dell Wyse 5070 Extended (this one has a PCIe slot for additional NICs) could do near 1gig speed WireGuard VPN - The WYSE 5070, a perfect little VyOS device – blog.kroy.io
Alternatively, the regular 5070, you might find a "wifi" m2 adapter to the ethernet. like this one. We have a whole large thread here on 5070 with tons of information.
There are official Dell parts for adding SFP and RJ45 NICs to the wifi port. I personally have the RJ45 part to use a 5070 Celeron J4105 as a PFSense box.
eBay links below:
SFP (DPN: PDYJ1 7P7GX P32XF 57GKF): Genuine Dell Wyse 5070 Network Card IO Port With Cable PDYJ1 7P7GX P32XF 57GKF | eBay
RJ45 (DPN: 442XR 565JP 9DJC4): New OEM Dell WYSE 5070 RJ 45 Port Daughter board w/ cable - 442XR 565JP 9DJC4 | eBay
 
Last edited:
  • Like
Reactions: Samir

Samir

Post Liker and Deal Hunter Extraordinaire!
Jul 21, 2017
3,257
1,447
113
49
HSV and SFO
  • Like
  • Love
Reactions: itronin and zac1

Chewy0645

New Member
Jan 25, 2023
7
9
3
I bought the hardware from @zac1. So far, so good!

OPNSense is installed and running, and I am starting the config now. Low profile NVME card arrives tomorrow and then I should have the Proxmox server up and running too!
 
  • Like
Reactions: Samir and zac1