Hi everyone,
I've been a long-time lurker here and on other forums, usually handling issues on my own. However, this time I'm looking for some advice, as I'm not fully familiar with the DIY parts market. I believe many of you have recent experience with DIY builds and might be able to help. I’ll start by outlining my current setup and what I'm aiming for.
1) Networking
During my recent home renovation, I upgraded most of my homelab network from old copper wiring to modern fiber optics. I now have three 25Gbps-capable switches connected via 100Gbps optics. This setup seems stable, and I don't anticipate any changes here.
2) Workstations
I’m using two modern Dell workstations, each equipped with MLX5 network adapters. Previously, they were running on Cat6/10Gbps copper. I don’t expect any changes on this front.
3) Workload
My workload is primarily ML/AI-focused, with most network transfers being large (100GB+) sequential files. Smaller, random files make up a minor portion of transfers. On average, I have around 15TB of daily transfers between storage and workstations, and the setup performs well.
4) Storage Setup
I currently have two QNAP TS-873A units with the following specifications:
5) Issues with Current Setup
Both QNAP units draw around 130W each, which feels a bit high. After testing, it seems 40W is the base power draw without disks, leaving about 70W for add-ons. While that’s reasonable, I’d love to reduce CPU idling power by half if possible, as these units run 24/7/365.
6) Potential Upgrades
I could upgrade the network card to an MLX5, but since the PCIe slot is only 3x4, I doubt I’ll see better performance than the ~18Gbps I get now. Even though my HDD pool isn’t likely to exceed 25Gbps, I see some potential for improvement with the six NVMe drives in each device. The faster home network also makes this upgrade more tempting.
7) Possible Transition to DIY Build
I’m considering selling both QNAP units and putting the funds toward a DIY setup, but I’d like to understand the potential costs and benefits first. I estimate I could recoup about €800 per unit. Here’s what I’d like to retain:
Would this DIY setup be worth the investment and effort?
10) Constraints
I've been a long-time lurker here and on other forums, usually handling issues on my own. However, this time I'm looking for some advice, as I'm not fully familiar with the DIY parts market. I believe many of you have recent experience with DIY builds and might be able to help. I’ll start by outlining my current setup and what I'm aiming for.
1) Networking
During my recent home renovation, I upgraded most of my homelab network from old copper wiring to modern fiber optics. I now have three 25Gbps-capable switches connected via 100Gbps optics. This setup seems stable, and I don't anticipate any changes here.
2) Workstations
I’m using two modern Dell workstations, each equipped with MLX5 network adapters. Previously, they were running on Cat6/10Gbps copper. I don’t expect any changes on this front.
3) Workload
My workload is primarily ML/AI-focused, with most network transfers being large (100GB+) sequential files. Smaller, random files make up a minor portion of transfers. On average, I have around 15TB of daily transfers between storage and workstations, and the setup performs well.
4) Storage Setup
I currently have two QNAP TS-873A units with the following specifications:
- Ryzen Embedded V1500B CPU (4 cores)
- 64GB ECC 2400MHz memory
- 8x HDDs of the same type
- 2x PCI Express 3 x4 slots
- 2x onboard NVMe slots (PCI Express 3 x1)
- 1x Dual 10Gbps card in a 3x8 slot
- 4x PCIe 4.0 NVMe drives on a PCIe 3x8 card with a switch
- Unit 1: Exposed 50/50 as an SMB share (for one workstation) and iSCSI (for the other). Small files are stored on the NVMe drives and accessed via SMB.
- Unit 2: Replicates all data from Unit 1 nightly, which I’d like to keep.
5) Issues with Current Setup
Both QNAP units draw around 130W each, which feels a bit high. After testing, it seems 40W is the base power draw without disks, leaving about 70W for add-ons. While that’s reasonable, I’d love to reduce CPU idling power by half if possible, as these units run 24/7/365.
6) Potential Upgrades
I could upgrade the network card to an MLX5, but since the PCIe slot is only 3x4, I doubt I’ll see better performance than the ~18Gbps I get now. Even though my HDD pool isn’t likely to exceed 25Gbps, I see some potential for improvement with the six NVMe drives in each device. The faster home network also makes this upgrade more tempting.
7) Possible Transition to DIY Build
I’m considering selling both QNAP units and putting the funds toward a DIY setup, but I’d like to understand the potential costs and benefits first. I estimate I could recoup about €800 per unit. Here’s what I’d like to retain:
- Keep: 8x HDDs, 6x NVMe drives, and the switched x4 PCIe card (unless I can switch to a cheaper PCIe 4x4 card if the DIY build supports bifurcation).
- MLX5 network adapter in a motherboard slot
- ECC RAM
- Quality HBA (e.g., 9500-8i) to control all 8 HDDs
- PCIe slot for my current switched x4 card or a bifurcation adapter
- Onboard SATA for OS drives (SATA is fine)
- Modern ZFS OS like TrueNAS
- Lower power consumption, if possible
- Higher CPU performance which will be easy i think achieved with any AMD CPU. Think Intel is no go - no ECC support.
- Compact form factor (e.g., Jonsbo case), although I realize it’ll be hard to beat QNAP’s compact design
Would this DIY setup be worth the investment and effort?
10) Constraints
- I can't use a rack-mounted solution with loud fans.
Last edited: