Homelab / Worklab builds.. where to put the fastest drives..

[DocSal]

First post! .

Joined to get some advice, after years of lurking on the forum and yt, consuming valuable info.
I am rebuilding a Homelab server, and new-building a similar Worklab - here are the important specs:

Homelab "BoroGrove":
i9-9900K with corsair H115i AIO
Asus Z370-P mobo
32 gb 3000 NON-ECC RAM
video card as either a generic 1660S 6gb or MSI Vega Black 64 8gb

BoroGrove's storage and I/O:
NVME M2: 2x Samgung 960 500gb
SSD: 2x 870 evo 500 gb, 1x 870 evo 250gb
HHD: 8x Ironwolf 10TB
LSI 8 port HBA
Trendnet 2.5ghz realtek chipset nic

The Homelab will be rebuilt adding drives and HBA, and will have Proxmox with VMs and containers for: TrueNAS Scale, Jellyfin, MacOS, Ubuntu, pihole, etc. Planning on using ZFS in spite of not having ECC ram on this board/ CPU.



Worklab "TulgeyWood":
Ryzen 5700X with PA 120 cooler
Asrock X570 Velocita mobo
128 gb 3200 ECC RAM
Zotac 3060 12gb

TulgeyWood's storage and I/O:
NVME 2x P3Plus 2TB
SSD: 12x 870 evo 2TB, 1x 870 evo 500gb
HDD: 6x Ironwolf 12TB
ODD: slim dvd r/w
Asmedia chipset HBAs (2x) for 12 SATA ports
Icydocks for 8xSSD and 4xSSD+1Optical
intel 710 2x10gb nic

The Worklab will be built for Proxmox with VMs and containers for TrueNAS Scale, NextCloud, Ubuntu (MATLAB signal processing), Windows (signal processing software called Neuroguide - win only), Windows remote access desktops (for client record keeping, test administration, etc). Worklab will have storage pools in SDD (with either two or three 4x2tb disk pools in raidz1), and a backup sink as 6x12tb rust in raidz2.

Both motherboards have 2 NVME sockets, and can run them in mirror to make a super fast volume.
Both machine also have at least 1 SSD beyond the ones planed for storage pools.

I figure that the following uses may all benefit from quick access:
- proxmox boot drive
- truenas cache drive
- virtual machine disks
- iso storage

Which is most efficient to prioritize ? I am thinking of:
- SSD for TrueNAS cache
- SSD for boot/Proxmox
- 2xnvme in hardware raid for VMs <- not sure if this is overkill. ie.. should I use the NVMEs separately, perhaps one for the TrueNAS cache?
- extra space on boot SSD for ISO storage

thanks for any advice you have, or gotchas you spot!

 

[bigfellasdad]

What speed is your network please. I don't see the point having your storage bottle necked by network. Also how many simultaneous users accessing the storage?

 

[i386]

I would like to see performance numbers of the 12x 870 evos.
In the past consumer (sata) ssds held up their performance numbers for a few seconds until the cache was full and then the performance dropped in multiple steps to extremly low throughput/iops and high latency.

 

[DocSal]

What speed is your network please. I don't see the point having your storage bottle necked by network. Also how many simultaneous users accessing the storage?

2+1 gb fiber at home (2 providers - planning on bonding) The Worklab machine will move to a fiber gigabit location.
The Homelab will have only 1-2 users accessing it at a time. The Worklab up to 10 people, but not streaming, just VM access.

 

[DocSal]

I would like to see performance numbers of the 12x 870 evos.
In the past consumer (sata) ssds held up their performance numbers for a few seconds until the cache was full and then the performance dropped in multiple steps to extremly low throughput/iops and high latency.

Will let you know as I build it .

The main use for that 8x or 12x pool (may use 4 for system drives instead of the pool) is to act as a NextCloud hub storage for my company.

 

[DocSal]

Still wondering if the m.2 nvme's should be used for truenas cache, for VM's, or for both, and/or if the dual nvme's should be put in hardware raid..

Will truenas benefit from the m.2 nvme speed for its cache? or will an SDD be more than sufficient?

 

[NPS]

Will truenas benefit from the m.2 nvme speed for its cache? or will an SDD be more than sufficient?

What type of cache? L2ARC of SLOG? This topic is quite complex. Did you read up on this? It might take you some time to really get a grasp on it.

I guess the question regarding your network speed was more about your LAN not your internet connection but maybe I didn't read the part where you stated what services should be accessed over which connection and what your performance expectations are for these services.