A little update on planning here:
I’ve decided to go “big” with TrueNAS and a SM 846/847, likely leaning to SM 847 since it has 12 additional drives. I’m planning to use Core unless Scale is mature enough by the end of the year which is when I’ll be building. I may add a JBOD later, but would prefer to keep each box independent, or limit each box to a single JBOD chassis. Clustering on Scale would be a future project depending on maturity of the feature, and likely involve building entirely new systems and migrating after testing.
Using assumed 18 TB disks x 36, a decision point would be vdev width and zpool configuration. 6-wide z2 x 6 vdev would give ~392 TB usable, whereas 12-wide z2 x 3 vdev would give ~448 TB usable. I’m thinking that as the data will be mostly static after Plex metadata has been generated, it won’t be necessary to have a zpool with so many vdevs as random IO won’t be a huge factor. Data will be mostly sequential, so leaning towards 12-wide vdevs. I do worry about resilver times with such big disks though. I’ve never had to encounter that issue in FreeNAS as my existing FreeNAS uses much smaller disks.
I don’t imagine I’ll need a motherboard with a ton of IO and slots. 2 x8 slots for a fiber NIC and a HBA should be sufficient. Perhaps a third slot for an external HBA if I get a JBOD in the future.
An issue I’m researching now is that Xeon E-class/AM4-class boards usually don’t have sufficient slots, and ECC UDIMMs are much more expensive than RDIMM. Xeon Scalable and EPYC may be a bit overkill on the other hand. Power usage is also a concern as this will be living in the homelab, though yes I’m aware with this many disks the large part of power budget will be in the disks. I’d also like to stay away from “ancient” hardware, such as L-series Xeons, and perhaps Xeon v3 and prior.
