I'm trying to figure out a plan for a home setup for proxmox and freenas.
Eventually this will be the setup for a new home I plan to build. Since the new home has gotten delayed, I might as well figure out the server room now.
My goal would be to have one NAS and one virtualization server to store video, audio, etc connected to a A/V matrix. (+ dedicated OPNsense router)
In the past I have had one all in one fileserver/ NAS combo. This setup has evolved really only minimally since 2003.. haven't really looked at new industry/techniques for a while. I have decided it's time.
The NAS is pretty straightforward, and have it all setup w/ new hardware (Asrock D-1541 w/ 8x8TB raidz2 in Supermicro CSE-825). This is all setup, and have been migrating shares & stuff off my old file server (ubuntu with md 8x6TB RAID6). Since 10GbE is cheap now, I figured i'd test that out also.
I setup a couple of jails for experimentation, and threw a unifi controller in one. I'm not really planning on running a lot of VMs/containers in this box, the thought would be to put those in proxmox.
For proxmox the plan would be to run several VMs and a variety of containers. The key driver for the VMs would be to do PCIe passthrough for a couple of nvidia cards. My PVR of choice has been MythTv (for ~17 years), so the plan would be to run that on the virtualization server with a couple of VMs with pass though to the nvidia cards for NVDEC and HDMI audio. The containers would be the usual for Plex, home automation, etc.
I had a spare board with a i7-6700 so i loaded up proxmox and have prototyped some of this setup on it. I think the eventual plan would be a EPYC 7302P for production.
I threw a couple of old 3TB drives in an array in there for the time-being.
My biggest question is what to do with storage for the proxmox setup. I originally thought that I would like all of the storage on the freeNAS box, and host the drives for VMs and CTs on there. From what I have read it doesn't look like there is a straight-forward approach for that.
My 1st thought was to do iSCSI, but it appears that I cannot host filesystems for the containers that way. I also understand that I would need to have a ZIL/SLOG for freenas to have reasonable block performance.
My 2nd thought was maybe just do a couple of SSDs for a mirrored zfs vdev in the proxmox box, and backup/replicate those to the freenas box.
I looked at Ceph, but I don't have any desire to have any more boxes/ redundancy. I have to remember this is a home setup
Any thoughts?
Eventually this will be the setup for a new home I plan to build. Since the new home has gotten delayed, I might as well figure out the server room now.
My goal would be to have one NAS and one virtualization server to store video, audio, etc connected to a A/V matrix. (+ dedicated OPNsense router)
In the past I have had one all in one fileserver/ NAS combo. This setup has evolved really only minimally since 2003.. haven't really looked at new industry/techniques for a while. I have decided it's time.
The NAS is pretty straightforward, and have it all setup w/ new hardware (Asrock D-1541 w/ 8x8TB raidz2 in Supermicro CSE-825). This is all setup, and have been migrating shares & stuff off my old file server (ubuntu with md 8x6TB RAID6). Since 10GbE is cheap now, I figured i'd test that out also.
I setup a couple of jails for experimentation, and threw a unifi controller in one. I'm not really planning on running a lot of VMs/containers in this box, the thought would be to put those in proxmox.
For proxmox the plan would be to run several VMs and a variety of containers. The key driver for the VMs would be to do PCIe passthrough for a couple of nvidia cards. My PVR of choice has been MythTv (for ~17 years), so the plan would be to run that on the virtualization server with a couple of VMs with pass though to the nvidia cards for NVDEC and HDMI audio. The containers would be the usual for Plex, home automation, etc.
I had a spare board with a i7-6700 so i loaded up proxmox and have prototyped some of this setup on it. I think the eventual plan would be a EPYC 7302P for production.
I threw a couple of old 3TB drives in an array in there for the time-being.
My biggest question is what to do with storage for the proxmox setup. I originally thought that I would like all of the storage on the freeNAS box, and host the drives for VMs and CTs on there. From what I have read it doesn't look like there is a straight-forward approach for that.
My 1st thought was to do iSCSI, but it appears that I cannot host filesystems for the containers that way. I also understand that I would need to have a ZIL/SLOG for freenas to have reasonable block performance.
My 2nd thought was maybe just do a couple of SSDs for a mirrored zfs vdev in the proxmox box, and backup/replicate those to the freenas box.
I looked at Ceph, but I don't have any desire to have any more boxes/ redundancy. I have to remember this is a home setup
Any thoughts?