Hi everyone
I'm a bit at a loss of how to approach my... let's call it a challenge.
Goal:
Passing through Quadro P400 to Jellyfin VM.
Why this is a challenge:
My Huananzhi X79 Motherboards do not offer PCI virtualization/passthrough (or I am too blind to find it).
Setup:
I have an old 4U chassis from Overland that I "upgraded" the mainboard of (to Supermicro X79 dual socket) and am now running TrueNAS on. Used to be, I ran ESXi on the thing with a FreeNAS VM on it that had the SAS HBA added via PCI passthrough.
I wanted to get fancier a few months ago and built myself two hosts based on chinese X79 boards. I installed ESXi 7.0 on them and wanted to do shared storage but ended up just running the VMs off of internal NVMe storage anyway.
I have ONE Quadro P400 so if I succeed with the passthrough, the Jellyfin VM will be stuck on one host.
Possible solutions and their pros and cons:
Scenario 1:
I can install the NVIDA driver VIB that allows for virtual GPUs. Assuming that is supported on this card, I could buy a second card for the other host.
Pros:
vMotion would work
I wouldn't have to change the Jellyfin VM (I have not found a way to back up and restore the configuration and metadata. I would have to organize my media again).
Cons:
This is assuming vGPU works without the corresponding hardware virtualization features.
This is also assuming I can find the VIB, which is locked behind an Enterprise paywall.
Scnario 2:
I install the GPU in the Overland chassis and deploy a Jellyfin as a jail on TrueNAS
Pro:
No need for second card.
No need for Scenario 1 assumptions
Cons:
I have no experience with jails and quite frankly I don't trust the setup... I don't know how to handle backup on them (I am using Veeam currently)
I would have to put in all the work on my media in a new Jellyfin deployment.
Scenario 3:
I install a Hypervisor that handles ZFS (so the storage could be imported) and is better suited for virtualization than TrueNAS. Unraid is one candidate. I then migrate all VMs to this setup.
Pros:
Less power consumption due to only one server running
This server is known to handle PCI passthrough.
Cons:
Need to find suitable product.
Unraid e.g. costs licensing money.
Have no idea how backup on unraid would work.
If I cannot migrate the VMs, would have to basically redeploy everything.
Scenario 4:
I go back to my poor mans hyperconverged solution using ESXi on the Overland machine, isntalling TrueNAS on a VM and passing through the HBA.
Pros:
ZFS would remain intact.
Migration path proven. Have experience.
Would also save on power.
Could possibly still use other hosts in a cluster with this machine.
No need to redeploy anything. Since all are X79 CPUs, vMotion should be possible.
Backup solution with Veeam would continue to work.
Cons:
Whether the TrueNAS VM then provides iSCSI storage to its own host or not, the whole setup feels kinda janky.
Scenario 5:
I find BIOS updates for my chinese boards that give this option or someone can show me how to find the option I am too stupid to locate.
Pro:
Least work
Cons:
I think there are hacked BIOSes but on Russian sites. I do not speak Russian. I do not trust myself doing this without understanding sources.
Frankly, right now I am leaning towards Scenario 4. Janky as it may be, that sucker ran like this for several years without major issues.
Unless any of you have any input on how to make any other, more professional scenario viable?
I'd be happy for anyone providing input or acting as a sounding board.
Thank you.
I'm a bit at a loss of how to approach my... let's call it a challenge.
Goal:
Passing through Quadro P400 to Jellyfin VM.
Why this is a challenge:
My Huananzhi X79 Motherboards do not offer PCI virtualization/passthrough (or I am too blind to find it).
Setup:
I have an old 4U chassis from Overland that I "upgraded" the mainboard of (to Supermicro X79 dual socket) and am now running TrueNAS on. Used to be, I ran ESXi on the thing with a FreeNAS VM on it that had the SAS HBA added via PCI passthrough.
I wanted to get fancier a few months ago and built myself two hosts based on chinese X79 boards. I installed ESXi 7.0 on them and wanted to do shared storage but ended up just running the VMs off of internal NVMe storage anyway.
I have ONE Quadro P400 so if I succeed with the passthrough, the Jellyfin VM will be stuck on one host.
Possible solutions and their pros and cons:
Scenario 1:
I can install the NVIDA driver VIB that allows for virtual GPUs. Assuming that is supported on this card, I could buy a second card for the other host.
Pros:
vMotion would work
I wouldn't have to change the Jellyfin VM (I have not found a way to back up and restore the configuration and metadata. I would have to organize my media again).
Cons:
This is assuming vGPU works without the corresponding hardware virtualization features.
This is also assuming I can find the VIB, which is locked behind an Enterprise paywall.
Scnario 2:
I install the GPU in the Overland chassis and deploy a Jellyfin as a jail on TrueNAS
Pro:
No need for second card.
No need for Scenario 1 assumptions
Cons:
I have no experience with jails and quite frankly I don't trust the setup... I don't know how to handle backup on them (I am using Veeam currently)
I would have to put in all the work on my media in a new Jellyfin deployment.
Scenario 3:
I install a Hypervisor that handles ZFS (so the storage could be imported) and is better suited for virtualization than TrueNAS. Unraid is one candidate. I then migrate all VMs to this setup.
Pros:
Less power consumption due to only one server running
This server is known to handle PCI passthrough.
Cons:
Need to find suitable product.
Unraid e.g. costs licensing money.
Have no idea how backup on unraid would work.
If I cannot migrate the VMs, would have to basically redeploy everything.
Scenario 4:
I go back to my poor mans hyperconverged solution using ESXi on the Overland machine, isntalling TrueNAS on a VM and passing through the HBA.
Pros:
ZFS would remain intact.
Migration path proven. Have experience.
Would also save on power.
Could possibly still use other hosts in a cluster with this machine.
No need to redeploy anything. Since all are X79 CPUs, vMotion should be possible.
Backup solution with Veeam would continue to work.
Cons:
Whether the TrueNAS VM then provides iSCSI storage to its own host or not, the whole setup feels kinda janky.
Scenario 5:
I find BIOS updates for my chinese boards that give this option or someone can show me how to find the option I am too stupid to locate.
Pro:
Least work
Cons:
I think there are hacked BIOSes but on Russian sites. I do not speak Russian. I do not trust myself doing this without understanding sources.
Frankly, right now I am leaning towards Scenario 4. Janky as it may be, that sucker ran like this for several years without major issues.
Unless any of you have any input on how to make any other, more professional scenario viable?
I'd be happy for anyone providing input or acting as a sounding board.
Thank you.