ESXI 6.5u3 - GPU passthrough to single VM

Discussion in 'VMware, VirtualBox, Citrix' started by poto, Oct 5, 2019.

  1. poto

    poto Active Member

    Joined:
    May 18, 2013
    Messages:
    193
    Likes Received:
    59
    Hi all, I'm planning to virtualize personal workstation to minimize electrical cost and space; would appreciate experiences/opinions regarding GPU choice. Primary goal is to have stable GPU passthrough to single VM with minimum of tweaking. Rack is close enough to attach monitors, kb/mouse, audio directly, no vSGA or vGPU needed.

    Current W/S:
    SM X10SRA, E5-2667 v3, 32GB ram, Intel P3605 1.6tb NVME AIC, Nvidia GTX660 (3x HD monitors), Windows Server 2012R2
    - this machine never "breaks a sweat", but the ancient GPU is pretty power-hungry even at idle

    Target host:
    SM X10SRL, E5-2697 v3, 128GB ram, Intel 750 1.2tb NVME AIC
    Intel 10Gbase-T nic, 2x LSI 9207-8e (FreeNAS backup chassis)
    - 5-8 VM's (media-management - Sonarr, Radarr, etc)
    - plenty of ram/cpu to spare, planning to use P3605 from W/S to store VM's, iops should be sufficient

    Use case:
    basic business apps - MS Office, security cam monitoring, web browsing, occasional light gaming (vintage low-res stuff), must be able to light up 3x HD monitors so probably 4x mini-DP, single-slot and power-efficient, Windows Server 2012 R2

    Plan:
    1 - convert/clone WS to VM and migrate to secondary host as proof-of-concept
    2 - validate operation via RDP (spare laptop will become host controller for maint/upgrade/migration)
    3 - passthrough usb hub and test kb/mouse/audio
    4 - install/passthrough GPU, reboot to validate passthrough VM hardware persistence and host stability
    4 - attach monitors and profit

    Incomplete information:
    From reviewing available threads last 24 months, I'm concluding a safe choice would be Quadro P2000, currently between $250-$350. It seems to be supported for vDGA, and would also be useful if future needs included streaming transcodes.

    Would appreciate hearing from anyone with similar OS/hardware experience. Also, if I'm missing any value choices for stable passthrough on ESXI 6.5u3 to single VM, saving a few $$$ is always welcome!

    Cheers,
    John
     
    #1
    Last edited: Oct 8, 2019
  2. besterino

    besterino New Member

    Joined:
    Apr 22, 2017
    Messages:
    6
    Likes Received:
    3
    You forgot to mention what OS the VM with GPU will be running. As you mentioned ms office, I suppose Windows?

    Not running your specific use case, or even close, but I currently have a X11SPi-TF/4108 with P400 and Renesas USB 3 controller pass-through’ed for a 24/7 browsing / Video watching / light weight application VM. Works like a charm. All with ESXi 6.7 („no U“) and Windows 10 VM. Uptime is months at a time (shutdown only for maintenance/ tinkering etc.). Setup was also easy. ;)

    As another system I run a Threadripper 1920X (X399D8A-T2) with a 2080Ti on ESXi 6.7U3.... totally different story. TR4-plattform causes a few more headaches than Intel for me, plus getting the consumer GPU to work in Passthrough also took a while. Currently use it as a „high end“ Gaming rig for testing purposes only, but 6 weeks now and no stability issues. CPU and RAM overclocked by the way. Yeah well, not recommended for production. ;)

    Long story short, Intel E5 plattform (or SP) is IMHO the best way to get it to work. No guarantee on the 2000P though, since NVIDIA is very reluctant to give away official information of what works other than the big Tesla’s and more recent Quadros (IIRC some game streaming services also use Quadros).

    Out of curiosity I googled a bit for the 2000p, but only got mixed results / user stories. :(
     
    #2
    poto likes this.
  3. poto

    poto Active Member

    Joined:
    May 18, 2013
    Messages:
    193
    Likes Received:
    59
    Good point, updated post with target OS.

    When you mention "P400", is that a Quadro P400 GPU? If so, was it "plug & play" after standard esxi passthrough configuration, or were additional measures required? They are available < $100, so considerable cost savings vs Quadro P2000.

    Common use case for Quadro P2000 seems to be plex servers providing transcoding streams to friends/family. That is really driving the prices up. Probably overkill for my needs, but I'm willing to pay a bit more for more stable/supported solution.

    Regards,
    John
     
    #3
  4. marv

    marv Member

    Joined:
    Apr 2, 2015
    Messages:
    103
    Likes Received:
    21
    I run X9SRL-F with Xeon E5-2660v2 using vt-d for Radeon 7850 and VIA USB3 card into Windows 10 VM.

    It works stable enough, but I had issues with GPU drivers. With Radeon Adrenaline 2019 series, sound was crapping out after a few hours-days and reboot was needed. I am now on 18.2.1 which seem stable.
     
    #4
    poto likes this.
  5. poto

    poto Active Member

    Joined:
    May 18, 2013
    Messages:
    193
    Likes Received:
    59
    Thanks to those who contributed data points - much appreciated.

    Migration checkpoints resolved - even managed to passthrough ancient GTX460, although drivers reset themselves semi-regularly when stressed.

    All that remains is buying Quadro P2000, but current price trends point toward a seller's market. I'll put this project on "pause" while I wait for Quadro P2000 prices to trend downward a bit. One constant that can be counted on is the pipeline of new products, and the relentless upgrade cycle!
     
    #5
Similar Threads: ESXI 65u3
Forum Title Date
VMware, VirtualBox, Citrix Help choosing m.2 NVMe enterprise drives for ESXi servers Nov 2, 2019
VMware, VirtualBox, Citrix ESXi 6.0 GPU Passthru + Hw Virtualization Oct 25, 2019
VMware, VirtualBox, Citrix Workstation (ESXi lab) to full Type 1 ESXi 6.7u3 storage issues Oct 24, 2019
VMware, VirtualBox, Citrix ESXi NFS Rescan Oct 5, 2019
VMware, VirtualBox, Citrix Installed ESXi 6.7 on a 10 year old Dell server. vCenter worthwhile? Aug 25, 2019

Share This Page