GPU Passthrough Radeon GPUs to Client VM

[Rand__]

So I need a 1slot GPU for my backup server which doubles as media converter. I used to have a P4000 in it but I since moved that back to an ESXi box so the Kid can play again. Now I haven't found any 1 slot GPU that was to my liking so was thinking about moving the P4000 back, but then need a new GPU for games...
I know Nvidia was a pain in the backside (non quadros) so wondered re Radeons...
Have not seen too many threads which might be a good sign, but thought I'd ask.

Alternative is to get a cheap old NVS until the RTX get cheap(er-ish) in a few months...

 

[Rand__]

Nobody using AMD GPUs or nobody passing them through to VMs?

 

[muhfugen]

Nvidia GTX and RTXes have worked fine under vSphere for me for the past several years. For GeForces set
hypervisor.cpuid.v0 = false
in the .VMX file/advanced settings of the VM to get around the virtualization check in the driver, and disable the built in VMware SVGA display adapter in device manager otherwise it will cause the Nvidia driver to constantly crash. You may also want to set latency sensitivity to high/100% CPU reservations if you have enough spare capacity. The only thing i've never gotten to work well was sli in full screen mode, sli would work in full screen windowed mode, but in normal full screen, it would have some kind of corruption in the output.

 

[Rand__]

Never could get my 1080 to work properly with that...

 

[muhfugen]

What went wrong? i've done this with a pair of GTX 980s and a RTX 2080ti for gaming and CUDA/OpenCL apps, and single slot GTX 1050s and 1060s for media transcoding. if it becomes unresponsive, blue screens or restarts itself it is a TDR issue. When installing the drivers, ensure the display isnt connected, after installing the drivers disable the VMware SVGA graphics adapter, and then reconnect the display, and configure Windows to just not display anything on the virtual monitor on the VMware SVGA adapter. There are registry keys you can set to alter TDR behavior but all it does is keeps windows from rebooting itself.

 

[Rand__]

Didnt have a display, wanted to remote in (vdi)
But it was reboots and bluescreens so maybe it was TDR. Might need to give it another try

 

[ttabbal]

I do passthrough with AMD. It's an older RX580, but it works great. I can't offer any advice on ESXI though, as I do it in Linux/VFIO. I didn't have any trouble on the Windows side with drivers etc, and it knows it's virtualized.

 

[muhfugen]

Didnt have a display, wanted to remote in (vdi)
But it was reboots and bluescreens so maybe it was TDR. Might need to give it another try

Hmm… well if you want to give it a try again set the following values to disable TDR. You could also probably set TdrDebugMode to 1 to do the same thing.
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers
TdrLimitTime (DWORD) 0xFFFFFFF
TdrLimitCount (DWORD) 0xFFFFFFFF

Testing and Debugging TDR During Driver Development - Windows drivers

TDR testing and debugging information for developers

docs.microsoft.com

 

[Rand__]

Will need to obtain a new card to test first

 

[randman]

I got an Nvidia Quadro P2200 last week and installed it in my HPE ML30 Gen9 running ESXi 6.5 U3. I have Plex running on an Ubuntu 18.04 VM, and the P2200 runs well (hardware encode and decode work well for Plex). The P2200 is a one slot GPU.

 

[Rand__]

yes thanks, I know these work fine, but price performance ratio of these is not great

 

[vangoose]

Just saw this video, may give you some idea. It's for 7.0 but it may help on 6.7 as well.

 

[Rand__]

Interesting, but it would have been more impressive if they had not used quadro cards

 

[randman]

A new feature in ESXi 7.0 is Dynamic DirectPath I/O. Seems interesting:

vSphere 7 - Assignable Hardware

Hardware accelerators are great for performance but can compromise application availability. The Assignable Hardware framework in vSphere 7 solves this.

blogs.vmware.com

So, if one has an extra graphics card to put on a second ESXi host, seems this feature will allow a VM using passthrough to more easily be migrated from one ESXi host to another. Seems it would allow, at least, manual migration via power off, migrate, power on, given that my license doesn't have vMotion. Or a more easy restore procedure from a backup to second ESXi host if the first host goes down.

And, if one feels guilty about the 2nd graphics card just sitting around for backup, see previous folding@home video to feel better about buying a spare graphics card

 

[Rand__]

Yeah I wondered how it would work for GPUs, but since I have not upgraded yet i was not able to test

 

[gb00s]

I only got GTX 970/80's or 1070/1080's passthrough done on Proxmox. No problems there. Surprisingly no 50/60's of each series. No Experience with AMD GPU's. But there is a 1070 'Katana' which is a 1slot GPU. The problem : You barely find them and if, they are still expansive.