Troubleshooting GPU passthrough ESXi 6.5

soild79

New Member
Apr 30, 2017
10
0
1
40
I'm having BSOD after doing passthrough for my GTX 1050.... on a windows 7 64bits. Everything seems to be fine but crashes everytimes when i tried launching games.
Keep hitting nvlddmkm bsod....
 

Ch33rios

Member
Nov 29, 2016
102
6
18
39
I'm having BSOD after doing passthrough for my GTX 1050.... on a windows 7 64bits. Everything seems to be fine but crashes everytimes when i tried launching games.
Keep hitting nvlddmkm bsod....
What version of the nvidia drivers are you using? Did you follow all the steps highlighted in this thread about disabling the vmware version of the display driver?

Also you say this ONLY happens when you run a game? Is it ALL games? Only a specific one? Essentially you can run normal desktop apps (word processing, browsers, watch youtube, etc) without any sort of issue but when you hit a game its problematic?

Sorry for the barrage of ???'s but narrowing down the scenario can help :)
 
  • Like
Reactions: marcoi

soild79

New Member
Apr 30, 2017
10
0
1
40
I think I have used the following

Processor = i5 3.4GHz
Nvidia driver = 381.89
Motherboard = GA-Z270M-D3H (VT-d Enabled)
Graphic Card = Zotac GTX 1050
Windows 7 64bits

When I disabled the VMware SVGA 3D from the device manager, the pass-through does not work until I re-enable it.

The crashes will happen surely during the start of any 3D game and randomly if I run dxdiag. I accessing the VM using RDP.
 
Last edited:

soild79

New Member
Apr 30, 2017
10
0
1
40
I have downgraded the nvidia to 378.92 but still facing BSOD on nvlddmkm ...
Is it a must to disable the SVGA? cos when i disable the passthrough dun work...
 

marcoi

Well-Known Member
Apr 6, 2013
1,404
226
63
Gotha Florida
what is the configuration of your VM?
Did you assign all the memory?
Did you expose the CPU to the VM and put all the added parameters in the advance section of the vm?
 

soild79

New Member
Apr 30, 2017
10
0
1
40
With this it's works, passthrough seems to work as the VM display is shown in the monitor
upload_2017-5-9_23-27-38.png
Disabling the SVGA, the passthrough dun works...no display
upload_2017-5-9_23-28-15.png
 

marcoi

Well-Known Member
Apr 6, 2013
1,404
226
63
Gotha Florida
did you try the following?
First make sure vnc is working, then turn off the VM monitor using display properties. Select only show on monitor 2(whichever is attached to your Nvidia card.) Then go into device manager and disable the VM svga. It might take a few tried since the BSOD can happen randomly. If done you should not see any video in VM console.

It might be the process you are following to disable needs to change.

The other thing to test is have you tried your setup using a direct windows install IE can you install windows, the drivers and run games on the hardware? Might be some other issue like not enough power from he PSU or bad video card, etc..
 

marcoi

Well-Known Member
Apr 6, 2013
1,404
226
63
Gotha Florida
I dont use the esxi viewer app, it will stop working when you turn off the video card thus i installed vnc and use that
yes display only on the 1050 card monitor/tv that is hooked up. Dont display on the vmware svga 3d monitor. once you do that the screen in viewer will stop working. Then go into Deice manager and disable the VMware SVGA 3d card.

make sure that the monitor/tv connected to the 1050 video card is on at the time.
 

soild79

New Member
Apr 30, 2017
10
0
1
40
However when i disable the SVGA from the device manager and reboot, the passthrough will fail
 

marcoi

Well-Known Member
Apr 6, 2013
1,404
226
63
Gotha Florida
reboots work ok for me, when i do a shutdown though i need to reboot the esxi host.
Can you try it with a win 10 vm?
 

soild79

New Member
Apr 30, 2017
10
0
1
40
I have finally solved the nivida driver crashes issue.. however my configuration is a bit different. I have to left my SVGA enable otherwise the passthrough will fail. I have to add this parameter into the VM VMX configuration
  • Edit the pciPassthru0.msiEnabled and change the option to false
This is taken from the nvidia tech note
VMware vDGA / GPU Passthrough Requires That MSI is Disabled on VMs | NVIDIA

Once again, thanks everyone for giving me a hand.
 

helsyeah

Active Member
Aug 22, 2015
111
30
28
39
A little late to this party, but wanted to add another success story to the mix.

Setup:
  • ESX 6.0 Update 3
  • Supermicro X10SRH
  • Xeon E5-2630 V4 QS
  • 64 GB
  • GTX 970
Short version:
  • Added hypervisor.cpuid.v0 = false on W10 VM via VSphere VM Settings->VM Options->Advanced->Configuration.
  • Passthrough GTX970 to VM via standard VSphere web client.
  • Installed latest drivers (382 something)
  • After passthrough (and many BSODs), confirmed that GTX 970 is pciPassthru0 and added pciPassthru0.msiEnabled= false on W10 VM via VSphere VM Settings->VM Options->Advanced->Configuration.
  • Did NOT need to disable SVGA or the console display.
Long version:

I had no issue passing the 970 through to the VM and getting the latest (382 something) to install on the VM after setting hypervisor.cpuid.v0 = false.

At this point, I had NOT set pciPassthru0.msiEnabled = false on the VM yet.

As soon as I connect an external monitor though I would continually get VIDEO_TDR_FAILURE BSOD's on the VM. I was able to unplug the external monitor and use the VM Console to disable the SVGA driver, but this did not resolve the BSOD when reconnecting the external monitor. Also the BSOD would happen fast enough that I could not even get the VM console display to disable via Windows.

I eventually was able to modify a TDR time out registry setting remotely on the VM before attempting to login, which allowed me to get the console display disabled (but the video would still freeze/pause at the the timeout value I set).

Disabling the console display did NOT help either. I reverted the TDR registry setting remotely again, and set pciPassthru0.msiEnabled = false. This solved the BSOD and I was able to keep the Console display enabled with the SVGA driver enabled as well if I liked.

The 970 seems to be working normally, games seem to be running well, etc.
 
  • Like
Reactions: Ch33rios

TLN

Active Member
Feb 26, 2016
411
55
28
31
A little late to this party, but wanted to add another success story to the mix.
Hi.

May I ask you, do you still see Vmware SVGA adapter in mac os X?
i.e. when you work wiht single display, it shows two displays - your connected monitor and additional
 

helsyeah

Active Member
Aug 22, 2015
111
30
28
39
Hi.

May I ask you, do you still see Vmware SVGA adapter in mac os X?
i.e. when you work wiht single display, it shows two displays - your connected monitor and additional
Since I have not installed Max OSX, I can't say, I would assume so?? Hopefully someone else has used OS X on a VM and can chime in.
 

Ch33rios

Member
Nov 29, 2016
102
6
18
39
A little late to this party, but wanted to add another success story to the mix.

Setup:
  • ESX 6.0 Update 3
  • Supermicro X10SRH
  • Xeon E5-2630 V4 QS
  • 64 GB
  • GTX 970
Short version:
  • Added hypervisor.cpuid.v0 = false on W10 VM via VSphere VM Settings->VM Options->Advanced->Configuration.
  • Passthrough GTX970 to VM via standard VSphere web client.
  • Installed latest drivers (382 something)
  • After passthrough (and many BSODs), confirmed that GTX 970 is pciPassthru0 and added pciPassthru0.msiEnabled= false on W10 VM via VSphere VM Settings->VM Options->Advanced->Configuration.
  • Did NOT need to disable SVGA or the console display.
Long version:

I had no issue passing the 970 through to the VM and getting the latest (382 something) to install on the VM after setting hypervisor.cpuid.v0 = false.

At this point, I had NOT set pciPassthru0.msiEnabled = false on the VM yet.

As soon as I connect an external monitor though I would continually get VIDEO_TDR_FAILURE BSOD's on the VM. I was able to unplug the external monitor and use the VM Console to disable the SVGA driver, but this did not resolve the BSOD when reconnecting the external monitor. Also the BSOD would happen fast enough that I could not even get the VM console display to disable via Windows.

I eventually was able to modify a TDR time out registry setting remotely on the VM before attempting to login, which allowed me to get the console display disabled (but the video would still freeze/pause at the the timeout value I set).

Disabling the console display did NOT help either. I reverted the TDR registry setting remotely again, and set pciPassthru0.msiEnabled = false. This solved the BSOD and I was able to keep the Console display enabled with the SVGA driver enabled as well if I liked.

The 970 seems to be working normally, games seem to be running well, etc.

Very interesting that you didn't have to do the whole disable SVGA adapter. What exactly is the msiEnabled setting for?

Glad it worked though and thanks for contributing to the ever growing thread!