ESXI 5.5 GPU passthrough - Finally.

[kujinke]

Hi,

I Wanted to share my experience with GPU passthrough on ESXI 5.5U1 so other people could also use my setup. it has been alot of headbreaking and tear breaking untill i managed to get it to work.
for several months i was searching for the right hardware - setup to get it working without dropping 2k$ on a single video card.
I managed to borrow a couple of cards: GTX 660, GTX 770, ATI RX 470, none of those actually worked - same error (code 43).
for some reason i couldnt get even closer to that code 43 error using a newer versions of esxi (e.g 6.0, 6.5) - everytime i was trying to add a pcie hardware to the vm, I was experiencing crashes and freezing of the entire box. only version that really made the trick was 5.5U1.

Main Hardware:
Mobo: Supermicro X9DRI-F
CPU: Dual E5-2670
Video Card: Gigabyte GTX 1060 Mini ITX OC 3GB
Sound Card: Creative Sound Blaster Z


So I managed to passthrough a GTX 1060 3gb to my vm doing the following steps:
1. first ive created a new win10 vm - added pcie device - Nvidia Card (couldnt add the audio device - BSODS)
2. added 4 gigs of ram to the vm and fired up the vm installing the OS.
3. once ive finished installing the os, I installed the NVIDIA Drivers.
4. experienced the code 43 again.
5. turned the vm off, edit the vmx file and added the following parameters:
pciHole.start = “1200″
pciHole.end = “2200″
hypervisor.cpuid.v0 = FALSE
6. saved and uploaded it to the datastore.
7. turned the vm on - again code 43, so i uninstalled the device on "Device Manager".
8. rescan for new devices and video card suddenly appeared as working properly.

 

[MumbleFysh]

Could you do some benchmarks? I'd be interested in seeing how much of a hit it'd take versus in a normal configuration.

 

[kujinke]

ofc, also judging by fps - gaming performance seems to be almost identical.

but i will post some benchmarks.

 

[acmcool]

Did you get audio working at all?

 

[kujinke]

ofc, also judging by fps - gaming performance seems to be almost identical.

but i will post some benchmarks.

ofc, i also passed a creative sound card. it feels just like a local machine with a boot times on steroids.

 

[nk215]

did it work with remote desktop (horizon view)? Many ppl get gtx card to work on esxi 6+ as a multi head machine but not as remote virtual desktop.

 

[Diavuno]

I'm interested in this... I could part out my X8 dtn+ desktop (grab ram+GPU+sound?) And add that to my esxi boxes.... The primary box could then hold my desktop and I can use my laptop to rdp to the horsepower and do my 3d work/games

I currently run a HD 9850

Sent from my 0PJA2 using Tapatalk

 

[acmcool]

But did anyone get gpu audio working ? I dont want to use separate audio card


Sent from my iPhone using Tapatalk

 

[kujinke]

PASSMARK 8.0

 

[kujinke]

Managed to passthrou the audio device also

what I did ?

1. Removed GPU PCIE.
2. Added Audio pcie alone.
3. Started VM, Saw it on Device Manager.
4. Shutdown, Added the GPU again.
5. Started the VM. everything works like a charm.

 

[Ch33rios]

I'm interested in this... I could part out my X8 dtn+ desktop (grab ram+GPU+sound?) And add that to my esxi boxes.... The primary box could then hold my desktop and I can use my laptop to rdp to the horsepower and do my 3d work/games

I currently run a HD 9850

Sent from my 0PJA2 using Tapatalk

Im currently running a fully virtualized Win10 machine with both a USB PCIe and GTX970 graphics card passed through without issue and with great performance. Admittedly the setup took some persistence but thanks to a specific member of this forum I got it working

See my thread HERE

As far as performance, I was coming from an i5-3470k w/ 8GB RAM to my virtualized Win10 image that was assigned 8GB RAM and 4 cores from the Xeon E3-1230v5. My 3D Mark scores were damn near identical (and in some run throughs actually was BETTER on the vm) and my FPS in Overwatch (my only real point of reference at the moment) are about the same as well (Ultra settings except for dynamic reflections are set to off in the game as it was a killer to frame rates in both of the setups). Thats at 1080p resolution.

I've read around that using EFI instead of the standard BIOS setup (which VMware defaults to) also provides a more bare-metal performance figure but I haven't tried it just yet. I have a new Win10 VM built already but I haven't done the passthrough setup to test things out. One of these days I'll do it

 

[Tux]

Hi,

I Wanted to share my experience with GPU passthrough on ESXI 5.5U1 so other people could also use my setup. it has been alot of headbreaking and tear breaking untill i managed to get it to work.
for several months i was searching for the right hardware - setup to get it working without dropping 2k$ on a single video card.
I managed to borrow a couple of cards: GTX 660, GTX 770, ATI RX 470, none of those actually worked - same error (code 43).
for some reason i couldnt get even closer to that code 43 error using a newer versions of esxi (e.g 6.0, 6.5) - everytime i was trying to add a pcie hardware to the vm, I was experiencing crashes and freezing of the entire box. only version that really made the trick was 5.5U1.

Main Hardware:
Mobo: Supermicro X9DRI-F
CPU: Dual E5-2670
Video Card: Gigabyte GTX 1060 Mini ITX OC 3GB
Sound Card: Creative Sound Blaster Z


So I managed to passthrough a GTX 1060 3gb to my vm doing the following steps:
1. first ive created a new win10 vm - added pcie device - Nvidia Card (couldnt add the audio device - BSODS)
2. added 4 gigs of ram to the vm and fired up the vm installing the OS.
3. once ive finished installing the os, I installed the NVIDIA Drivers.
4. experienced the code 43 again.
5. turned the vm off, edit the vmx file and added the following parameters:
pciHole.start = “1200″
pciHole.end = “2200″
hypervisor.cpuid.v0 = FALSE
6. saved and uploaded it to the datastore.
7. turned the vm on - again code 43, so i uninstalled the device on "Device Manager".
8. rescan for new devices and video card suddenly appeared as working properly.



View attachment 4931

View attachment 4932

Hi,

thanks for sharing your steps, with that, I've managed to get it working on ESXi 6.0U3 and I think that it will also work with 6.5.

Just one thing that you need to do with ESXi >5.5 is to use VM version 10, not 11 (create custom VM or edit vmx file).

I don't know what pciHole parameters do, but I can confirm that it works that way.

I'm just testing that on my workstation, but I think that is is same everywhere:
Asus X99-E WS
i7-6800K
Gigabyte GTX 960 4GB (with 387.92 drivers)
VMware ESXi 6.0u3 (build 5050593)
Win 10 64 bit build 1703

This board has PLX PCIe switches so I thought that pciHole mean something with PCIe addressing, but as it is working, I've forgotten about that teory.

PCIe ASMedia USB 3.0 is also working.

Source for "idea" to use VM version 10 is that I tried that long time ago without success. Meantime my friend managed to get it working with AMD cards without any issues, so I knew that nVidia Windows drivers block to run consumer cards within VM, so I tried to fire up macOS 10.12 with nVidia drivers in and it worked, so the main source is guide for macOS unlocker, where they say to use VM verison 10.