Troubleshooting GPU passthrough ESXi 6.5

Discussion in 'VMware, VirtualBox, Citrix' started by Ch33rios, Jan 2, 2017.

  1. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    First, my build:

    Gigabyte MX31-BS0
    Xeon E3-1230v5
    2x16GB DDR4
    MSI 970 GTX
    Esxi 6.5

    VTd is enabled in the BIOS and best I can tell all other settings are good to go. Boot up the system and go to the web client and on the tab for hardware mgmt I can see the GTX card detected but it is grayed out. Oddly enough the audio controller on the card IS available for passthrough (note I had an HDMI cable plugged into my monitor at the time...Not sure if that makes any difference)

    I know Nvidia is troublesome getting passthrough but all my reading indicated that was more for the driver piece in the guest VM.

    My question here is what can I do, if anything, to further diagnose what COULD be the problem? Any CLI specific things to check (still new to esxi)? I'm also thinking of trying a KVM based distribution like unRAID or Proxmox since it seems people have the most luck in those vs esxi. But obviously before I fully go down those paths I figured I'd try to get some additional diagnostics.

    Thanks in advance!
     
    #1
  2. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,641
    Likes Received:
    357
    I assume you have set Primary Display to sth different than PCI-e?
    And you are not using the GTX to display anything?;)
     
    #2
    Sascha Ha├čler likes this.
  3. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    Nvidia a PITA but i got my 750 TI working after a lot of research and playing.


    1. In the gui enable the HDMI but dont reboot. Copy down the pci info, you will need it for next steps.
    750 ti -1.JPG
    2. you need to enable esxi SSH access then putty into box and edit some settings.
    3. Run the following from putty after logging in as root
    vi /etc/vmware/esx.conf
    4. add under device section the nvidia card. Since you enabled the HDMI you will already see a section for that. you can copy the three lines and modify them for the main card then paste them back in.
    In my case, I had to add the first three lines and update the device id to match what was shown in the gui.
    750 ti -2.JPG
    5. Save the changes and reboot the host. You will eventually lose the esxi boot screen as the card will be passthrough mode.
    6. Log back into the GUI and you will now see both card and hdmi enabled even though the card is still greyed out.

    First part done!!!

    Next when building the VM. I did windows 10 pro. Build out the OS without adding any pci devices. you need to install esxi guest drivers and VNC, I used tightvnc. You should also download Nvidia drivers but dont run them yet. Power off the VM then edit the settings as follows:
    1. Edit CPU and enable Expose hardware assisted virtualization to the guest OS.
    2. Change the memory to reserve all memory.
    htpc settings.JPG
    3. Tab over to VM Options, then expand Advance and click on Edit Configuration
    upload_2017-1-2_15-38-53.png
    4. add parameter: hypervisor.cpuid.v0 with value of FALSE click on ok and save changes.
    config para.JPG
    This part you might need to mess with and try it a few times
    5. Start the VM back up and start the nvidia driver install. it will fail since you havent passed the card yet. but it will expand the files.
    6. shutdown and add the PCI card and hdmi and usb passthought for keyboard to the settings then start it back up.
    7. install the drivers. you might need to reboot, etc..
    8. At this point, you should see the display change over to your vm display. If you do you can then disable the esxi vga card from device management. if you dont it will crash on you. if you need to work on the vm use VNC as the console will not work right.

    Good luck hope this helps.
     
    #3
    Tha_14, das1996, david1028 and 7 others like this.
  4. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    Awesome I will try this ASAP!! Thanks for the detailed steps!

    Just out of curiosity, is this process easier on an AMD GPU? Is Nvidia that finicky?
     
    #4
  5. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    similar process but the AMD seems to play nicer with esxi and I was able to get my r9 290X running with less hassle. The Nvidia is like an exact science to get it working....
     
    #5
  6. RyC

    RyC Active Member

    Joined:
    Oct 17, 2013
    Messages:
    351
    Likes Received:
    83
    With AMD cards, there shouldn't be any need to edit the /etc/vmware/esx.conf or add the vm parameter. Just passthrough and go!
     
    #6
    Dean and Ch33rios like this.
  7. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    Typical :)

    I only really have a GTX 970 at my disposal so until I figure out all the quirks and finalise my setup, I can't justify buying something different.

    Now for your setups, does the GPU 'power down' when the VM it's assigned to powers off? The very first post I read on reddit which talked about virtualizing a gaming setup mentioned how the card didn't even spin up the fans which I sort of read as it being powered down almost completely... Now that might be perhaps a bit unrealistic but I'm wondering if there is any potential for NOT blowing out the power consumption due to a beefy gaming GPU running all the time.
     
    #7
  8. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    @Ch33rios -I just tested it. with two VMs running, one being the HTPC with pass through it idled at 72 watts. When i shutdown the htpc vm it dropped down to 60. So my assumptions is it would drop the power down. but only one way to really tell is testing it out.
     
    #8
  9. nk215

    nk215 Active Member

    Joined:
    Oct 6, 2015
    Messages:
    287
    Likes Received:
    85
    I am not sure if anything was changed from ESXi 6 to ESXi 6.5 but when I tried the pass-thru of GTX card (on ESXi 6 way back), it sort of worked but not completely.

    + Access the VM via RDP worked

    + Monitor plugged into the video out port of the card properly display the VM screen.

    + Access the VM via view client didn't work

    Basically, it works as a multi-head game system but not as VDI remote login via horizon view.

    On my GTX-580, then fan spin with the VM powered down (if I remember correctly). Haven't try it again for a while since I need horizon view for my zero clients.
     
    #9
  10. acmcool

    acmcool Banned

    Joined:
    Jun 23, 2015
    Messages:
    612
    Likes Received:
    76
    Did your audio work well?
     
    #10
  11. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    I havent tested the hdmi yet since the pc still sitting near my desk as i finish the build out but I did pass through the MB audio and that works fine. I plan to use the spdif out to receiver when the pc makes it way back to the tv area. I also got PowerDVD working and tested a blueray movie without issues using the blue-ray drive.
     
    #11
  12. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    on a side note vmtools no longer runs since it doesnt detect the OS running on a virtual host now lol.
    It did run with AMD card.
     
    #12
  13. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,641
    Likes Received:
    357
    #13
    Last edited: Jan 3, 2017
  14. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    I dont believe the 3rd point would be an issue directly for me since my primary use cases are it being a Steam Link streaming host (no real interaction unless Im doing maintenance which would be over RDP anyway) and then possibly down the road a full gaming desktop replacement.

    Power consumption will be something to check...looks like my first Belkin WeMo insight is in my future since I cant seem to directly monitor power consumption on the board (my PSU doesn't have a PMBus plug)
     
    #14
  15. nk215

    nk215 Active Member

    Joined:
    Oct 6, 2015
    Messages:
    287
    Likes Received:
    85
    No I have not tried display emulator. I connected the monitor during my test however. if I remember correctly w/o a monitor attached, AMD driver wont even load properly.

    I currently use a bunch of quadro 4000s as they fit my needs as a cheap grid workaround.

    By the time grid cards get to $6-800 or so I'll just get one.
     
    #15
  16. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    @marcoi - so quick update. I got everything setup I believe and installed the nvidia drivers but Im getting the dreaded Code 43 at the moment. No output to the secondary video output.

    Is there a specific order of operations for ensuring the hypervisor.cpuid.v0 setting is input? Like before the systems very first boot even? Perhaps I need to UNINSTALL vmware tools and just manually install the drivers one by one? Also do I need to do anything with the "Video Card" device in the VM settings? Your's shows custom settings but is it anything special?

    Additionally, I've been unsuccessful in getting ESXi to allow me to pass a USB device connected to the host into a VM. I thought in ESXi 6.x you could just plug a device into a USB slot on the ESXi machine and then in the web client you could physically pass that device to the VM, there-by avoiding any need for an additional PCIe USB controller card...I'll keep trying I guess.
     
    #16
  17. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    988
    Likes Received:
    156
    if the usb device is a HID (keyboard or mouse) it wont pass through. You need to get a usb card either on the MB or PCIE and pass that into the vm with keyboard. if you need usb card this one works. Amazon.com: StarTech.com 2 Port PCI Express PCIe SuperSpeed USB 3.0 Controller Card with SATA Power - Dual Port PCI Express USB 3 Adapter PEXUSB3S23: Computers & Accessories make sure to get the eltron chipset.

    As for 43, that means it still seeing it as a VM. so you might missed a step in the settings.
    1. Edit CPU and enable Expose hardware assisted virtualization to the guest OS.
    2. Change the memory to reserve all memory.
    3. add the hypervisor.cpuid.v0 = FASLE to configuration.

    Also does your GPU init with the pc starts then shows esxi loading then stops mid loading screen? If not you dont have the GPU set to primary in the bios. if you have CPU GPU you need to disable it and select PEG (pcie graphics) which will use your main video card. Not sure why this is needed but it wouldnt work for me with IGP enabled.
     
    #17
  18. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    Ok that last part I definitely have screwed up. When I first went down this path I changed the primary display to the on-board graphics (changed away from PCIe)...perhaps this would explain some of the other strange things I've been experiencing. I'll change that for sure and try again. Having the USB keyboard attached shouldn't be a requirement to at least get the GPU passed through (I wouldn't think it would anyway...cant still do RDP or tightVNC as was previously mentioned).

    And thanks for the add-on recommendation, one-day shipping FTW!
     
    #18
  19. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    Well after another hour or so of messing around and double/triple checking my setup and reinstalling Windows (win10 enterprise) from scratch, Im unhappy to report no further progress. Still Code 43

    Tried EFI/BIOS
    Tried NOT installing VM tools

    I know the card is being passed to the VM because on boot of said VM it spins the GPU fan up like its under full load and then turning the VM off puts it back to a normal state of quietness (like its getting power but not doing anything). Most people seem to be successfully doing this on ESXi 6 where as I'm on 6.5....possibility?

    Im going to take a break for tonight and recheck things again maybe tomorrow or Friday. I'll also bite the bullet and pull my current desktops 970 GTX out and stick that in there to see how IT fairs. Who knows but its worth a shot :)
     
    #19
    Last edited: Jan 5, 2017
  20. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    2,641
    Likes Received:
    357
    Not the same use case but when I tried passing my GTX1080 the only thing I did was to enable the passthrough and add the hypervisor line
    hypervisor.cpuid.v0 = "FALSE" (correctly written and with quotes;)).
    That passed the GPU. I then deactivated the SVGA card from vmware tools in hardware to make sure it wouldnt be in use.
    No fiddling around in bios at all (primary display set to AST2400).

    3dMark didnt run either (config check failed, it never got to run a 3d load) and when I couldnt connect via view i stopped messing around so am not sure whether it would work with a monitor.
     
    #20
Similar Threads: Troubleshooting passthrough
Forum Title Date
VMware, VirtualBox, Citrix troubleshooting vmware Aug 2, 2017
VMware, VirtualBox, Citrix GPU Passthrough - ESXi 6.7.0 - AMD and NVIDIA Friday at 8:58 AM
VMware, VirtualBox, Citrix Chelsio T440 Quad port 10G SFP+ Passthrough Oct 31, 2018
VMware, VirtualBox, Citrix ESXi 6.7 on Ryzen x399 for VMs + GPU passthrough desktop Jun 13, 2018
VMware, VirtualBox, Citrix ESXI 6.7 GPU PassThrough and "Expose hardware assisted virtualization to the guest OS" not working. May 22, 2018

Share This Page