Help: ESXI 6.5U1 passthrough GTX560ti successfully, but fail on...

[davidng]

I am new to ESXI, recently I came across the following hardware
Supermicro X10DRi + 32G*4 Samsung ECC RAM + E5-2650L V4*2

I want this machine to be my server, gaming pc and workstation at the same time.
So I add MSI GTX560 Ti+ USB3.0 PCIe + SATA3.0 PCIe + 4 port Lan PCIe + Sound Card
I know GTX 560 is an old GPU, but I only got this graphic card.
I have done my research, I install ESXI 6.5 U1 on a USB, and follow the below steps, it works.

esxi host bios
above 4G decoding = disabled (enable also ok)
SR-IOV support = disabled

ssh to vm host, and modify /etc/vmware/passthru.map to
#NVIDIA ( it was 10de ffff bridge false )
10de ffff link false

Create new windows 10 64bit guest VM.
Expose CPU (14 core * 1 socket)
reserved all ram (16G ram)

More guest VM Settings->VM Options->Advanced->Configuration.
hypervisor.cpuid.v0 = FALSE
pciHole.start = 1200
pciHole.end = 2200
pciPassthru0.msiEnabled = FALSE

install windows 10 enterprise
install vm tools
shutdown guest VM

passthrough GTX560 ti
boot up the guest VM, install nVidia 391.01 driver.
reboot again, both vm graphic card and GTX 560 ti are working fine, but sometimes the GTX560 will fail after some mouse move.
I later found out, if I disable the vm graphic card in windows 10 device management, then everything works, the 3d gaming also working.

Then I add more passthrough to this guest VM, like usb host, sound card.
So this is my guest VM final Hardware Configuration
CPU 14 vCPUs
Memory 16 GB
Hard disk 1 64 GB
Hard disk 2 1,024 GB
Network adapter 1 VM Network (Connected)
Video card 4 MB
CD/DVD drive 1 ATAPI
PCI device 0 GF114 [GeForce GTX 560 Ti]
PCI device 1 GF114 HDMI Audio Controller
PCI device 2 Sound Core3D [Sound Blaster Recon3D / Z-Series]
PCI device 3 VL805 USB 3.0 Host Controller
PCI device 4 VL805 USB 3.0 Host Controller



-= Problem =-
When I shutdown my win 10 guest VM (shutdown inside win 10 or using esxi web interface to shut it down)
then start the win 10 guest VM again in esxi web interface, it will try to power up, then it shutdown automatically.
here is some of the important log (I think), I also attached the detail log.

2018-03-18T18:30:41.090Z| vcpu-0| I125: PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:03:00.0
2018-03-18T18:30:41.099Z| vcpu-0| I125: PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:81:00.0
2018-03-18T18:30:41.108Z| vcpu-0| I125: PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:02:00.0
2018-03-18T18:30:41.108Z| vcpu-0| I125: PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:02:00.1
2018-03-18T18:30:41.109Z| vcpu-0| I125: PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:82:00.0

2018-03-18T18:30:41.287Z| vcpu-0| E105: PANIC: PCIPassthruChangeIntrSettings: 0000:81:00.0 failed to register interrupt (error code 195887105)
2018-03-18T18:30:42.112Z| vcpu-0| W115: A core file is available in "/vmfs/volumes/5a9af3a8-eff1cb22-beac-ac1f6b450ede/Win10Enterprise/vmx-zdump.001"
2018-03-18T18:30:42.119Z| vcpu-0| I125: Writing monitor file `vmmcores.gz`

2018-03-18T18:30:53.062Z| vcpu-0| I125: SymBacktrace[12] 0000002c0966fea8 rip=0000000000000000
2018-03-18T18:30:53.062Z| vcpu-0| I125: Msg_Post: Error
2018-03-18T18:30:53.062Z| vcpu-0| I125: [msg.log.error.unrecoverable] VMware ESX unrecoverable error: (vcpu-0)
2018-03-18T18:30:53.062Z| vcpu-0| I125+ PCIPassthruChangeIntrSettings: 0000:81:00.0 failed to register interrupt (error code 195887105)

the strange thing is
1. if I reboot the esxi host, the guest win 10 vm will boot fine.
and if I shutdown the guest win 10 vm, same problem, can not start guest vm again unless I reboot the esxi host again.

2. if I remove all PCI passthrough, and only passthrough the gtx 560 ti and its HDMI audio
PCI device 0 GF114 [GeForce GTX 560 Ti]
PCI device 1 GF114 HDMI Audio Controller
the shutdown guest VM, can not start guest VM again issue resolved.
so this is not the graphic card problem.

3. so the issue seems to be in PCIe sound card or the PCIe usb (I try them individually, all fail, same issue)
MB: Supermicro X10DRi has 2 onboard usb controller, one for usb 3.0 port*5, another one for usb 2.0*4
I passthrough the usb 3.0 controller to win 10 guest VM,
the shutdown guest VM, can not start again issue resolved.
however, if I passthrough the usb 2.0 controller, it happen again, can not boot after shutdown.

4. I later try add this to vm guest vm, no luck
pciPassthru1.msiEnabled = FALSE
pciPassthru2.msiEnabled = FALSE
pciPassthru3.msiEnabled = FALSE
pciPassthru4.msiEnabled = FALSE

5. and also try ssh to vm host
esxcli system module set -m=vmkusb -e=FALSE
or
esxcli system module set -m=vmkusb -e=TRUE
problem persists.

6. and also try vm host bios setting
SR-IOV support= enable
after the first guest VM shutdown, start again, this time, it stuck at the start up proces (no video output).

 

[Ub3r]

Salutations, davidng!

Did you ever figure this out? I just ran across your thread having already tried everything you've stated. I'm also having the USB 3.0 passthrough issue on a gaming VM :-(. I'm passing through a 1070 TI just fine and have a GRID K1 doing vGPU as well without issue on other VMs. When I add the passthrough PCIe USB 3.0 controller for keyboard/mouse I get screwed. I remove it and it boots fine. I've been reduced to using a super shitty program named Synergy which constantly freezes for 5-30seconds at a time to do keyboard/mouse sharing on the VM.

For what it's worth, here's a link to the USB controller I'm using which does not work: https://www.amazon.com/gp/product/B07Z34BJYH/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1