Server 2016 & GPU pass-through with Direct Device Assignment (DDA)

Laugh|nGMan

Member
Nov 27, 2012
30
6
8
Just made a topic to collect some info about GPU that works without issues in Hyper-V Guest thru DDA.

I'm tested Quadro K1200 and GTX 750 Ti, both have "Code 43" error in Device manager after first reboot.

Platform: Supermicro x9srl-f + E5-1650 v2
Host: Windows Server 2016
Guest: Windows Server 2016

NVIDIA Quadro Drivers: 372.54
 

RyC

Active Member
Oct 17, 2013
357
89
28
Nvidia GeForce drivers are known to shutdown the GPU and throw Code 43 when they detect it's running in a VM (at least on ESXi). Quadro drivers don't, but I don't remember if it has to be a specific type of Quadro in order to work
 

aero

Active Member
Apr 27, 2016
312
54
28
50
I have found the same thing occurring with Windows guests on KVM host. I said to heck with it and went with ATI/AMD instead.
 

Laugh|nGMan

Member
Nov 27, 2012
30
6
8
Just pitty that NVIDIA in Quadro driver release notes doesn't mention Hyper-V and pass-through supported cards. There is list only for ESXi.
Would be great to test some "low cost" Quadro cards that works thru DDA.
Something like:
Midrange quadro Kepplers solutions - perhaps works. Disadvantage they are too much power hungry imho.
K2200 (similar to GTX 750 Ti in performance) - need to verify...probably works.
M2000 (GTX 950) - don't think so.
M4000 (GTX 960) - perhaps works.
Sure there should not be problems with M6000, GRID K1 K2 and Tesla M60 cards, but that's different price segment.
 

ycp

Member
Jun 22, 2014
175
8
18
Nvidia GeForce drivers are known to shutdown the GPU and throw Code 43 when they detect it's running in a VM (at least on ESXi). Quadro drivers don't, but I don't remember if it has to be a specific type of Quadro in order to work

Are you saying that GeForce cards cant be used in a RemoteFX environment or in DDA environment ? or both?

Because i was thinking about using Geforce cards in VM using RemoteFX.
 

RyC

Active Member
Oct 17, 2013
357
89
28
Are you saying that GeForce cards cant be used in a RemoteFX environment or in DDA environment ? or both?

Because i was thinking about using Geforce cards in VM using RemoteFX.
RemoteFX might work (or at least should work according to Google). But if DDA is anything like PCI passthrough on ESXi/KVM, Nvidia actively blocks GeForce cards from working in that scenario. If it ends up working with DDA now, I wouldn't be surprised if it stops working in a future driver update.
 

Laugh|nGMan

Member
Nov 27, 2012
30
6
8
Just tested Quadro M4000 in DDA enviroment. Looks like works, managed pass-through nvidia adapter. Drivers installed without problems, no issues with reboot.

Enabled Remote Desktop Protocol AVC/H.264 in AVC444 mode + HW encoding when used in conjunction with the AVC444 mode.
 

ycp

Member
Jun 22, 2014
175
8
18
Just tested Quadro M4000 in DDA enviroment. Looks like works, managed pass-through nvidia adapter. Drivers installed without problems, no issues with reboot.

Enabled Remote Desktop Protocol AVC/H.264 in AVC444 mode + HW encoding when used in conjunction with the AVC444 mode.
Hey can you try a regualr geforce card if you have one lying around, in the DDA environment,.

Thanks
 

ycp

Member
Jun 22, 2014
175
8
18
RemoteFX might work (or at least should work according to Google). But if DDA is anything like PCI passthrough on ESXi/KVM, Nvidia actively blocks GeForce cards from working in that scenario. If it ends up working with DDA now, I wouldn't be surprised if it stops working in a future driver update.

Hey,
I have an Intel S2600CP2J motherboard with dual xeon 65-2670. I just tried a GTX 560 with RemoteFX in Windows Server 2016. That works fine.

Now i am going to see if i can try to get DDA to work with this card. But i dont know how to do this. Maybe
Laugh|nGMan can help on how to try this.
 

Laugh|nGMan

Member
Nov 27, 2012
30
6
8
Required beforehand: two VGA (secondary for DDA), remove tick in Hyper-V settings (use this gpu for remote fx).

PowerShell commands:
Code:
$MyNVIDIA = Get-PnpDevice | Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”}
$MyNVIDIA | ft -AutoSize

$MyNVidiaQuadroK1200 = Get-PnpDevice -PresentOnly| Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”} | Where-Object {$_.FriendlyName -eq “NVIDIA Quadro K1200”}
$MyNVidiaQuadroK1200 | ft -AutoSize

Disable-PnpDevice -InstanceId $MyNVidiaQuadroK1200[0].InstanceId -Confirm:$false

$MyNVidiaQuadroK1200 = Get-PnpDevice -PresentOnly| Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”} | Where-Object {$_.FriendlyName -eq “NVIDIA Quadro K1200”}
$MyNVidiaQuadroK1200 | ft -AutoSize

$DataOfGPUToDDismount = Get-PnpDeviceProperty DEVPKEY_Device_LocationPaths -InstanceId $MyNVidiaQuadroK1200[0].InstanceId
$DataOfGPUToDDismount | ft -AutoSize

$locationpath = ($DataOfGPUToDDismount).data[0]
$locationpath | ft -AutoSize

Dismount-VmHostAssignableDevice -locationpath $locationpath -force

Get-VMHostAssignableDevice

Set-VM RFX-WIN2016STD -GuestControlledCacheTypes $True -LowMemoryMappedIoSpace 2000MB -HighMemoryMappedIoSpace 4000MB
Get-VM RFX-WIN2016STD | fl *

Add-VMAssignableDevice -LocationPath $locationpath -VMName RFX-WIN2016STD
 

ycp

Member
Jun 22, 2014
175
8
18
Required beforehand: two VGA (secondary for DDA), remove tick in Hyper-V settings (use this gpu for remote fx).


Ok i tried the GTX 560 but that didnt work so i Tried a GTX 750 ti. And that worked but got Code 43. In the VM. So it looks like the geforce drivers are stopping the device from running in the VM.
 

ycp

Member
Jun 22, 2014
175
8
18
Ok so i tried an AMD Radeon R9 200 series Card and i was able to get it working in Windows 10 VM, but in GPU-Z and CPU-Z no info for the GPU was shown but in Device Manager it shows the card as an OK status.

Also sometimes when i start this VM it makes the Whole server restart. I guess there are some Hardware issues with this card and the server.

If anyone has any tips I have attached some screenshots of the VM
 

Attachments

Last edited:

ycp

Member
Jun 22, 2014
175
8
18
Hey,

I got my hands on some other Graphics Cards. I tried an old Quadro 4000 and it worked perfectly on the first shot with DDA passthrough.

Then i tried a AMD Firepro V7900 and this card doesnt work in a VM with DDA passthrough. The card works fine on the host system but in the VM the driver wont start. The VM boots up fine but the driver wont load. The R9 280X that i tried kept freezing and prevented the VM to boot up. So i think that there is a problem with the current AMD drivers that doesnt allow their cards to work with Passthrough.
If anyone else has any other experiences with other cards please add to this forum.
 

mixtecinc

New Member
Feb 18, 2013
29
0
1
Hey,

I got my hands on some other Graphics Cards. I tried an old Quadro 4000 and it worked perfectly on the first shot with DDA passthrough.

Then i tried a AMD Firepro V7900 and this card doesnt work in a VM with DDA passthrough. The card works fine on the host system but in the VM the driver wont start. The VM boots up fine but the driver wont load. The R9 280X that i tried kept freezing and prevented the VM to boot up. So i think that there is a problem with the current AMD drivers that doesnt allow their cards to work with Passthrough.
If anyone else has any other experiences with other cards please add to this forum.
 

mixtecinc

New Member
Feb 18, 2013
29
0
1
o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks
 
Last edited:

ycp

Member
Jun 22, 2014
175
8
18
o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks

hey

i have never tried any low profile cards. My motherboard doesnt support graphics cards without extra power cable needed. so any low power card i cant test on my motherboard.
 

psannz

Member
Jun 15, 2016
62
14
8
35
o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks
Nvidia Quadro K620 (2GB) and K1200 (4GB) come to mind.
Had a K620 running in a Hyper-V 2012R2 RemoteFX lab myself.

AMD's brand new Radeon Pro WX4100 should fit the bill, too. No idea if the drivers allow for RemoteFX, or installation on Windows Servers in general yet.


:edit: Please keep in mind that your server's PCIe Slots may not be able to supply 75W of power. Dell T110ii for instance only supply up to 25W per PCIe Slot.
 
Last edited:

waith

Member
Aug 3, 2015
39
4
8
40
Anyone successfully passed through a consumer grade AMD card? I'm having some issues getting an RX 470 work in a Windows 10 VM. Freezes, Code 43, or messages that the card isn't present. Card works fine when not passed through.