Server 2016 & GPU pass-through with Direct Device Assignment (DDA)

[Laugh|nGMan]

Just made a topic to collect some info about GPU that works without issues in Hyper-V Guest thru DDA.

I'm tested Quadro K1200 and GTX 750 Ti, both have "Code 43" error in Device manager after first reboot.

Platform: Supermicro x9srl-f + E5-1650 v2
Host: Windows Server 2016
Guest: Windows Server 2016

NVIDIA Quadro Drivers: 372.54

 

[cesmith9999]

Discrete Device Assignment — GPUs

Chris

 

[RyC]

Nvidia GeForce drivers are known to shutdown the GPU and throw Code 43 when they detect it's running in a VM (at least on ESXi). Quadro drivers don't, but I don't remember if it has to be a specific type of Quadro in order to work

 

[aero]

I have found the same thing occurring with Windows guests on KVM host. I said to heck with it and went with ATI/AMD instead.

 

[Laugh|nGMan]

Just pitty that NVIDIA in Quadro driver release notes doesn't mention Hyper-V and pass-through supported cards. There is list only for ESXi.
Would be great to test some "low cost" Quadro cards that works thru DDA.
Something like:
Midrange quadro Kepplers solutions - perhaps works. Disadvantage they are too much power hungry imho.
K2200 (similar to GTX 750 Ti in performance) - need to verify...probably works.
M2000 (GTX 950) - don't think so.
M4000 (GTX 960) - perhaps works.
Sure there should not be problems with M6000, GRID K1 K2 and Tesla M60 cards, but that's different price segment.

 

[ycp]

Nvidia GeForce drivers are known to shutdown the GPU and throw Code 43 when they detect it's running in a VM (at least on ESXi). Quadro drivers don't, but I don't remember if it has to be a specific type of Quadro in order to work

Are you saying that GeForce cards cant be used in a RemoteFX environment or in DDA environment ? or both?

Because i was thinking about using Geforce cards in VM using RemoteFX.

 

[RyC]

Are you saying that GeForce cards cant be used in a RemoteFX environment or in DDA environment ? or both?

Because i was thinking about using Geforce cards in VM using RemoteFX.

RemoteFX might work (or at least should work according to Google). But if DDA is anything like PCI passthrough on ESXi/KVM, Nvidia actively blocks GeForce cards from working in that scenario. If it ends up working with DDA now, I wouldn't be surprised if it stops working in a future driver update.

 

[Laugh|nGMan]

Just tested Quadro M4000 in DDA enviroment. Looks like works, managed pass-through nvidia adapter. Drivers installed without problems, no issues with reboot.

Enabled Remote Desktop Protocol AVC/H.264 in AVC444 mode + HW encoding when used in conjunction with the AVC444 mode.

 

[ycp]

Just tested Quadro M4000 in DDA enviroment. Looks like works, managed pass-through nvidia adapter. Drivers installed without problems, no issues with reboot.

Enabled Remote Desktop Protocol AVC/H.264 in AVC444 mode + HW encoding when used in conjunction with the AVC444 mode.

Hey can you try a regualr geforce card if you have one lying around, in the DDA environment,.

Thanks

 

[ycp]

RemoteFX might work (or at least should work according to Google). But if DDA is anything like PCI passthrough on ESXi/KVM, Nvidia actively blocks GeForce cards from working in that scenario. If it ends up working with DDA now, I wouldn't be surprised if it stops working in a future driver update.

Hey,
I have an Intel S2600CP2J motherboard with dual xeon 65-2670. I just tried a GTX 560 with RemoteFX in Windows Server 2016. That works fine.

Now i am going to see if i can try to get DDA to work with this card. But i dont know how to do this. Maybe
Laugh|nGMan can help on how to try this.

 

[Laugh|nGMan]

Required beforehand: two VGA (secondary for DDA), remove tick in Hyper-V settings (use this gpu for remote fx).

PowerShell commands:

$MyNVIDIA = Get-PnpDevice | Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”}
$MyNVIDIA | ft -AutoSize

$MyNVidiaQuadroK1200 = Get-PnpDevice -PresentOnly| Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”} | Where-Object {$_.FriendlyName -eq “NVIDIA Quadro K1200”}
$MyNVidiaQuadroK1200 | ft -AutoSize

Disable-PnpDevice -InstanceId $MyNVidiaQuadroK1200[0].InstanceId -Confirm:$false

$MyNVidiaQuadroK1200 = Get-PnpDevice -PresentOnly| Where-Object {$_.Class -eq “Display”} | Where-Object {$_.Service -eq “nvlddmkm”} | Where-Object {$_.FriendlyName -eq “NVIDIA Quadro K1200”}
$MyNVidiaQuadroK1200 | ft -AutoSize

$DataOfGPUToDDismount = Get-PnpDeviceProperty DEVPKEY_Device_LocationPaths -InstanceId $MyNVidiaQuadroK1200[0].InstanceId
$DataOfGPUToDDismount | ft -AutoSize

$locationpath = ($DataOfGPUToDDismount).data[0]
$locationpath | ft -AutoSize

Dismount-VmHostAssignableDevice -locationpath $locationpath -force

Get-VMHostAssignableDevice

Set-VM RFX-WIN2016STD -GuestControlledCacheTypes $True -LowMemoryMappedIoSpace 2000MB -HighMemoryMappedIoSpace 4000MB
Get-VM RFX-WIN2016STD | fl *

Add-VMAssignableDevice -LocationPath $locationpath -VMName RFX-WIN2016STD

 

[ycp]

Required beforehand: two VGA (secondary for DDA), remove tick in Hyper-V settings (use this gpu for remote fx).

Ok i tried the GTX 560 but that didnt work so i Tried a GTX 750 ti. And that worked but got Code 43. In the VM. So it looks like the geforce drivers are stopping the device from running in the VM.

 

[ycp]

Ok so i tried an AMD Radeon R9 200 series Card and i was able to get it working in Windows 10 VM, but in GPU-Z and CPU-Z no info for the GPU was shown but in Device Manager it shows the card as an OK status.

Also sometimes when i start this VM it makes the Whole server restart. I guess there are some Hardware issues with this card and the server.

If anyone has any tips I have attached some screenshots of the VM

 

[ycp]

Hey,

I got my hands on some other Graphics Cards. I tried an old Quadro 4000 and it worked perfectly on the first shot with DDA passthrough.

Then i tried a AMD Firepro V7900 and this card doesnt work in a VM with DDA passthrough. The card works fine on the host system but in the VM the driver wont start. The VM boots up fine but the driver wont load. The R9 280X that i tried kept freezing and prevented the VM to boot up. So i think that there is a problem with the current AMD drivers that doesnt allow their cards to work with Passthrough.
If anyone else has any other experiences with other cards please add to this forum.

 

[mixtecinc]

Hey,

I got my hands on some other Graphics Cards. I tried an old Quadro 4000 and it worked perfectly on the first shot with DDA passthrough.

Then i tried a AMD Firepro V7900 and this card doesnt work in a VM with DDA passthrough. The card works fine on the host system but in the VM the driver wont start. The VM boots up fine but the driver wont load. The R9 280X that i tried kept freezing and prevented the VM to boot up. So i think that there is a problem with the current AMD drivers that doesnt allow their cards to work with Passthrough.
If anyone else has any other experiences with other cards please add to this forum.

 

[mixtecinc]

o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks

 

[ycp]

o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks

hey

i have never tried any low profile cards. My motherboard doesnt support graphics cards without extra power cable needed. so any low power card i cant test on my motherboard.

 

[psannz]

o you know of any low profile cards that will work with RemoteFX and Windows Server 2016?

Thanks

Nvidia Quadro K620 (2GB) and K1200 (4GB) come to mind.
Had a K620 running in a Hyper-V 2012R2 RemoteFX lab myself.

AMD's brand new Radeon Pro WX4100 should fit the bill, too. No idea if the drivers allow for RemoteFX, or installation on Windows Servers in general yet.


:edit: Please keep in mind that your server's PCIe Slots may not be able to supply 75W of power. Dell T110ii for instance only supply up to 25W per PCIe Slot.

 

[mirrorneuron]

if you can prevent the driver somehow from detecting it is in a VM, that could help with the geforce cards
, i searched a little bit and found some things about how they do it on vmware

Can you hide virtual machine from a program? | VMware Communities

maybe someone knows how to do that in hyper-v?

 

[waith]

Anyone successfully passed through a consumer grade AMD card? I'm having some issues getting an RX 470 work in a Windows 10 VM. Freezes, Code 43, or messages that the card isn't present. Card works fine when not passed through.