Tesla P4 - 8GB / US$ 100 or even lower

[efschu3]

Would you mind tell me how can I do that? right now I have

args: -device 'vfio-pci,sysfsdev=/sys/bus/mdev/devices/<mdev-uuid>,display=off,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-pci-vendor-id=0x10de,x-pci-device-id=0x1bb0' -uuid <mdev-uuid>

in my proxmox vm config. if i want to pass through P4 directly, how should I spoof it?

thanks.

You can do it on the Webinterface:


Select your card, fill in the IDs - done

 

[63z5mgng]

You can do it on the Webinterface:

Damn, never pay attention on the webui. Thanks a lot.

 

[gizzard]

You can do it on the Webinterface:
View attachment 25477

Select your card, fill in the IDs - done

So, if you pass through the P4 as a P4, by default the card is in TCC mode in the VM? If you pass through the P4 spoofed as a Quadro, the card is in WDDM mode?

Earlier in the thread:

NVIDIA Tesla cards in particular can bet set to compute-only (TCC) and graphics (WDDM) modes, and default to TCC out-of-the-box. I believe you will need the GRID vGPU driver to put them in WDDM mode, and the regular Quadro driver does not let you do that.

 

[efschu3]

Well, I would test it. You cannot use Quadro Driver with non-spoofed Grid cards. But if spoofed, I think it'll work.

 

[theyost]

I am able to install the Quadro P5000 driver for a spoofed Tesla P4
.. and at first Device Manager says "working properly"
.. but on reboot it shows the dreaded "Error 43"

Any ideas?

Here is my Proxmox PCI Setup (Spoofed as P5000):



I have been trying to install on Windows 11 and maybe this is my problem.
My next attempt probably will be on Windows 10.

 

[Clampazzo]

I was able to get this to work as a spoofed P4000, using the below method, but I also had this configure for vGPU, it should work similarly without vGPU. Also, would use DDU to wipe the old drivers, and re-install. Sometimes the newer drivers are pickier too, try 513.46, or 512.78

maybe you can try and use the sub-device ID 11B2, like found here, NVIDIA Quadro P5000 VBIOS
Would be curious to hear if passing through as P5000 works.


Has anyone had luck with this card in Ubuntu and using nvenc? I have gotten it to work but often it crashes when transcoding with Plex, especially with multiple streams. Curious if specific drivers might work better, or a good guide exists that I have not found yet.

 

[iGene]

I was able to get this to work as a spoofed P4000, using the below method, but I also had this configure for vGPU, it should work similarly without vGPU. Also, would use DDU to wipe the old drivers, and re-install. Sometimes the newer drivers are pickier too, try 513.46, or 512.78
View attachment 25796
maybe you can try and use the sub-device ID 11B2, like found here, NVIDIA Quadro P5000 VBIOS
Would be curious to hear if passing through as P5000 works.


Has anyone had luck with this card in Ubuntu and using nvenc? I have gotten it to work but often it crashes when transcoding with Plex, especially with multiple streams. Curious if specific drivers might work better, or a good guide exists that I have not found yet.

It seems like Windows version will also affect if the drivers work. I once have a VM with a working driver (forgot the exact version), after Windows update it became unusable (code 43).

Latest Windows 10/Windows 11 release seems not working with 513.46 and 512.78 at my side. Still trying to find the best combination available.

 

[Clampazzo]

It seems like Windows version will also affect if the drivers work. I once have a VM with a working driver (forgot the exact version), after Windows update it became unusable (code 43).

Latest Windows 10/Windows 11 release seems not working with 513.46 and 512.78 at my side. Still trying to find the best combination available.

Yeah, I tried direct passthrough to Windows 10 and I had zero luck getting the spoof to work. Was a Windows 10 20H1 install if I recall, but no major updates.

The best way seems to be using the Grid / vgpu unlock (at least for windows), I followed this guide without patching the driver and it worked for windows but spoofed to a P4000, but I got troubles trying to also run an ubuntu vm for nvenc, PolloLoco / NVIDIA vGPU Guide · GitLab

Direct PCI passthrough to ubuntu without spoofing worked better using the grid guest drivers, surprisingly.

 

[FlorianZ]

I am using one in direct passthrough to a Windows 2019 guest with the data center driver installed. Afaik, that driver supports TCC only, but I am using it for AI inference which seems to work well. Just throwing this out there as another option.

 

[iGene]

Yeah, I tried direct passthrough to Windows 10 and I had zero luck getting the spoof to work. Was a Windows 10 20H1 install if I recall, but no major updates.

The best way seems to be using the Grid / vgpu unlock (at least for windows), I followed this guide without patching the driver and it worked for windows but spoofed to a P4000, but I got troubles trying to also run an ubuntu vm for nvenc, PolloLoco / NVIDIA vGPU Guide · GitLab

Direct PCI passthrough to ubuntu without spoofing worked better using the grid guest drivers, surprisingly.

I'm also using GRID/vgpu-unlock, but I couldn't get the drivers worked when spoofing the vGPU as either P4000 or P5000. Can you share the Windows version and the driver version you are using? Thanks!

 

[Clampazzo]

I'm also using GRID/vgpu-unlock, but I couldn't get the drivers worked when spoofing the vGPU as either P4000 or P5000. Can you share the Windows version and the driver version you are using? Thanks!

Sure thing! curiously it shows up as a Quadro P4000 in device manager but the Nvidia drivers do still detect its a Grid-P4-4Q. Used the same settings as above for the PCI device. Will have to test it out for a bit see if it fails later, along with some windows updates.

OS: Windows 10 21H2 19044.2251
Driver: 513.46-quadro-rtx-desktop-notebook-win10-win11-64bit-international-dch-whql
Edit: Also, on Proxmox 7.3-3 - fresh install for just this

I also have this in the profile_override.toml

[profile.nvidia-65]


[mdev.00000000-0000-0000-0000-000000000100]
#frl_enabled = 60
pci_device_id = 0x1BB1
pci_id = 0x1BB111A3

 

[gb00s]

I suggest staying < 512.xx drivers ... like 511.65 >> LINK

EDIT: You can always ask here on DISCORD

 

[chereszabor]

What is everyone using to cool this card?

I recently received mine and printed the following 40mm fan shroud: Tesla T4/P4 to 40mm Fan adapter by klipper

Using a Noctua NF-A4x20 running at full speed, the card still hits ~70C under 50% load, i have not tried going further...

Thanks!

 

[FlorianZ]

What is everyone using to cool this card?

I use the same or similar 3D printed shroud, but the Noctua 40mm fan definitely won't cut it. I am using a 1U server-style 40x28mm fan. Mine doesn't need to run at full tilt to keep this card cool.

 

[Markess]

It's now sitting in my Pop_OS 22.04 LTS desktop (that's Linux btw, for you Windows users) along side my VEGA64 and next to my dual PM1725.

eduncan911 , if I read this right, you're using the P4 as the primary graphics device on a bare metal POP OS desktop install, and routing to the monitor via a second device (the Vega 64?)?

I was wanting to try this on a Linux desktop as well, mostly to see if I can and what the performance is like. I was thinking of using my i5-12600K's iGP, or if that doesn't work a Quadro P620 that's not in use right now. But, while I'm finding tons of stuff on how to make this work in Windows, I can't seem to find much on the Linux side outside of virtualization. So, knowing if someone has already gotten this work would help!

 

[jycai]

I was able to get this to work as a spoofed P4000, using the below method, but I also had this configure for vGPU, it should work similarly without vGPU. Also, would use DDU to wipe the old drivers, and re-install. Sometimes the newer drivers are pickier too, try 513.46, or 512.78
View attachment 25796
maybe you can try and use the sub-device ID 11B2, like found here, NVIDIA Quadro P5000 VBIOS
Would be curious to hear if passing through as P5000 works.


Has anyone had luck with this card in Ubuntu and using nvenc? I have gotten it to work but often it crashes when transcoding with Plex, especially with multiple streams. Curious if specific drivers might work better, or a good guide exists that I have not found yet.

Hi Clampazzo,
Can you tell me the steps to get P4 working in Proxmox and Ubuntu? I followed this guide - PolloLoco / NVIDIA vGPU Guide · GitLab and got winows 10/11/2022 server all working with spoofing to Quadro P4000 as you did.

I have tried to run Ubuntu 22.04 with spoof/not spoof, but the vm crashed each time with all the different guest nvidia drivers I installed on Ubuntu. Did you got Ubuntu working with spoofing the T4 to Quadro P4000?

Thanks.

 

[Clampazzo]

Hi Clampazzo,
Can you tell me the steps to get P4 working in Proxmox and Ubuntu? I followed this guide - PolloLoco / NVIDIA vGPU Guide · GitLab and got winows 10/11/2022 server all working with spoofing to Quadro P4000 as you did.

I have tried to run Ubuntu 22.04 with spoof/not spoof, but the vm crashed each time with all the different guest nvidia drivers I installed on Ubuntu. Did you got Ubuntu working with spoofing the T4 to Quadro P4000?

Thanks.

Hi @jycai, I don't recall having any issues in ubuntu with the vm crashing on boot up. I only had crashing with Plex, but this could also be the 4k streams I was attempting to transcode. Though I could not get any spoof to work, and what I read this is not possible in linux distros. It is possible that the efi / legacy boot method in your install could be part of the issue. IIRC, if your proxmox install is UEFI then your guests also need to configured to use UEFI.

In my system now the proxmox install is legacy, so my vm configuration is very basic, just the default for everything and pci-device + mdev, it gets detected with nvidia-smi after installing the grid guest drivers, and blacklisting the nouveau driver. I haven't had trouble with 22.04(updated to 22.10), or 20.04. The mdev type is mostly only important for compatibility, I found for linux C or Q types worked best. I don't know that any Quadro driver will work correctly in linux as it will still detect quadro.

 

[jycai]

Hi @jycai, I don't recall having any issues in ubuntu with the vm crashing on boot up. I only had crashing with Plex, but this could also be the 4k streams I was attempting to transcode. Though I could not get any spoof to work, and what I read this is not possible in linux distros. It is possible that the efi / legacy boot method in your install could be part of the issue. IIRC, if your proxmox install is UEFI then your guests also need to configured to use UEFI.

In my system now the proxmox install is legacy, so my vm configuration is very basic, just the default for everything and pci-device + mdev, it gets detected with nvidia-smi after installing the grid guest drivers, and blacklisting the nouveau driver. I haven't had trouble with 22.04(updated to 22.10), or 20.04. The mdev type is mostly only important for compatibility, I found for linux C or Q types worked best. I don't know that any Quadro driver will work correctly in linux as it will still detect quadro.

View attachment 25982
View attachment 25983

Thank you Clampazzo! I changed the VM from UEFI to default and follow this instruction Install drivers for NVIDIA RTX virtual workstations | Compute Engine Documentation | Google Cloud to install the grid driver in ubuntu 22.04, nvidia-smi is working and no crash after the reboot.

Since I am running the grid driver without license, it should only work for 20 minutes or so. Do you have the script to make it run for 24 hours?

 

[63z5mgng]

Found this windows driver and it claims working out of box without any spoof or limitation.

513.46_grid_win10_win11_server2019_server2022_64bit_international.exe官方版下载丨最新版下载丨绿色版下载丨APP下载-123云盘

123云盘为您提供513.46_grid_win10_win11_server2019_server2022_64bit_international.exe最新版正式版官方版绿色版下载,513.46_grid_win10_win11_server2019_server2022_64bit_international.exe安卓版手机版apk免费下载安装到手机,支持电脑端一键快捷安装

www.123pan.com

code: D05E

 

[jycai]

Found this windows driver and it claims working out of box without any spoof or limitation.

513.46_grid_win10_win11_server2019_server2022_64bit_international.exe官方版下载丨最新版下载丨绿色版下载丨APP下载-123云盘

123云盘为您提供513.46_grid_win10_win11_server2019_server2022_64bit_international.exe最新版正式版官方版绿色版下载,513.46_grid_win10_win11_server2019_server2022_64bit_international.exe安卓版手机版apk免费下载安装到手机,支持电脑端一键快捷安装

www.123pan.com

code: D05E

Fake driver. The frame rate on jellyfish webgl dropped from 60 to 15 after 20 minutes.