The Official NVIDIA GRID M40 Thread

[Patrick]

There are a number of folks on the forums working with the NVIDIA GRID M40 cards. This will be the official thread for these cards going forward.


STH NVIDIA GRID M40 Resources

• The main site NVIDIA GRID M40 Reference
• nvidia-smi issues? Setting BIOS settings for the NVIDIA GRID M40

Additional resources

• Trying 32 GPUs (8x 4GPU cards) in the ASRock Rack 8 GPU server

Where to get them

Currently the best place to find these cards is via ebay. Here is a GRID M40 ebay search Given what we have seen, do not bid more than $1000 for these cards.

 

[gigatexal]

n00b question warning: what are owners' use cases here for these cards. CAD/VDI/something else?

 

[Patrick]

@whitey and @markpower28 are still doing work on the VDI side. We seem to be missing the required driver and lack hacking skills to get us there.

Now, if you needed lots of h264 transcoding (NVIDIA accelerated) or wanted to try out a 4 or 8 node machine learning cluster, these are probably the lowest power, CUDA compute 5.0 cards on the market. Two cards ~130w each (max) and you have an 8x GPU compute cluster. Very useful if you were running containerized applications and wanted to run different models on different numbers of GPUs or wanted to see how well an application scales with larger GPU counts.

If you were to compare, each of these GPUs is about 1.4TFLOPS (single precision) so you have more TFLOPs per watt, and a lower price point than the new Tesla M4 card, plus you can cram them in a system.

 

[Patrick]

Updated with the tome of knowledge around the NVIDIA GRID M40.

 

[Figi]

Hey All, just wondering if there was much progress around the M40 and VDI or using the card as a shared PCI device under esxi. I managed to get my hands on one and I have been tinkering around to see if I can get this to work even know it is not officially supported by VMware.

Thanks

 

[Patrick]

Currently stuck without a driver. I have made the request to NVIDIA.

 

[badskater]

Hi Patrick,

just to know, are we using the Tesla M40 driver for this one, or another driver, since i looked in the GRID drivers, and none for the M40 ?

Thanks,

 

[Patrick]

Hi Patrick,

just to know, are we using the Tesla M40 driver for this one, or another driver, since i looked in the GRID drivers, and none for the M40 ?

Thanks,

We have gotten the CUDA drivers to work. I believe where we are stuck is the drivers required for vDGA, vGPU and etc. that you normally can get working with GRID cards.

Compute wise they work well, VDI wise they work less well.

 

[badskater]

Thanks for that information. Also, it's more to get stuff working thru Windows. So it's perfect for my use. For VDI, i'm gonna wait on more infos.

 

[chune]

Patrick- I'm sure you know this, but just wanted to confirm 100% that you tried this. Above 4g decoding has to be DISABLED for vDGA to work. Can you try booting a blade using this bios setting with just one grid card attached? This way it should at least let you passthrough a GPU to the VM and boot it, however this still probably won't be much help until there is a suitable windows driver available. Also, I would be doing all the passthrough config from the c# client connecting directly to the esxi host rather than through the vcenter web gui.

The only other thing i can think of is that there is some sort of baked-in grid 2.0 licensing protection on newer esxi (and/or vcenter) releases. Just for jokes, you should try booting the above config in esxi 5.0 u3, 5.5 u3b, and 6.0 u2

For vSGA I'm sure we are completely at the mercy of nvidia/vmware for whatever draconian licensing models they concoct, but you would think we could at least get vDGA working here. What happens when you boot a physical box with one of these? No generic drivers install or anything?

 

[Patrick]

@chune want to give it a try?

 

[chune]

yes please

 

[chune]

so heres an update from my end:
I was able to get the cards configured for pci passthrough to test vDGA. The vm boots with the GPU attached however we do not have a windows driver. This is presumably due to the fact that all cards ship in compute mode and need to be switch to gpu mode using the gpumodeswitch utility. Unfortunately the utility just errors out when you try to access these cards. I have tried it using the windows tool in a VM with a GPU passed through and using the VIB method on vsphere. The recommended method is to use the supposedly bootable ISO that comes with the gpumodeswitch utility but for the life of me i could not get the IPMI to boot from it. I'm going to try making an IMA once i have some more free time but i have a feeling that the gpumodeswitch utility does some sort of device ID check before it will let you run it. Might be time to break out the soldering iron! [MOVED] Hacking NVidia Cards into their Professional Counterparts - Page 1

 

[Chuntzu]

Does this you have 4x nvenc on this one card? Does gpu have support for her h265 encoding support with its nvenc? Looking at these for some super fast h265 reencoding.

Sent from my SM-N920T using Tapatalk

 

[Chuntzu]

Looks to be gm107 chip so no h265 but 480fps for h264, please correct me if I'm wrong.

Sent from my SM-N920T using Tapatalk

 

[Patrick]

Looks to be gm107 chip so no h265 but 480fps for h264, please correct me if I'm wrong.

Sent from my SM-N920T using Tapatalk

I think you are right on this. I have a feeling that is what the big web 2.0 company that used these cards was using them for.

 

[Chuntzu]

Haha makes sense these are pretty awesome for that use specifically

Sent from my SM-N920T using Tapatalk

 

[gigatexal]

Is handbrake GPU aware? How would you harness this for media encoding?

 

[Patrick]

Is handbrake GPU aware? How would you harness this for media encoding?

One example: http://developer.download.nvidia.co...with-NVIDIA-Acceleration-on-Ubuntu_UG_v01.pdf

 

[Chuntzu]

Staxrip is fantastic and works great. Ffmpeg is how I'm currently using quick sync and nvenc from my laptop.

Sent from my SM-N920T using Tapatalk