Dell Poweredge R730 & Rack 7910 Riser GPU power pinout question

[What_Is_Computer]

Been getting ready to mess around with a Poweredge R730 and Rack 7910 today. I was verifying the pinouts on the riser and found something odd. Sense pin B appears to be putting out -3.3V. The official Dell power cable has a jumper wire going between sense pin A to sense pin B. This wiring seems odd to me. Plus this is the first time I have seen any voltage on a PCI Express power pin that isn't +12V. I am no expert, and I seem to be striking out on google. Is this the "proper" way to use the sense pin? I'm guessing if I dig hard enough I could find the official power pinout standards for PCI express, but google is fighting me.

Also my pinout image is if you are looking directly at the riser connector. If you flip the image on the vertical axis that would be the pinout of the cable.

 

[RolloZ170]

All Dell power interposer board's have a current sense chip with PMBus interface.
this information can be used to detect existance of GPU and verify for proper power supply/addtional FANs.
i.e. on a T640: if you install two GPU you have to install two air shrouds and add. FAN's to get the IDRAC errors/warnings away.

 

[What_Is_Computer]

Interesting. Until recently most of the GPU power cables available were aftermarket ones. The aftermarket ones aren't wired like the genuine Dell cables.

On the 12th gen Dells the GPU power connectors don't have the -3.3V pin. The cables that I have been making that are wired straight through have been working. Although my R720 won't ramp up the fans with my Tesla M40 like it did with the Tesla K80. I was under the impression that was a firmware issue, not a power sense issue.

 

[RolloZ170]

be very careful with the Dell cables. if got a T640 system board and wanted to run it.
because the T640s power interposer boards are sold by scaplers i thought it can be powered with a T630 PIB.
bought one but found the pinouts are very different. now it works, got a PIB from official Dell distributor little cheaper.

 

[What_Is_Computer]

The biggest problem I found was the aftermarket ones aren't keyed correctly for PCI express power. Tried plugging one into my Tesla K80, and basically shorted 12V to ground since it requires EPS 12V. The notch between pins 7 and 8 was missing to stop me from doing that. I will probably have to do more research and testing if I want to make more of my own power cables for the newer stuff.

 

[mobilenvidia]

Be vey careful, frying Mobo is very easy to do, says he who did it with a Dell SC8000 (Driveless R720, but crammed with HBA's)
I had a Tesla M60 running in it (eventually), and it ran very well after some tweaking of air flow
I needed to use both Power plugs on the PCIE risers to make up the 8pin cable to plug into M60
I'll see if I can find what I had

 

[mobilenvidia]

Dang, last 3 years has not been kind on my memory, this sytem got scorched in the Wildfire in 2020, as I chose to rescue my Data server instead

But Dell do a GPU Enablement Kit
I based my cables on this, the M60 had a 8pin power plug and needed some special attention from I think using both Riser board power plugs going to a single GPU 8pin plug

But be super careful and measure each pin voltage to make sure the cable(s) will be correct

Here some more info on the kit


GPU card installation guidelines

• Ensure that both the processors are installed.
• The processor must be of 135 W or less.
• The processor must use a GPU kit low-profile heat sink.
• Due to the high power consumption of GPUs, the ambient inlet temperature is restricted to 30°C to ensure adequate cooling when one or more GPU cards are installed.
• Ensure that the GPU enablement kit is available.
  The GPU enablement kit includes:
  • Low-profile heat sinks
  • Power cables for the GPU cards
  • Filler brackets with closeout EMI shield for unoccupied PCIe slots
• All GPU cards must be of the same type and model.
• You can install up to two double-wide GPU cards.
  NOTE Two double-wide GPU cards configuration requires optional riser 3.
• You can install up to four single-wide GPU cards.
• Ensure that your system uses the redundant 1100 W power supplies.
  NOTE When using the system with the Nvidia K80 GPU card, ensure that you install both PSUs with a minimum of 1100 W each and set the PSU configuration to non-redundant mode.

 

[mobilenvidia]

8pin Riser cable to to 2x 6pin GPU cable for R720/730
https://www.amazon.com/dp/B06W56XMHP

8pin Riser to 8 pin GPU cable
https://www.amazon.com/Power-Cable-DELL-R730-Nvidia/dp/B08NX2YFGR

From memory do NOT use the 6+2 pin cable on 8 pin GPU's

 

[What_Is_Computer]

8pin Riser cable to to 2x 6pin GPU cable for R720/730
https://www.amazon.com/dp/B06W56XMHP

8pin Riser to 8 pin GPU cable
https://www.amazon.com/Power-Cable-DELL-R730-Nvidia/dp/B08NX2YFGR

From memory do NOT use the 6+2 pin cable on 8 pin GPU's

Some minor concerns with those cables. Neither one of them have the jumper wire going from sense pin A to sense pin B. I am not sure how much that matters with the 13th gen and newer servers. I haven't tried one of my cables on a R730 yet. It shouldn't be a problem for 12th gen from my experience.

The cable for Tesla EPS 12V support doesn't indicate which way it goes. It could be keyed to force the user to use it a specific way, but it isn't. Also the labeling doesn't indicate what goes where.

 

[mobilenvidia]

Triple check before using, I doubt a R730 Mobo will be cheap if you get it wrong, but cheaper than a Tesla

 

[What_Is_Computer]

I do know from experience that the server will try to protect itself if you short 12V to ground on the riser. When I first got my Tesla K80 I didn't realize it was EPS 12V, and the low quality aftermarket cable wasn't keyed correctly to stop me. That is when I learned about the notch on PCIe 8-pin power connectors... My R720 is still running luckily. I may try risking a R730 for the sake of science.

I don't trust these aftermarket parts sellers and their compatibility claims. It must work if they are selling them though.

 

[Dareposte]

I know this is a bit of an old post, but I just ran into this myself and figured may as well post for others that have this problem. Many of the 3rd party cables seem to be wired for EPS12v to PCIe / 8-pin, but they just honor +12v and ground without really connecting the sense pins properly. For an R730 you need the sense pins to be connected as well.

I've got a mess of cables, some worked and some didn't, so I rang them all out and figured out that pretty much just a few pins really matter to enable the GPU.

EPS Pin 1 must have continuity to PCIe pins 8 and 4
EPS Pin 4 must have continuity to PCIe pin 6

In most cases, it was EPS/4 to PCIe/6 that was causing the cables to not work for me, and swapping them around got my GPU's powered up.

Attached are some visuals for what worked for me.

 

[kroem]

I'm totally confused. I have a R720 that I wanna run a P40 in, or two maybe.

I wont be able to use a normal CPU 8-pin EPS12v cable from some modular PSU?

I've hooked it up, PSU is showing amber so I guess something is wrong - might be some limitation, but I might also have the wrong cable

 

[Dareposte]

Generally no, this is a good way to fry your card, your motherboard, or both, and is unlikely to work.

At a minimum please get or borrow a cheap multimeter and ring out the cables to make them match the document uploaded a couple posts above this. If the cable you have physically fits into the plugs on both ends and has the right number of wires, you can de-pin the wires and move them around to build a cable that works, but if you haven't de-pinned a micro connector before you are in for a fun time. If you want to try, I recommend the de-pinning tools that look like a pair of tweezers with two flat blades poking out - see photo.

The good news is the R720's are less sensitive to the cable pinouts than the 730's are, so you can probably just order one from E-bay and it is reasonably expected to work. Something like this should be good, but still check it out to make sure before you fry your card or MB: For DELL R720Xd R730 K80 M60 M40 8pin to 8pin Power Cable GPU Power Supply Line | eBay

 

[kroem]

Hehe ok, sounds like a good idea not to tinker too much myself.

The p40 cards can with a cable splitter, so I guess I should be able to use the normal "N08NH 9H6FV" cable to that?

 

[blueprint]

First of all, thank you all for this thread, this solved my issue for my Dell PowerEdge R730 Server that I am setting up as a AI HomeLab Server for running LLMs and image generation. I'm seeding this reply with keywords in hopes in helps someone in the future find this thread and save them the 20 hours I went through debugging this.

I have two Nvidia Tesla P40s in the server. I kept having issues setting up the GPU / PCI pass through from the server through Proxmox to the VMs. The VMs could recognize the GPUs when I ran "lspci -nn | grep -i nvidia" but when running " nvidia-smi" the Nvidia drivers could not recognize the GPUs. I eventually found out running "nvidia-bug-report.sh" gave me a much more detailed log. In that log I found the error "NVRM: GPU 0000:00:10.0: GPU does not have the necessary power cables connected."

At first I incorrectly assumed my 750W PSUs did not have enough output for running the two P40 gpus and the base server config. When my 1100W PSUs the problem still persisted. That's when I finally found this thread, this thread led me to testing my power cables with my multimeter and I found that they were both wired incorrectly (and both wired differently from each other) for this server model and GPU despite claims from the listing showing compatibility. The cables I purchase were: https://www.amazon.com/dp/B08N4BJL2J (GinTai 8(pin) to 8(pin) Power Cable Replacement for DELL R730 and Nvidia K80/M40/M60/P40/P100 PCIE GPU), I do not reccomend buying them but they were easy enough to rewire to the correct specifications as they had the right number of wires and splices/splits.

I saved myself from having to buy an ATX pin remover tool that I would likely not use again in the next two years following this tutorial: https://www.youtube.com/watch?v=x5_58o0WaTM

Since I am not an electrician and haven't played around with PSUs outside of building my own desktop and using manufacture spec cables this was a slightly daunting task as I didn't want to mess something up and fry my hardware. To help myself with this I drew out a new diagram for the wiring based on information in this thread and this video from Computers Cats and More:

Hopefully someone else will find this diagram helpful in the future and I'm greatful for any input on if I have incorrectly labeled or documented anything. I also have a PDF to the diagram linked on my site so the links are clickable: Dell Poweredge R730 to Nvidia Tesla P40 Power Cable Wiring Diagram – Blueprint Coders

Cheers.

 

[avnkhanh]

thank you i followed the picture and my GPU works fine on R730xd