[Resolved] Supermicro H11SSL-I too many PCIE cards to add Tesla P4?

[cbarbie]

I have the following:
H11SSL-I
Firmware Revision: 01.52.15
Firmware Build Time: 11/08/2021
BIOS Version: 2.4
BIOS Build Time: 12/27/2021
Redfish Version: 1.8.0
CPLD Version: 02.B4.02
BIOS settings attached
All three power connector are attached - JPW1 (4pin), JPWR1 (8pin), and JPWR2 (24 pin)


EPYC 7302P

2 HBAs Slots 3 & 5 - 3.0x8
1 10g card - Slot 1 - 3.0x8

2 SATA SSDs

ProxMox - pve-manager/7.3-3/c3928077

Everything works great.

When I add the Tesla P4 in Slot 6 - 3.0x16, various things stop working.
One one boot, the onboard sata ports don't work.
On the next boot, the 10g card doesn't work.
On the next boot the system hangs, have to power off with the power button.

I remove the P4 and everything works again.


Do I have a bad P4?
Is there a setting I need to change in the BIOS?

Any assistance would be appreciated. Thank you.

 

[RolloZ170]

you can fix the slot to gen3.

All three power connector are attached - JPW1 (4pin), JPWR1 (8pin), and JPWR2 (24 pin)

nice. 250W PSU ?

 

[cbarbie]

you can fix the slot to gen3.

nice. 250W PSU ?

Thanks.

For the H11SSL-I, all the six slots are PCI-E 3.0 3@x16 3@x8

Power = 2 x PWS-1K28P-SQ

 

[RolloZ170]

For the H11SSL-I, all the six slots are PCI-E 3.0 3@x16 3@x8

sounds like a gen detection issue, board is gen3 but cpu is gen4.
fix slot to gen3.

 

[cbarbie]

sounds like a gen detection issue, board is gen3 but cpu is gen4.
fix slot to gen3.

The attachments in the original post (BIOS 1&2) show they are gen3 and I have not found any other BIOS settings that could change that.
Maybe I need to set OPROM for slot6 to none?

 

[RolloZ170]

The attachments in the original post (BIOS 1&2) show they are gen3

only a string they can 3.0.
i see many legacy. set UEFI only and disable CSM.

 

[RolloZ170]

missing Target Link Speed which BIOS ver.?

 

[RolloZ170]

...but can be also a BAD P4 .

 

[cbarbie]

I appreciate the suggestions.


BIOS Version: 2.4 (BIOS_H11SSL-1A03_20211227_2.4_STDsp)

I tried setting boot to UEFI only. Turned CSM off.
It boots into shell. I have had very little experience with UEFI. How do I get it to boot off the disks into ProxMox?

 

[RolloZ170]

How do I get it to boot off the disks into ProxMox?

I have had very little experience with ProxMox.
you have to install in EFI mode.

 

[RolloZ170]

When I add the Tesla P4 in Slot 6 - 3.0x16, various things stop working.
One one boot, the onboard sata ports don't work.
On the next boot, the 10g card doesn't work.
On the next boot the system hangs, have to power off with the power button.

but the P4 is working ?

 

[cbarbie]

I saw it as a device in ProxMox once. That time it showed up, but the 10g card disappeared.

 

[CyklonDX]

Try not using pcie 6 for non-video output. Slot6 does not share pcie lanes with other slots.
Have you tried JPG1 set to disable (if you must use slot6)? Have you tried it on slot4.

 

[cbarbie]

I moved the P4 from slot6 to slot4. Everything worked!
I was able to pass the P4 thru to a VM.

I wonder what makes slot6 different.

 

[CyklonDX]

slot6 is default VGA output port. All funny things can happen with compute cards in them, unless you use jumpers/bios settings to change/disable default vga/display pcie port.

 

[RolloZ170]

I moved the P4 from slot6 to slot4. Everything worked!
I was able to pass the P4 thru to a VM.

I wonder what makes slot6 different.

can you install a real GPU in slot6 ? to very @CyklonDX theory

 

[cbarbie]

can you install a real GPU in slot6 ? to very @CyklonDX theory

If I had a real GPU, I could try it.

 

[RolloZ170]

If I had a real GPU, I could try it.

that is true !