Both ports on Mellanox ConnectX-3 FDR/EN have the same PCI function ID | HP 649282-B21

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

einstom

New Member
Jun 16, 2023
1
0
1
Germany
Hi,

I recently started to use the HP Mellanox QSFP+ cards (both 649282-B21 and 764285-B21) but I'm facing the issue that both ports on the NIC have the exact same PCI address (even the same function id). I wanted to use them for a VPP router but this does not work as DPDK can't handle this edge case.

I had a look on the mlxpci tool but there's literally no public documentation available and the exported python pickle is also not really self-explaining.

The issue also appears when I install it in a HP server with FlexLOM slot.

lshw:
Code:
[root@chicken-legend ~]# lshw -c network -businfo
Bus info          Device      Class          Description
========================================================
pci@0000:82:00.0  enp130s0    network        MT27500 Family [ConnectX-3]
pci@0000:82:00.0  enp130s0d1  network        Ethernet interface

mlxconfig:
Code:
[root@chicken-legend ~]# mlxconfig -d /dev/mst/mt4099_pci_cr0 q

Device #1:
----------

Device type:    ConnectX3
Device:         /dev/mst/mt4099_pci_cr0

Configurations:                                      Next Boot
         SRIOV_EN                                    False(0)
         NUM_OF_VFS                                  16
         WOL_MAGIC_EN_P2                             False(0)
         LINK_TYPE_P1                                ETH(2)
         PHY_TYPE_P1                                 XFI(2)
         XFI_MODE_P1                                 _10G(0)
         FORCE_MODE_P1                               False(0)
         LINK_TYPE_P2                                ETH(2)
         PHY_TYPE_P2                                 XFI(2)
         XFI_MODE_P2                                 _10G(0)
         FORCE_MODE_P2                               False(0)
         LOG_BAR_SIZE                                5
         BOOT_PKEY_P1                                0
         BOOT_PKEY_P2                                0
         BOOT_OPTION_ROM_EN_P1                       False(0)
         BOOT_VLAN_EN_P1                             False(0)
         BOOT_RETRY_CNT_P1                           0
         LEGACY_BOOT_PROTOCOL_P1                     PXE(1)
         BOOT_VLAN_P1                                1
         BOOT_OPTION_ROM_EN_P2                       False(0)
         BOOT_VLAN_EN_P2                             False(0)
         BOOT_RETRY_CNT_P2                           0
         LEGACY_BOOT_PROTOCOL_P2                     PXE(1)
         BOOT_VLAN_P2                                1
         IP_VER_P1                                   IPv4(0)
         IP_VER_P2                                   IPv4(0)
         CQ_TIMESTAMP                                True(1)
         STEER_FORCE_VLAN                            False(0)

Any ideas? Has anyone also faced this issue or solved it?
 

klui

༺༻
Feb 3, 2019
867
478
63
You can try one of two things.
  1. Use CX4 or newer.
  2. Create some VFs, each will have a different id.