ConnectX-2 MHRH2A-XSR 2.10 firm changes DEV_ID from 26418 to 26428

[Dark09]

Well I jumped on the bandwagon last week, and purchased a bunch of infiniband equipment on ebay. I started installing the adapters in a few servers today, and cabled them between one another. Initially I was expecting the PCIe cards to have similar specs, but found that 4 of the 5 are these MHRH2A-XSR adapters (DDR 20Gb). The other one is a QDR, which is what I was expecting.

The problem is that I worked though the few different articles posted over flashing 2.10 onto the QDR adapter without a hitch, then I attempted the same thing with the MHRH2A adapter, and after the reboot I wasn't able establish a connect onto the fabric using the opensm software. I found that comparing the flint query output shows that the DEV_ID changes from the DDR ID (26418) to the QDR ID (26428), but the PSID stays the same (MT_0F90120008). Below are the outputs:

Before firmware update:

flint -d mt26418_pci_cr0 query
Image type:          FS2
FW Version:          2.9.1000
Device ID:           26418
Description:         Node             Port1            Port2            Sys imag
e
GUIDs:               0002c90300b7d9c0 5849560eeea10a01 5849560eeea10a02 0002c903
00b7d9c3
MACs:                                     0002c9b7d9c0     0002c9b7d9c1
VSD:
PSID:                MT_0F90120008

After firmware update:

flint -d mt26418_pci_cr0 query
Image type:          FS2
FW Version:          2.10.0720
FW Version(Running): 2.9.1000
FW Release Date:     12.3.2012
Device ID:           26428
Description:         Node             Port1            Port2            Sys imag
e
GUIDs:               0002c90300b7d9c0 5849560eeea10a01 5849560eeea10a02 0002c903
00b7d9c3
MACs:                                     0002c9b7d9c0     0002c9b7d9c1
VSD:
PSID:                MT_0F90120008

At this point I rolled back the firmware, but any insight would be very much appreciated.

Thanks!

 

[i386]

wqhen you are updating or burning the firmware you could use an argument (
-allow_psid_change burn -> Flash OEM (IBM) Mellanox ConnectX-3 EN with stock firmware?)
that allows you to change the psid

 

[Dark09]

Thanks i386, I attempted ixz's guide, and using the allow_psid_change option, but opensm still will not go into an up state, after the firmware update it just says its down. I tried changing the -g option to the second port on both the pcie cards, and the state will still not go up. I'm still thinking it is related to maybe the adapter is attempting to run at 40GB, when it is only compatible to run at 20GB, and from what I can see on the ConnectX3IB firmware page there isn't any DDR firmware, only QDR, and FDR, or maybe i'm just confusing myself >.<. Here's the output of the mlxburn -vdp:

mlxburn -d mt26418_pciconf0 -vpd

  VPD-KEYWORD    DESCRIPTION             VALUE
  -----------    -----------             -----
Read Only Section:

  PN             Part Number             MHRH2A-XSR
  EC             Revision                A3
  SN             Serial Number           MT1303U02461
  V0             Misc Info               PCIe Gen2 x8
  RV             Checksum Complement     0xa0
  IDTAG          Board Id                OWL DDR

Since the firmware update is switching the DEV_ID to 26428 would it make since to change the ini filename, PS_INFO Name, ADAPTER PSID to one of the 26428 PSID on the firmware page, so it matches 26428?:

Rev X1-X3:
MT_0DC0110009

Rev X3-X5:
MT_0DD0120009

I have four of these PCIe cards, so I have multiple chances to get it right I guess...

 

[Robert Townley]

The "FW Version" versus "FW Version(Running)" lines tells me a graceful shutdown and cold restart may have helped.

flint -d mt26418_pci_cr0 query

Image type: FS2
FW Version: 2.10.0720
FW Version(Running): 2.9.1000
FW Release Date: 12.3.2012