Mellanox CX311A-XCAT - write counter to semaphore: Operation not permitted

[reefland]

I have several Mellanox CX311A-XCAT, they all seem to be functional. However, in poking around at them trying to see what Firmware they have and if I should update it, I get some unusual results and I'm not sure what it's telling me.

Using Ubuntu 20.04.4:

$ lspci |grep -i Mellanox
0a:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]

$ sudo mstconfig -d 0a:00.0
query Fails: E- Failed to open the device

$ sudo mstflint -d 0a:00.0 q
write counter to semaphore: Operation not permitted
write counter to semaphore: Operation not permitted
-E- Cannot open Device: 0a:00.0. No such file or directory. MFE_CR_ERROR

Google has not been my friend. So many links to Mellanox do not work with the redirection to Nvidia.

I've seen hints that current tools do not support these older cards, should be be looking for older versions of these tools?

 

[necr]

can you try:

sudo mstflint -d mlx4_0 q
sudo ibstat

Which verrsion of mstflint do you have? What's the OS?

 

[reefland]

@necr Thanks for the reply. Running msflint 4.13.3 on Ubuntu 20.04

$ sudo mstflint -d mlx4_0 q
write counter to semaphore: Operation not permitted
write counter to semaphore: Operation not permitted
-E- Cannot open Device: mlx4_0. No such file or directory. MFE_CR_ERROR

$ sudo ibstat
CA 'mlx4_0'
        CA type: MT4099
        Number of ports: 1
        Firmware version: 2.42.5000
        Hardware version: 1
        Node GUID: [redacted]
        System image GUID: [redacted]
        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 10
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x00010000
                Port GUID: [redacted]
                Link layer: Ethernet

Thoughts?

 

[necr]

2.42.5000 is the latest version, and you have 10G link. You're all set, enjoy your NIC and try some RDMA goodness if you wish.

 

[reefland]

Good to know firmware version is up-to-date!

I'm still concerned that the mstflint tool is unable to work with the device, it's not clear if this is expected or not (unsupported card, etc.)

The reason I started looking at Firmware was a due to not getting link light without reinserting transceiver (STH thread). Was hoping a newer firmware would fix. I'll keep digging.

Thank again.

 

[necr]

You can try to clear flash semaphore (whatever that means), force flash new firmware, set jumper to get into recovery mode if it's not working. Back up old FW beforehand. Still, if the core functionality is there, I wouldn't waste time unless you want to know all the knobs and details about CX cards.

 

[fohdeesha]

I have several Mellanox CX311A-XCAT, they all seem to be functional. However, in poking around at them trying to see what Firmware they have and if I should update it, I get some unusual results and I'm not sure what it's telling me.

Using Ubuntu 20.04.4:

$ lspci |grep -i Mellanox
0a:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]

$ sudo mstconfig -d 0a:00.0
query Fails: E- Failed to open the device

$ sudo mstflint -d 0a:00.0 q
write counter to semaphore: Operation not permitted
write counter to semaphore: Operation not permitted
-E- Cannot open Device: 0a:00.0. No such file or directory. MFE_CR_ERROR

Google has not been my friend. So many links to Mellanox do not work with the redirection to Nvidia.

I've seen hints that current tools do not support these older cards, should be be looking for older versions of these tools?

have you tried using mellanox's official tools instead of their open source mstflint version? eg: https://forums.servethehome.com/ind...net-dual-port-qsfp-adapter.20525/#post-198015

 

[reefland]

have you tried using mellanox's official tools instead of their open source mstflint version? eg: https://forums.servethehome.com/ind...net-dual-port-qsfp-adapter.20525/#post-198015

Thanks for the link, I'll give it a try.

 

[plunko]

For the sake of people who may find this thread on Google, this is the error you'll see if you have secure boot enabled and the kernel is in lockdown mode. If you look in dmesg you should see some complaints from the kernel refusing access. Temporarily booting with secure boot disabled should allow you to use mstflint/mstconfig as expected.