Mellanox MCX354A-FCBS not able to turn-on SR-IOV

[salvadorb]

Hi guys,
I have a dozen of Mellanox MCX354A-FCBS Rev A3 cards (dual 40GbE), I was able to convert them into Ethernet using the firmware in the Nvidia/Mellanox website Firmware for ConnectX®-3 EN, I'm using firmware 2.42.5000 (MCX314A-BCBT with PSID MT_1090110023) and both ports are connected at 40GbE. But I haven't been able to enable SR-IOV, when I enter mst status, the MFT package never recogizes the card, I even tried reflashing it with MT_1090120019 (Firmware for ConnectX®-3 IB/VPI for MCX354A-FCBT), but same problem, the card is not recogized by the mst status command, and when I enter the commands

sudo mlxconfig -d /dev/mst/mt4099_pci_cr0 q | egrep 'SRIOV_EN|NUM_OF_VFS|LOG_BAR'
sudo mlxconfig -d /dev/mst/mt4099_pci_cr0 set SRIOV_EN=1 NUM_OF_VFS=32 LOG_BAR_SIZE=5

It always says Permission denied or Failed to open the device

and commands like "sudo mlxconfig -d 0000:82:00.0 q" say "-E- Failed to open the device"

Any idea what could be wrong?

More info from outputs:



sudo mst status -v





Starting MST (Mellanox Software Tools) driver set





[warn] mst_pci is already loaded, skipping





Loading MST PCI configuration module - Success





Create devices





Unloading MST PCI module (unused) - Success





Unloading MST PCI configuration module (unused) - Success





total 0





MST modules:





------------





    MST PCI module is not loaded





    MST PCI configuration module is not loaded











    No MST devices were found or MST modules are not loaded.





    You may need to run 'mst start' to load MST modules.

More outputs

$ sudo mstflint  -d 0000:82:00.0 q
Image type:            FS2
FW Version:            2.42.5000
FW Release Date:       5.9.2017
Product Version:       02.42.50.00
Rom Info:              type=PXE version=3.4.752
Device ID:             4099
Description:           Node             Port1            Port2            Sys image
GUIDs:                 0002c90300fb6120 0002c90300fb6121 0002c90300fb6122 0002c90300fb6123
MACs:                                       0002c9fb6120     0002c9fb6121
VSD:                   
PSID:                  MT_1090110023

 

[mrpops2ko]

for updating the most success i've had is just slapping windows on a usb stick as a live boot and flashing that way (using that windows2go or whatever its called)

make sure you aren't actively using the nic in VMs / have it bound to something, i've had issues there before, make sure you also have all the common grub stuff enabled

iommu etc

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=1 iommu=pt

check bios for sr-iov / iommu enabling too

it would be helpful to go over what steps you've already tried, because it seems like by the looks of it you don't have iommu enabled