Error enabling switchdev for ConnectX-5 Ex in Proxmox 8.4.1

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

eric_h

New Member
Feb 9, 2025
2
1
1
I got a ConnectX-5 Ex to play with switchdev mode offloading and I'm following the steps in this guide, but the command to enable switchdev mode returns an invalid argument error:

Code:
root@pve:~# /usr/sbin/devlink dev eswitch set pci/0000:41:00.0 mode switchdev
Error: mlx5_core: Failed setting eswitch to offloads.
kernel answers: Invalid argument
I found other guides on using switchdev, but none mentioned needing to make any changes before running the devlink command to enable switchdev mode, and I didn't see anything in mstconfig that looked related to enabling it.

Is there something I've missed, or could this be a difference between the ConnectX-5 and the ConnectX-5 Ex?

Thanks in advance for any help.
 

Attachments

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,806
1,197
113
Is there something I've missed, or could this be a difference between the ConnectX-5 and the ConnectX-5 Ex?
Did you unbind the driver from all VFs before changing the mode to eswitch?

My standard script basically works like this

1. Set number of VFs (/usr/bin/echo 8 > /sys/class/net/ens21f0np0/device/sriov_numvfs)
2. Set Static MACs for VFs (/usr/bin/ip link set ens21f0np0 vf 0 mac d2:77:a4:7c:0d:02)
3. Unbind VFs (echo 0000:b3:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind)
4. Change eSwitch mode from Legacy to Switchdev (/usr/sbin/devlink dev eswitch set pci/0000:b3:00.0 mode switchdev)

(Code snippets are just examples)
 

eric_h

New Member
Feb 9, 2025
2
1
1
Thank you both for your help with this.

The VFs are being created successfully, and I was getting the same error when unbinding all the VFs before changing the switch mode.

I found this thread about updating the firmware on the same Dell branded card I'm using, and after updating the firmware from 16.26.6000 to 16.26.6000 (from Mellanox ConnectX-5 Ethernet Adapter Firmware | Driver Details | Dell US), and rebooting it's showing up in switchdev mode:
Code:
root@pve:~# devlink dev eswitch show pci/0000:41:00.0
pci/0000:41:00.0: mode switchdev inline-mode none encap-mode basic
 
  • Like
Reactions: necr