QNAP QXG-10G2T-X710 discussion thread

marcoi · Mar 26, 2025

so i was able to reverse the red and black lines on the fans and they run, super quiet. I also just used one screw to hold it in place vs gluing it down. see how it goes. Once it setup on my desk it wont be moving often so i think 1 screw is good enough for now.

heromode · Mar 26, 2025

https://nl.aliexpress.com/item/1005...ey6KgzBJ&utparam-url=scene:search|query_from:

That IS the exact same fan. I have no idea why it isn't working

heromode · Mar 26, 2025

marcoi said:
so i was able to reverse the red and black lines on the fans and they run, super quiet. I also just used one screw to hold it in place vs gluing it down. see how it goes. Once it setup on my desk it wont be moving often so i think 1 screw is good enough for now.

OH!

Great! I will try that. You're a lifesaver

marcoi · Mar 26, 2025

heromode said:
OH!

Great! I will try that. You're a lifesaver

I was like something wrong, no way all the fans were bad. then i double checked the power and they were in the wrong place. you need a small flat head to pry open the tab the holds the pins into the plastic, then plug the line out and swap. hopefully that your issue as well.

heromode · Mar 26, 2025

marcoi said:
I was like something wrong, no way all the fans were bad. then i double checked the power and they were in the wrong place. you need a small flat head to pry open the tab the holds the pins into the plastic, then plug the line out and swap. hopefully that your issue as well.

Yup, confirmed, mine is spinning now.

Those small connector's are a pain for my old eyes, but i've done alot of such work in the past.. Single screw should be ok, if you want to make sure it doesn't move around, single screw plus a tiny bit of superglue on just one of the other spans should be good, and that would make it possible to remove the fan as well relatively easy.

This is the way.

tillburn · May 10, 2025

I am a total script numb cake here, and I appreciate what you've learned, taught and done here, Thank you!

Couple of questions please to help me understand what's happening.

" ExecReload=/usr/bin/bash -c 'echo 0000:01:00.3 > /sys/bus/pci/devices/0000\\:01\\:02.1/driver/unbind' "

It looks like this script is either remapping or readdressing the PCI from 1:00.3 to 1:02.1 Is that right? If that is the case, what's the purpose for changing the address?

I have the X710-4 SFP+ variant, and I also would like to have 7 VF's available to VM's and 1 VF available to the Proxmox host. @ PCI address 62.0.0, 62.0.1, 62.0.2 and 62.0.3
I also have installed a dual port connect-x 5 100g card that I would also like to have 7 VF's available to VM's and 1VF available to the Promos host.@ PCI address 21.0.0 and 21.0.1

If I understand what your doing and why then I can monkey see monkey do using your script as a jump off point (thank you again).

Best regards,

Tillburn

heromode said:

update:

My standard systemd proxmox script for sr-iov partitioning and detaching vf's from host seem to be somewhat working, i've modified the lspci command to list the vf's. I obtained the lspci -D -d 8086:154c command by trying various combo's from the output of lspci -vvm -nn -s 0000:01:02.0

Code:

#lspci -vvm -nn -s 0000:01:02.0
Device: 01:02.0
Class:  Ethernet controller [0200]
Vendor: Intel Corporation [8086]
Device: Ethernet Virtual Function 700 Series [154c]
SVendor:        QNAP Systems, Inc. [1baa]
SDevice:        Ethernet Virtual Function 700 Series [0000]
Rev:    02
ProgIf: 00
NUMANode:       0
IOMMUGroup:     158

Each physical port can be divided to 64 partitions iirc. This script creates 8 virtual functions per physical port, set static MAC's for them (VF's get random generated MAC's each time they are created), and then detaches 7 VF's per PF for use with VM's. I wanted to leave 1 VF per port for the proxmox host for use with things like iscsi. The actual PF is also used by proxmox as a bridged port, for VM's that use it via standard virtio.

If you add or remove pcie devices you need to check if the pcie address of your nic changes, and edit the script accordingly. I wish it wasn't so, but i have yet to figure out another way. Proxmox now supports device mapping for pcie passthrough devices, to enable migration support.

I wish the Proxmox team could make a GUI function for partitioning and detaching network devices as well. I don't have the skills or the time to get into all that.

Code:

[Unit]
Description=Enable SR-IOV and detach guest VFs from host
Requires=network.target
After=network.target
Before=pve-firewall.service
[Service]
Type=oneshot
RemainAfterExit=yes
# Create NIC VFs
ExecStart=/usr/bin/bash -c 'echo 8 > /sys/class/net/ens2f0np0/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c 'echo 8 > /sys/class/net/ens2f1np1/device/sriov_numvfs'
# Set static MACs for VFs
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 0 mac 76:9e:17:83:39:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 1 mac 46:2c:6d:24:6b:1b'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 2 mac 3e:47:48:12:ed:94'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 3 mac be:e3:6a:f3:8f:ac'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 4 mac 62:8f:3d:bb:02:08'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 5 mac ae:91:57:b9:14:7f'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 6 mac 5a:c2:08:a9:68:a7'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 7 mac b2:f0:18:af:cb:c5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 0 mac 16:47:7c:a8:95:98'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 1 mac a6:c7:c5:7f:9c:22'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 2 mac b6:0f:45:34:5e:19'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 3 mac 2a:f7:37:84:31:30'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 4 mac 8a:fa:f8:c5:0b:93'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 5 mac b2:f5:d5:2f:79:06'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 6 mac c2:92:f5:fa:32:20'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 7 mac 2e:fb:29:1e:48:31'
# Detach VFs from host
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.1 > /sys/bus/pci/devices/0000\\:01\\:02.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.2 > /sys/bus/pci/devices/0000\\:01\\:02.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.3 > /sys/bus/pci/devices/0000\\:01\\:02.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.4 > /sys/bus/pci/devices/0000\\:01\\:02.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.5 > /sys/bus/pci/devices/0000\\:01\\:02.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.6 > /sys/bus/pci/devices/0000\\:01\\:02.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:02.7 > /sys/bus/pci/devices/0000\\:01\\:02.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.1 > /sys/bus/pci/devices/0000\\:01\\:0a.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.2 > /sys/bus/pci/devices/0000\\:01\\:0a.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.3 > /sys/bus/pci/devices/0000\\:01\\:0a.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.4 > /sys/bus/pci/devices/0000\\:01\\:0a.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.5 > /sys/bus/pci/devices/0000\\:01\\:0a.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.6 > /sys/bus/pci/devices/0000\\:01\\:0a.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:01:0a.7 > /sys/bus/pci/devices/0000\\:01\\:0a.7/driver/unbind'
# List new VFs
ExecStart=/usr/bin/lspci -D -d 8086:154c
# Destroy VFs
ExecStop=/usr/bin/bash -c 'echo 0 > /sys/class/net/ens2f0np0/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c 'echo 0 > /sys/class/net/ens2f1np1/device/sriov_numvfs'
# Reload NIC VFs
ExecReload=/usr/bin/bash -c 'echo 0 > /sys/class/net/ens2f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c 'echo 0 > /sys/class/net/ens2f1np1/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c 'echo 8 > /sys/class/net/ens2f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c 'echo 8 > /sys/class/net/ens2f1np1/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 0 mac 76:9e:17:83:39:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 1 mac 46:2c:6d:24:6b:1b'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 2 mac 3e:47:48:12:ed:94'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 3 mac be:e3:6a:f3:8f:ac'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 4 mac 62:8f:3d:bb:02:08'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 5 mac ae:91:57:b9:14:7f'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 6 mac 5a:c2:08:a9:68:a7'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f0np0 vf 7 mac b2:f0:18:af:cb:c5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 0 mac 16:47:7c:a8:95:98'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 1 mac a6:c7:c5:7f:9c:22'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 2 mac b6:0f:45:34:5e:19'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 3 mac 2a:f7:37:84:31:30'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 4 mac 8a:fa:f8:c5:0b:93'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 5 mac b2:f5:d5:2f:79:06'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 6 mac c2:92:f5:fa:32:20'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set ens2f1np1 vf 7 mac 2e:fb:29:1e:48:31'
ExecReload=/usr/bin/bash -c 'echo 0000:01:00.3 > /sys/bus/pci/devices/0000\\:01\\:02.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:00.4 > /sys/bus/pci/devices/0000\\:01\\:02.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:00.5 > /sys/bus/pci/devices/0000\\:01\\:02.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:00.6 > /sys/bus/pci/devices/0000\\:01\\:02.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:00.7 > /sys/bus/pci/devices/0000\\:01\\:02.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.0 > /sys/bus/pci/devices/0000\\:01\\:02.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.1 > /sys/bus/pci/devices/0000\\:01\\:02.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.3 > /sys/bus/pci/devices/0000\\:01\\:0a.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.4 > /sys/bus/pci/devices/0000\\:01\\:0a.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.5 > /sys/bus/pci/devices/0000\\:01\\:0a.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.6 > /sys/bus/pci/devices/0000\\:01\\:0a.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:01.7 > /sys/bus/pci/devices/0000\\:01\\:0a.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:02.0 > /sys/bus/pci/devices/0000\\:01\\:0a.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:01:02.1 > /sys/bus/pci/devices/0000\\:01\\:0a.7/driver/unbind'
ExecReload=/usr/bin/lspci -D -d 8086:154c
[Install]
WantedBy=multi-user.target

Enable with:

Code:

systemctl daemon-reload
systemctl enable sriov-vfs.service

Usage:

Code:

systemctl start sriov-vfs.service
systemctl stop sriov-vfs.service
systemctl reload sriov-vfs.service
systemctl status sriov-vfs.service

It seems to be working, however, after proxmox boot i get:

Code:

Mar 16 08:02:17 pve kernel: iavf 0000:01:02.0: Failed to communicate with PF; waiting before retry
Mar 16 08:02:22 pve kernel: iavf 0000:01:02.0: Hardware came out of reset. Attempting reinit.
Mar 16 08:02:22 pve kernel: iavf 0000:01:02.0: Multiqueue Enabled: Queue pair count = 4
Mar 16 08:02:22 pve kernel: iavf 0000:01:02.0: MAC address: 76:9e:17:83:39:e5
Mar 16 08:02:22 pve kernel: iavf 0000:01:02.0: GRO is enabled
Mar 16 08:02:22 pve kernel: iavf 0000:01:02.0 ens2f0v0: renamed from eth0

I don't know why this is..

after posting this i will modify my script and set new random MAC's, the above MAC's only used for this example.

UPDATE:

I was looking for the command to list the randomly generated MAC addresses for my VF's, in old script for solarflare it was easy..
Turns out that it seems Intel cards do not generate random mac's when partition, as seen by output of the command

ip link show dev ens2f0np0

Code:

2: ens2f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP mode DEFAULT group default qlen 1000
    link/ether 24:5e:be:67:fe:a4 brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 1     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 2     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 3     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 4     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 5     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 6     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    vf 7     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
    altname enp1s0f0np0

So it seems i have to find a random MAC address generator to set static MAC's
like this one
Previously with solarflare i just copied the one time randomly generated ones and made them permanent

If someone has more info about this please say what's going on.

UPDATE 2:

Seems like some, but not all of the MAC addresses generated by that web page don't work! So now i have to manually test setting the address for each VF, and see if it accepts it. What a pain

heromode · May 10, 2025

tillburn said:
I am a total script numb cake here, and I appreciate what you've learned, taught and done here, Thank you!

Couple of questions please to help me understand what's happening.

" ExecReload=/usr/bin/bash -c 'echo 0000:01:00.3 > /sys/bus/pci/devices/0000\\:01\\:02.1/driver/unbind' "

It looks like this script is either remapping or readdressing the PCI from 1:00.3 to 1:02.1 Is that right? If that is the case, what's the purpose for changing the address?

I have the X710-4 SFP+ variant, and I also would like to have 7 VF's available to VM's and 1 VF available to the Proxmox host. @ PCI address 62.0.0, 62.0.1, 62.0.2 and 62.0.3
I also have installed a dual port connect-x 5 100g card that I would also like to have 7 VF's available to VM's and 1VF available to the Promos host.@ PCI address 21.0.0 and 21.0.1

If I understand what your doing and why then I can monkey see monkey do using your script as a jump off point (thank you again).

Best regards,

Tillburn

You caught an error i missed, i guess i never tested the script enough
the unbind command should be for the same pcie address
what it does is detach the VF from the host, so it can be assigned to a VM. The Proxmox host does not see it anymore after these commands.
So there should be 7 unbound commands per port.
Thanks for noticing the error. Again, go though your script and make sure 0000:01:00.3 matches /sys/bus/pci/devices/0000\\:01\\:00.3/driver/unbind'

and so forth. I will fix it when i get the chance

edit: yes, all the unbind commands in the ExecReload part of the script are wrong, i forgot to copypaste the new pcie addresses. They are from an old script i used with solarflare cards. The commands are correct in the ExecStart section

The ExecReload commands are run when you do 'systemctl reload sriov-vfs.service'

and i never tested that

Edit2: Fixed now. Thanks again for alerting me.

heromode · May 10, 2025

tillburn said:
I have the X710-4 SFP+ variant, and I also would like to have 7 VF's available to VM's and 1 VF available to the Proxmox host. @ PCI address 62.0.0, 62.0.1, 62.0.2 and 62.0.3
I also have installed a dual port connect-x 5 100g card that I would also like to have 7 VF's available to VM's and 1VF available to the Promos host.@ PCI address 21.0.0 and 21.0.1

If I understand what your doing and why then I can monkey see monkey do using your script as a jump off point (thank you again).

Best regards,

Tillburn

I have tested this method on solarflare cards and intel X710. but i think it should work on mellanox as well, not sure.

You can't choose the pcie addresses. At first you should simply do 'echo 8 > /sys/class/net/ens2f0np0/device/sriov_numvfs'
for both ports of your card (change ens2f0np0 accordingly)

Then after that using lspci, you will see the pcie addresses for your Physical Functions, and 8 newly created Virtual Functions

Then you start changing the addresses in the script accordingly.

the 'ExecStart=/usr/bin/lspci -D -d 8086:154c' command in the script will need to be changed for each type of card you have. It simply lists the VF's to be displayed in the logs.

For intel products, it's even more complicated, as many other components share some ID part, that's why i came up with 8086:154c.
On my old Solarflare cards, it was enough to do 'lspci -D -d1924:'

Basically, you will have to play around with lspci to get the desired output. Also the lspci command is not needed in the script, you can just leave it out as well. the output from 'lspci -D -d 8086:154c' on my current X710 card is:

Code:

# lspci -D -d 8086:154c
0000:01:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
0000:01:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

heromode · May 10, 2025

As an unrelated update, i recently tested out the Resource Mapping function in proxmox for these VF's.

And i noticed, that the resource mapping function in proxmox just assigns the first available free VF to whatever VM that you start, depending on the order in which i started the VM's, Network Manager would not recognize the interface as the same, even if the interface name in the VM stays the same.

So the VM (in this case running KDE Plasma) would just automatically create new entries for the ports on boot, called 'wired connection 1' and 'wired connection 2'
And because no IP's were assigned to the new entries, they would not work.

So i haven't had time to test this yet, but i'm thinking that if using mapped resources, it might be best not to set the static MAC addresses in the script, and instead do that in the VM at boot.

But i haven't had time to test if it is the MAC addresses that cause this behaviour in network manager, or something else.
If anyone has information on how to be able to use resource mappings without network manager in the VM losing the settings for the interfaces at boot, please let us know..

heromode · May 10, 2025

And in a yet another unrelated update, i was unable to ping or communicate from a VM with a virtio network interface using the linux bridge assigned to the Physical Function of the X710 nic in proxmox, to a VF assigned as a pcie device in a VM.

This worked fine on the solarflare cards, but not on the X710. I found a thread that might or might not hold the answer, but again, haven't had time to look into this. If i won't get it to work, i will just have to use a virtio interface connected to the proxmox bridge for the VM's and containers i need to communicate with eachother via the X710.

This thread might hold the clue: CARP IP not pingable from other SR-IOV virtual function on same host

tillburn · May 10, 2025

heromode said:
As an unrelated update, i recently tested out the Resource Mapping function in proxmox for these VF's.

And i noticed, that the resource mapping function in proxmox just assigns the first available free VF to whatever VM that you start, depending on the order in which i started the VM's, Network Manager would not recognize the interface as the same, even if the interface name in the VM stays the same.

So the VM (in this case running KDE Plasma) would just automatically create new entries for the ports on boot, called 'wired connection 1' and 'wired connection 2'
And because no IP's were assigned to the new entries, they would not work.

So i haven't had time to test this yet, but i'm thinking that if using mapped resources, it might be best not to set the static MAC addresses in the script, and instead do that in the VM at boot.

But i haven't had time to test if it is the MAC addresses that cause this behaviour in network manager, or something else.
If anyone has information on how to be able to use resource mappings without network manager in the VM losing the settings for the interfaces at boot, please let us know..

I swear I watched a video from Wendel @ levelone on this. But I can't find it.

heromode · May 10, 2025

tillburn said:
I swear I watched a video from Wendel @ levelone on this. But I can't find it.

It's really easy using the GUI in Proxmox. After it's setup it's much easier assigning VF's to VM, you simply choose X710_VF_ port0 or X710_VF_port1, or whatever name you assign. And you don't have to keep tabs on which pcie address is already in use.

But again, when i started the 2 VM's using the mapped resources in different order, Network Manager sees that as a new interface, and won't use the IP settings for the previous interface, even if the IF names stay the same. And with this script setting static MAC's it could be the reason, or then it's something else. I'm sure there is a solution though, network manager is pretty flexible and configurable.

Please let me know if you come up with a solution

tillburn · May 10, 2025

[Unit]
Description=Enable SR-IOV and detach guest VFs from host
Requires=network.target
After=network.target
Before=pve-firewall.service
[Service]
Type=oneshot
RemainAfterExit=yes

# Create NIC VFs
# Starting SR-IOV
ExecStart=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f0np0/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f1np1/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f2np2/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f3np3/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c '/usr/bin/echo 8 > /sys/class/net/enp33s0f0np0/device/sriov_numvfs'
ExecStart=/usr/bin/bash -c '/usr/bin/echo 8 > /sys/class/net/enp33s0f1np1/device/sriov_numvfs'

# Setting static MAC for VFs

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 0 mac aa:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 1 mac aa:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 2 mac aa:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 3 mac aa:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 4 mac aa:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 5 mac aa:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 6 mac aa:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 7 mac aa:a7:86:7d:85:e7'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 8 mac aa:a7:86:7d:85:e8'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 9 mac aa:a7:86:7d:85:e9'

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 0 mac ba:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 1 mac ba:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 2 mac ba:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 3 mac ba:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 4 mac ba:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 5 mac ba:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 6 mac ba:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 7 mac ba:a7:86:7d:85:e7'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 8 mac ba:a7:86:7d:85:e8'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 9 mac ba:a7:86:7d:85:e9'

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 0 mac ca:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 1 mac ca:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 2 mac ca:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 3 mac ca:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 4 mac ca:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 5 mac ca:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 6 mac ca:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 7 mac ca:a7:86:7d:85:e7'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 8 mac ca:a7:86:7d:85:e8'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 9 mac ca:a7:86:7d:85:e9'

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 0 mac da:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 1 mac da:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 2 mac da:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 3 mac da:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 4 mac da:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 5 mac da:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 6 mac da:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 7 mac da:a7:86:7d:85:e7'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 8 mac da:a7:86:7d:85:e8'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 9 mac da:a7:86:7d:85:e9'

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 0 mac ea:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 1 mac ea:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 2 mac ea:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 3 mac ea:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 4 mac ea:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 5 mac ea:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 6 mac ea:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 7 mac ea:a7:86:7d:85:e7'

ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 0 mac fa:a7:86:7d:85:e0'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 1 mac fa:a7:86:7d:85:e1'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 2 mac fa:a7:86:7d:85:e2'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 3 mac fa:a7:86:7d:85:e3'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 4 mac fa:a7:86:7d:85:e4'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 5 mac fa:a7:86:7d:85:e5'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 6 mac fa:a7:86:7d:85:e6'
ExecStart=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 7 mac fa:a7:86:7d:85:e7'

# Detach VFs from host

ExecStart=/usr/bin/bash -c 'echo 0000:62:02.1 > /sys/bus/pci/devices/0000\\:62\\:02.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.2 > /sys/bus/pci/devices/0000\\:62\\:02.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.3 > /sys/bus/pci/devices/0000\\:62\\:02.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.4 > /sys/bus/pci/devices/0000\\:62\\:02.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.5 > /sys/bus/pci/devices/0000\\:62\\:02.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.6 > /sys/bus/pci/devices/0000\\:62\\:02.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:02.7 > /sys/bus/pci/devices/0000\\:62\\:02.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:03.0 > /sys/bus/pci/devices/0000\\:62\\:03.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:03.1 > /sys/bus/pci/devices/0000\\:62\\:03.1/driver/unbind'

ExecStart=/usr/bin/bash -c 'echo 0000:62:06.1 > /sys/bus/pci/devices/0000\\:62\\:06.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.2 > /sys/bus/pci/devices/0000\\:62\\:06.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.3 > /sys/bus/pci/devices/0000\\:62\\:06.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.4 > /sys/bus/pci/devices/0000\\:62\\:06.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.5 > /sys/bus/pci/devices/0000\\:62\\:06.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.6 > /sys/bus/pci/devices/0000\\:62\\:06.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:06.7 > /sys/bus/pci/devices/0000\\:62\\:06.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:07.0 > /sys/bus/pci/devices/0000\\:62\\:07.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:07.1 > /sys/bus/pci/devices/0000\\:62\\:07.1/driver/unbind'

ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.1 > /sys/bus/pci/devices/0000\\:62\\:0a.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.2 > /sys/bus/pci/devices/0000\\:62\\:0a.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.3 > /sys/bus/pci/devices/0000\\:62\\:0a.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.4 > /sys/bus/pci/devices/0000\\:62\\:0a.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.5 > /sys/bus/pci/devices/0000\\:62\\:0a.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.6 > /sys/bus/pci/devices/0000\\:62\\:0a.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0a.7 > /sys/bus/pci/devices/0000\\:62\\:0a.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0b.0 > /sys/bus/pci/devices/0000\\:62\\:0b.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0b.1 > /sys/bus/pci/devices/0000\\:62\\:0b.1/driver/unbind'

ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.1 > /sys/bus/pci/devices/0000\\:62\\:0e.1/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.2 > /sys/bus/pci/devices/0000\\:62\\:0e.2/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.3 > /sys/bus/pci/devices/0000\\:62\\:0e.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.4 > /sys/bus/pci/devices/0000\\:62\\:0e.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.5 > /sys/bus/pci/devices/0000\\:62\\:0e.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.6 > /sys/bus/pci/devices/0000\\:62\\:0e.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0e.7 > /sys/bus/pci/devices/0000\\:62\\:0e.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0f.0 > /sys/bus/pci/devices/0000\\:62\\:0f.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:62:0f.1 > /sys/bus/pci/devices/0000\\:62\\:0f.1/driver/unbind'

ExecStart=/usr/bin/bash -c 'echo 0000:21:00.3 > /sys/bus/pci/devices/0000\\:21\\:00.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:00.4 > /sys/bus/pci/devices/0000\\:21\\:00.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:00.5 > /sys/bus/pci/devices/0000\\:21\\:00.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:00.6 > /sys/bus/pci/devices/0000\\:21\\:00.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:00.7 > /sys/bus/pci/devices/0000\\:21\\:00.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:01.0 > /sys/bus/pci/devices/0000\\:21\\:01.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:01.1 > /sys/bus/pci/devices/0000\\:21\\:01.1/driver/unbind'

ExecStart=/usr/bin/bash -c 'echo 0000:21:04.3 > /sys/bus/pci/devices/0000\\:21\\:04.3/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:04.4 > /sys/bus/pci/devices/0000\\:21\\:04.4/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:04.5 > /sys/bus/pci/devices/0000\\:21\\:04.5/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:04.6 > /sys/bus/pci/devices/0000\\:21\\:04.6/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:04.7 > /sys/bus/pci/devices/0000\\:21\\:04.7/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:05.0 > /sys/bus/pci/devices/0000\\:21\\:05.0/driver/unbind'
ExecStart=/usr/bin/bash -c 'echo 0000:21:05.1 > /sys/bus/pci/devices/0000\\:21\\:05.1/driver/unbind'

# List new VFs
ExecStart=/usr/bin/lspci -D -d 8086:154c
ExecStart=/usr/bin/lspci -D -d 15b3:101a

# Destroy VFs
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f0np0/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f1np1/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f2np2/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f3np3/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp33s0f0np0/device/sriov_numvfs'
ExecStop=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp33s0f1np1/device/sriov_numvfs'

# Reload NIC VFs
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f1np1/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f2np2/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp98s0f3np3/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp33s0f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 0 > /sys/class/net/enp33s0f1np1/device/sriov_numvfs'

ExecReload=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f1np1/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f2np2/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp98s0f3np3/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 8 > /sys/class/net/enp33s0f0np0/device/sriov_numvfs'
ExecReload=/usr/bin/bash -c '/usr/bin/echo 8 > /sys/class/net/enp33s0f1np1/device/sriov_numvfs'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 0 mac aa:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 1 mac aa:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 2 mac aa:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 3 mac aa:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 4 mac aa:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 5 mac aa:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 6 mac aa:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 7 mac aa:a7:86:7d:85:e7'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 8 mac aa:a7:86:7d:85:e8'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f0np0 vf 9 mac aa:a7:86:7d:85:e9'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 0 mac ba:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 1 mac ba:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 2 mac ba:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 3 mac ba:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 4 mac ba:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 5 mac ba:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 6 mac ba:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 7 mac ba:a7:86:7d:85:e7'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 8 mac ba:a7:86:7d:85:e8'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f1np1 vf 9 mac ba:a7:86:7d:85:e9'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 0 mac ca:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 1 mac ca:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 2 mac ca:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 3 mac ca:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 4 mac ca:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 5 mac ca:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 6 mac ca:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 7 mac ca:a7:86:7d:85:e7'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 8 mac ca:a7:86:7d:85:e8'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f2np2 vf 9 mac ca:a7:86:7d:85:e9'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 0 mac da:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 1 mac da:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 2 mac da:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 3 mac da:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 4 mac da:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 5 mac da:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 6 mac da:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 7 mac da:a7:86:7d:85:e7'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 8 mac da:a7:86:7d:85:e8'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp98s0f3np3 vf 9 mac da:a7:86:7d:85:e9'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 0 mac ea:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 1 mac ea:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 2 mac ea:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 3 mac ea:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 4 mac ea:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 5 mac ea:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 6 mac ea:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f0np0 vf 7 mac ea:a7:86:7d:85:e7'

ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 0 mac fa:a7:86:7d:85:e0'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 1 mac fa:a7:86:7d:85:e1'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 2 mac fa:a7:86:7d:85:e2'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 3 mac fa:a7:86:7d:85:e3'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 4 mac fa:a7:86:7d:85:e4'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 5 mac fa:a7:86:7d:85:e5'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 6 mac fa:a7:86:7d:85:e6'
ExecReload=/usr/bin/bash -c '/usr/bin/ip link set enp33s0f1np1 vf 7 mac fa:a7:86:7d:85:e7'

ExecReload=/usr/bin/bash -c 'echo 0000:62:02.1 > /sys/bus/pci/devices/0000\\:62\\:02.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.2 > /sys/bus/pci/devices/0000\\:62\\:02.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.3 > /sys/bus/pci/devices/0000\\:62\\:02.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.4 > /sys/bus/pci/devices/0000\\:62\\:02.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.5 > /sys/bus/pci/devices/0000\\:62\\:02.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.6 > /sys/bus/pci/devices/0000\\:62\\:02.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:02.7 > /sys/bus/pci/devices/0000\\:62\\:02.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:03.0 > /sys/bus/pci/devices/0000\\:62\\:03.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:03.1 > /sys/bus/pci/devices/0000\\:62\\:03.1/driver/unbind'

ExecReload=/usr/bin/bash -c 'echo 0000:62:06.1 > /sys/bus/pci/devices/0000\\:62\\:06.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.2 > /sys/bus/pci/devices/0000\\:62\\:06.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.3 > /sys/bus/pci/devices/0000\\:62\\:06.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.4 > /sys/bus/pci/devices/0000\\:62\\:06.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.5 > /sys/bus/pci/devices/0000\\:62\\:06.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.6 > /sys/bus/pci/devices/0000\\:62\\:06.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:06.7 > /sys/bus/pci/devices/0000\\:62\\:06.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:07.0 > /sys/bus/pci/devices/0000\\:62\\:07.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:07.1 > /sys/bus/pci/devices/0000\\:62\\:07.1/driver/unbind'

ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.1 > /sys/bus/pci/devices/0000\\:62\\:0a.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.2 > /sys/bus/pci/devices/0000\\:62\\:0a.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.3 > /sys/bus/pci/devices/0000\\:62\\:0a.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.4 > /sys/bus/pci/devices/0000\\:62\\:0a.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.5 > /sys/bus/pci/devices/0000\\:62\\:0a.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.6 > /sys/bus/pci/devices/0000\\:62\\:0a.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0a.7 > /sys/bus/pci/devices/0000\\:62\\:0a.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0b.0 > /sys/bus/pci/devices/0000\\:62\\:0b.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0b.1 > /sys/bus/pci/devices/0000\\:62\\:0b.1/driver/unbind'

ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.1 > /sys/bus/pci/devices/0000\\:62\\:0e.1/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.2 > /sys/bus/pci/devices/0000\\:62\\:0e.2/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.3 > /sys/bus/pci/devices/0000\\:62\\:0e.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.4 > /sys/bus/pci/devices/0000\\:62\\:0e.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.5 > /sys/bus/pci/devices/0000\\:62\\:0e.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.6 > /sys/bus/pci/devices/0000\\:62\\:0e.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0e.7 > /sys/bus/pci/devices/0000\\:62\\:0e.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0f.0 > /sys/bus/pci/devices/0000\\:62\\:0f.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:62:0f.1 > /sys/bus/pci/devices/0000\\:62\\:0f.1/driver/unbind'

ExecReload=/usr/bin/bash -c 'echo 0000:21:00.3 > /sys/bus/pci/devices/0000\\:21\\:00.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:00.4 > /sys/bus/pci/devices/0000\\:21\\:00.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:00.5 > /sys/bus/pci/devices/0000\\:21\\:00.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:00.6 > /sys/bus/pci/devices/0000\\:21\\:00.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:00.7 > /sys/bus/pci/devices/0000\\:21\\:00.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:01.0 > /sys/bus/pci/devices/0000\\:21\\:01.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:01.1 > /sys/bus/pci/devices/0000\\:21\\:01.1/driver/unbind'

ExecReload=/usr/bin/bash -c 'echo 0000:21:04.3 > /sys/bus/pci/devices/0000\\:21\\:04.3/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:04.4 > /sys/bus/pci/devices/0000\\:21\\:04.4/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:04.5 > /sys/bus/pci/devices/0000\\:21\\:04.5/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:04.6 > /sys/bus/pci/devices/0000\\:21\\:04.6/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:04.7 > /sys/bus/pci/devices/0000\\:21\\:04.7/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:05.0 > /sys/bus/pci/devices/0000\\:21\\:05.0/driver/unbind'
ExecReload=/usr/bin/bash -c 'echo 0000:21:05.1 > /sys/bus/pci/devices/0000\\:21\\:05.1/driver/unbind'

ExecReload=/usr/bin/lspci -D -d 8086:154c
ExecReload=/usr/bin/lspci -D -d 15b3:101a

[Install]
WantedBy=multi-user.target

This is the PVE network screen in proxmox:

This is the IP A output:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9c brd ff:ff:ff:ff:ff:ff
altname enp4s0
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9d brd ff:ff:ff:ff:ff:ff
altname enp5s0
4: enp98s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:14 brd ff:ff:ff:ff:ff:ff
5: enp98s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:15 brd ff:ff:ff:ff:ff:ff
6: enp98s0f2np2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:16 brd ff:ff:ff:ff:ff:ff
7: enp98s0f3np3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:17 brd ff:ff:ff:ff:ff:ff
8: enp33s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:ce:f6:cf:93:d8 brd ff:ff:ff:ff:ff:ff
9: enp33s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:ce:f6:cf:93:d9 brd ff:ff:ff:ff:ff:ff
10: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9c brd ff:ff:ff:ff:ff:ff
inet 10.100.10.20/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::3eec:efff:fe6d:d69c/64 scope link
valid_lft forever preferred_lft forever
12: enp98s0f0v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether aa:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
26: enp98s0f1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ba:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
31: enp33s0f0v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 42:fa:7e:91:0e:9e brd ff:ff:ff:ff:ff:ff
34: enp98s0f2v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ca:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
42: enp98s0f3v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether da:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
59: enp33s0f1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 0e:3f:e7:70:4a:6e brd ff:ff:ff:ff:ff:ff

It seems strings setting the mac addresses for the VF on the Mellanox isn't right. Because both mac addresses listed from IP A commend went rogue on the VF's for the Mellanox cards... But the output using IP Link show Dev command outputs correctly... Anyone care to jump in here and educate me ?

I left the first VF's of all the root devices per the example (Mellanox PCI 21.04.2 & 21.00.2) out of the reload.

root@pve00:~# ip link show dev enp33s0f0np0
8: enp33s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether b8:ce:f6:cf:93:d8 brd ff:ff:ff:ff:ff:ff
vf 0 link/ether ea:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 1 link/ether ea:a7:86:7d:85:e1 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 2 link/ether ea:a7:86:7d:85:e2 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 3 link/ether ea:a7:86:7d:85:e3 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 4 link/ether ea:a7:86:7d:85:e4 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 5 link/ether ea:a7:86:7d:85:e5 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 6 link/ether ea:a7:86:7d:85:e6 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 7 link/ether ea:a7:86:7d:85:e7 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off

root@pve00:~# ip link show dev enp33s0f1np1
9: enp33s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether b8:ce:f6:cf:93:d9 brd ff:ff:ff:ff:ff:ff
vf 0 link/ether fa:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 1 link/ether fa:a7:86:7d:85:e1 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 2 link/ether fa:a7:86:7d:85:e2 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 3 link/ether fa:a7:86:7d:85:e3 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 4 link/ether fa:a7:86:7d:85:e4 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 5 link/ether fa:a7:86:7d:85:e5 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 6 link/ether fa:a7:86:7d:85:e6 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off
vf 7 link/ether fa:a7:86:7d:85:e7 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust off, query_rss off

Let me know if you have any ideas. My google foo didn't pop anything up.

heromode · May 11, 2025

It seems strings setting the mac addresses for the VF on the Mellanox isn't right. Because both mac addresses listed from IP A commend went rogue on the VF's for the Mellanox cards... But the output using IP Link show Dev command outputs correctly... Anyone care to jump in here and educate me ?

I left the first VF's of all the root devices per the example (Mellanox PCI 21.04.2 & 21.00.2) out of the reload.

Let me know if you have any ideas. My google foo didn't pop anything up.
[/QUOTE]

I don't really know your issue.. but generally:

What does the output of 'systemctl status sriov-vfs.service' and 'journalctl -u sriov-vfs.service' look like?

As i explained in the post, on my old solarflare cards, they would automatically generate random MAc addresses each time they were partitioned. So i used those addresses and made them static in the script.

But that didn't happen with the intel X710. It would initially set the macs to 00:00:00:00:00:00 on all vf's.

So i headed to a random mac generator on the web, and generated a set for my self. But it turns out, not all MAC addresses are created equal. Basically, about every other address was not accepted by the x710, and produced an error when tried to set it for the vf.

So i ended up having to manually test setting a MAC for each vf, until they all worked.

So wherever you see a problem (i haven't looked at your output closely, just waking up), first try to manually do the commands in the script, for example manually create the partitions, then check to see if mellanox nic has created random MAC's automatically. If it has, then those MAC's will work for sure.

Also manually try 'ip link set enp98s0xxxxx vf 1' and 'echo 0000:xx:xx.x > /sys/bus/pci/devices/0000\\:xx\\:xx.x/driver/unbind' etc etc, and check you don't get any errors..

edit: btw when you do the unbind manually, you should not use the double \\, just single.. use TAB to get it right.
The double \\ backlashes in the script is to prevent some annoying but non-fatal message a single \ produces.. something to do with bash.. i forget what it was..

tillburn · May 11, 2025

Thank you for the reply Heromode! (again).

I don't see any errors, looks clean. I will mess around with the the script in segments like you suggested see if I can find the point of failure, great advice. I will also search for SR IOV and the connectx5 cards see if I can find a post with people setting statics and compare scripts, last night I didn't find much.

root@pve00:~# journalctl -u sriov-vfs.service
May 10 18:56:29 pve00 systemd[1]: Starting sriov-vfs.service - Script to enable SR-IOV on boot...
May 10 18:56:46 pve00 systemd[1]: Finished sriov-vfs.service - Script to enable SR-IOV on boot.
May 10 20:03:41 pve00 systemd[1]: Stopping sriov-vfs.service - Enable SR-IOV and detach guest VFs from host...
May 10 20:04:03 pve00 systemd[1]: sriov-vfs.service: Deactivated successfully.
May 10 20:04:03 pve00 systemd[1]: Stopped sriov-vfs.service - Enable SR-IOV and detach guest VFs from host.
-- Boot bd087fda535b420a98d61b9acd9f62f6 --
May 10 20:06:57 pve00 systemd[1]: Starting sriov-vfs.service - Enable SR-IOV and detach guest VFs from host...
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:03.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:03.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:06.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:07.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:07.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0b.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0b.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0e.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0f.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5516]: 0000:62:0f.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.6 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:00.7 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:01.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:01.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.6 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:04.7 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:05.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 lspci[5518]: 0000:21:05.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 20:07:27 pve00 systemd[1]: Finished sriov-vfs.service - Enable SR-IOV and detach guest VFs from host.
May 10 21:11:21 pve00 systemd[1]: Stopping sriov-vfs.service - Enable SR-IOV and detach guest VFs from host...
May 10 21:11:30 pve00 systemd[1]: sriov-vfs.service: Deactivated successfully.
May 10 21:11:30 pve00 systemd[1]: Stopped sriov-vfs.service - Enable SR-IOV and detach guest VFs from host.
-- Boot 2577e8017b5244e7bea4ebef58318e9b --
May 10 21:14:27 pve00 systemd[1]: Starting sriov-vfs.service - Enable SR-IOV and detach guest VFs from host...
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:03.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:03.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:06.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:07.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:07.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0b.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0b.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0e.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0f.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5174]: 0000:62:0f.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.6 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:00.7 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:01.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:01.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.6 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.7 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:05.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:05.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 systemd[1]: Finished sriov-vfs.service - Enable SR-IOV and detach guest VFs from host.

&

root@pve00:~# systemctl status sriov-vfs.service
● sriov-vfs.service - Enable SR-IOV and detach guest VFs from host
Loaded: loaded (/etc/systemd/system/sriov-vfs.service; enabled; preset: enabled)
Active: active (exited) since Sat 2025-05-10 21:14:57 PDT; 9h ago
Process: 4311 ExecStart=/usr/bin/bash -c /usr/bin/echo 10 > /sys/class/net/enp98s0f0np0/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4453 ExecStart=/usr/bin/bash -c /usr/bin/echo 10 > /sys/class/net/enp98s0f1np1/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4531 ExecStart=/usr/bin/bash -c /usr/bin/echo 10 > /sys/class/net/enp98s0f2np2/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4565 ExecStart=/usr/bin/bash -c /usr/bin/echo 10 > /sys/class/net/enp98s0f3np3/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4603 ExecStart=/usr/bin/bash -c /usr/bin/echo 8 > /sys/class/net/enp33s0f0np0/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4815 ExecStart=/usr/bin/bash -c /usr/bin/echo 8 > /sys/class/net/enp33s0f1np1/device/sriov_numvfs (code=exited, status=0/SUCCESS)
Process: 4918 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 0 mac aa:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 4919 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 1 mac aa:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 4922 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 2 mac aa:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 4923 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 3 mac aa:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 4924 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 4 mac aa:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 4925 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 5 mac aa:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 4927 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 6 mac aa:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 4928 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 7 mac aa:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 4931 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 8 mac aa:a7:86:7d:85:e8 (code=exited, status=0/SUCCESS)
Process: 4932 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f0np0 vf 9 mac aa:a7:86:7d:85:e9 (code=exited, status=0/SUCCESS)
Process: 4933 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 0 mac ba:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 4935 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 1 mac ba:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 4936 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 2 mac ba:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 4937 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 3 mac ba:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 4938 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 4 mac ba:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 4939 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 5 mac ba:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 4943 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 6 mac ba:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 4944 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 7 mac ba:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 4955 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 8 mac ba:a7:86:7d:85:e8 (code=exited, status=0/SUCCESS)
Process: 4956 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f1np1 vf 9 mac ba:a7:86:7d:85:e9 (code=exited, status=0/SUCCESS)
Process: 4957 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 0 mac ca:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 4958 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 1 mac ca:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 4959 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 2 mac ca:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 4961 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 3 mac ca:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 4964 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 4 mac ca:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 4975 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 5 mac ca:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 4986 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 6 mac ca:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 4992 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 7 mac ca:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 4993 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 8 mac ca:a7:86:7d:85:e8 (code=exited, status=0/SUCCESS)
Process: 4994 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f2np2 vf 9 mac ca:a7:86:7d:85:e9 (code=exited, status=0/SUCCESS)
Process: 4995 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 0 mac da:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 4996 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 1 mac da:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 4997 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 2 mac da:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 4998 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 3 mac da:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 4999 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 4 mac da:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 5000 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 5 mac da:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 5001 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 6 mac da:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 5002 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 7 mac da:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 5003 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 8 mac da:a7:86:7d:85:e8 (code=exited, status=0/SUCCESS)
Process: 5004 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp98s0f3np3 vf 9 mac da:a7:86:7d:85:e9 (code=exited, status=0/SUCCESS)
Process: 5005 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 0 mac ea:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 5006 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 1 mac ea:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 5007 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 2 mac ea:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 5008 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 3 mac ea:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 5009 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 4 mac ea:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 5010 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 5 mac ea:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 5011 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 6 mac ea:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 5012 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 7 mac ea:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 5013 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 0 mac fa:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)
Process: 5014 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 1 mac fa:a7:86:7d:85:e1 (code=exited, status=0/SUCCESS)
Process: 5015 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 2 mac fa:a7:86:7d:85:e2 (code=exited, status=0/SUCCESS)
Process: 5016 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 3 mac fa:a7:86:7d:85:e3 (code=exited, status=0/SUCCESS)
Process: 5017 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 4 mac fa:a7:86:7d:85:e4 (code=exited, status=0/SUCCESS)
Process: 5018 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 5 mac fa:a7:86:7d:85:e5 (code=exited, status=0/SUCCESS)
Process: 5019 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 6 mac fa:a7:86:7d:85:e6 (code=exited, status=0/SUCCESS)
Process: 5020 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f1np1 vf 7 mac fa:a7:86:7d:85:e7 (code=exited, status=0/SUCCESS)
Process: 5021 ExecStart=/usr/bin/bash -c echo 0000:62:02.1 > /sys/bus/pci/devices/0000\:62\:02.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5028 ExecStart=/usr/bin/bash -c echo 0000:62:02.2 > /sys/bus/pci/devices/0000\:62\:02.2/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5032 ExecStart=/usr/bin/bash -c echo 0000:62:02.3 > /sys/bus/pci/devices/0000\:62\:02.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5035 ExecStart=/usr/bin/bash -c echo 0000:62:02.4 > /sys/bus/pci/devices/0000\:62\:02.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5038 ExecStart=/usr/bin/bash -c echo 0000:62:02.5 > /sys/bus/pci/devices/0000\:62\:02.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5040 ExecStart=/usr/bin/bash -c echo 0000:62:02.6 > /sys/bus/pci/devices/0000\:62\:02.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5042 ExecStart=/usr/bin/bash -c echo 0000:62:02.7 > /sys/bus/pci/devices/0000\:62\:02.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5044 ExecStart=/usr/bin/bash -c echo 0000:62:03.0 > /sys/bus/pci/devices/0000\:62\:03.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5046 ExecStart=/usr/bin/bash -c echo 0000:62:03.1 > /sys/bus/pci/devices/0000\:62\:03.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5048 ExecStart=/usr/bin/bash -c echo 0000:62:06.1 > /sys/bus/pci/devices/0000\:62\:06.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5050 ExecStart=/usr/bin/bash -c echo 0000:62:06.2 > /sys/bus/pci/devices/0000\:62\:06.2/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5055 ExecStart=/usr/bin/bash -c echo 0000:62:06.3 > /sys/bus/pci/devices/0000\:62\:06.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5057 ExecStart=/usr/bin/bash -c echo 0000:62:06.4 > /sys/bus/pci/devices/0000\:62\:06.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5059 ExecStart=/usr/bin/bash -c echo 0000:62:06.5 > /sys/bus/pci/devices/0000\:62\:06.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5061 ExecStart=/usr/bin/bash -c echo 0000:62:06.6 > /sys/bus/pci/devices/0000\:62\:06.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5085 ExecStart=/usr/bin/bash -c echo 0000:62:06.7 > /sys/bus/pci/devices/0000\:62\:06.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5087 ExecStart=/usr/bin/bash -c echo 0000:62:07.0 > /sys/bus/pci/devices/0000\:62\:07.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5090 ExecStart=/usr/bin/bash -c echo 0000:62:07.1 > /sys/bus/pci/devices/0000\:62\:07.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5092 ExecStart=/usr/bin/bash -c echo 0000:62:0a.1 > /sys/bus/pci/devices/0000\:62\:0a.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5094 ExecStart=/usr/bin/bash -c echo 0000:62:0a.2 > /sys/bus/pci/devices/0000\:62\:0a.2/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5096 ExecStart=/usr/bin/bash -c echo 0000:62:0a.3 > /sys/bus/pci/devices/0000\:62\:0a.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5098 ExecStart=/usr/bin/bash -c echo 0000:62:0a.4 > /sys/bus/pci/devices/0000\:62\:0a.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5100 ExecStart=/usr/bin/bash -c echo 0000:62:0a.5 > /sys/bus/pci/devices/0000\:62\:0a.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5103 ExecStart=/usr/bin/bash -c echo 0000:62:0a.6 > /sys/bus/pci/devices/0000\:62\:0a.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5105 ExecStart=/usr/bin/bash -c echo 0000:62:0a.7 > /sys/bus/pci/devices/0000\:62\:0a.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5107 ExecStart=/usr/bin/bash -c echo 0000:62:0b.0 > /sys/bus/pci/devices/0000\:62\:0b.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5109 ExecStart=/usr/bin/bash -c echo 0000:62:0b.1 > /sys/bus/pci/devices/0000\:62\:0b.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5111 ExecStart=/usr/bin/bash -c echo 0000:62:0e.1 > /sys/bus/pci/devices/0000\:62\:0e.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5113 ExecStart=/usr/bin/bash -c echo 0000:62:0e.2 > /sys/bus/pci/devices/0000\:62\:0e.2/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5115 ExecStart=/usr/bin/bash -c echo 0000:62:0e.3 > /sys/bus/pci/devices/0000\:62\:0e.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5117 ExecStart=/usr/bin/bash -c echo 0000:62:0e.4 > /sys/bus/pci/devices/0000\:62\:0e.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5119 ExecStart=/usr/bin/bash -c echo 0000:62:0e.5 > /sys/bus/pci/devices/0000\:62\:0e.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5121 ExecStart=/usr/bin/bash -c echo 0000:62:0e.6 > /sys/bus/pci/devices/0000\:62\:0e.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5123 ExecStart=/usr/bin/bash -c echo 0000:62:0e.7 > /sys/bus/pci/devices/0000\:62\:0e.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5125 ExecStart=/usr/bin/bash -c echo 0000:62:0f.0 > /sys/bus/pci/devices/0000\:62\:0f.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5127 ExecStart=/usr/bin/bash -c echo 0000:62:0f.1 > /sys/bus/pci/devices/0000\:62\:0f.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5129 ExecStart=/usr/bin/bash -c echo 0000:21:00.3 > /sys/bus/pci/devices/0000\:21\:00.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5131 ExecStart=/usr/bin/bash -c echo 0000:21:00.4 > /sys/bus/pci/devices/0000\:21\:00.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5133 ExecStart=/usr/bin/bash -c echo 0000:21:00.5 > /sys/bus/pci/devices/0000\:21\:00.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5138 ExecStart=/usr/bin/bash -c echo 0000:21:00.6 > /sys/bus/pci/devices/0000\:21\:00.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5145 ExecStart=/usr/bin/bash -c echo 0000:21:00.7 > /sys/bus/pci/devices/0000\:21\:00.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5147 ExecStart=/usr/bin/bash -c echo 0000:21:01.0 > /sys/bus/pci/devices/0000\:21\:01.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5152 ExecStart=/usr/bin/bash -c echo 0000:21:01.1 > /sys/bus/pci/devices/0000\:21\:01.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5154 ExecStart=/usr/bin/bash -c echo 0000:21:04.3 > /sys/bus/pci/devices/0000\:21\:04.3/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5156 ExecStart=/usr/bin/bash -c echo 0000:21:04.4 > /sys/bus/pci/devices/0000\:21\:04.4/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5158 ExecStart=/usr/bin/bash -c echo 0000:21:04.5 > /sys/bus/pci/devices/0000\:21\:04.5/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5163 ExecStart=/usr/bin/bash -c echo 0000:21:04.6 > /sys/bus/pci/devices/0000\:21\:04.6/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5168 ExecStart=/usr/bin/bash -c echo 0000:21:04.7 > /sys/bus/pci/devices/0000\:21\:04.7/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5170 ExecStart=/usr/bin/bash -c echo 0000:21:05.0 > /sys/bus/pci/devices/0000\:21\:05.0/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5172 ExecStart=/usr/bin/bash -c echo 0000:21:05.1 > /sys/bus/pci/devices/0000\:21\:05.1/driver/unbind (code=exited, status=0/SUCCESS)
Process: 5174 ExecStart=/usr/bin/lspci -D -d 8086:154c (code=exited, status=0/SUCCESS)
Process: 5176 ExecStart=/usr/bin/lspci -D -d 15b3:101a (code=exited, status=0/SUCCESS)
Main PID: 5176 (code=exited, status=0/SUCCESS)
CPU: 1.370s

May 10 21:14:57 pve00 lspci[5176]: 0000:21:01.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.6 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:04.7 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:05.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 lspci[5176]: 0000:21:05.1 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
May 10 21:14:57 pve00 systemd[1]: Finished sriov-vfs.service - Enable SR-IOV and detach guest VFs from host.

heromode · May 11, 2025

tillburn said:
Thank you for the reply Heromode! (again).

I don't see any errors, looks clean. I will mess around with the the script in segments like you suggested see if I can find the point of failure, great advice. I will also search for SR IOV and the connectx5 cards see if I can find a post with people setting statics and compare scripts, last night I didn't find much.

Main PID: 5176 (code=exited, status=0/SUCCESS)

There is no problem i can see, looks fine

And command ip a will not show the detached vf's at all, because they are detached

I don't know what you believe is the issue..

tillburn · May 11, 2025

I set the mac addresses in the script even for the VF's highlighted down below, but it seems to be showing a difference mac address than I called out in the script or was expecting to show in proxmox.

Red = PF
Orange = VF

As you can see the mac I called out in the script set correctly for the x710's VF's, but not the connectx5.

Process: 5005 ExecStart=/usr/bin/bash -c /usr/bin/ip link set enp33s0f0np0 vf 0 mac ea:a7:86:7d:85:e0 (code=exited, status=0/SUCCESS)

This is the IP A output:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9c brd ff:ff:ff:ff:ff:ff
altname enp4s0
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9d brd ff:ff:ff:ff:ff:ff
altname enp5s0
4: enp98s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:14 brd ff:ff:ff:ff:ff:ff
5: enp98s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:15 brd ff:ff:ff:ff:ff:ff
6: enp98s0f2np2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:16 brd ff:ff:ff:ff:ff:ff
7: enp98s0f3np3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:e0:ed:9a:0f:17 brd ff:ff:ff:ff:ff:ff
8: enp33s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:ce:f6:cf:93:d8 brd ff:ff:ff:ff:ff:ff
9: enp33s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:ce:f6:cf:93:d9 brd ff:ff:ff:ff:ff:ff
10: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:ec:ef:6d:d6:9c brd ff:ff:ff:ff:ff:ff
inet 10.100.10.20/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::3eec:efff:fe6d:d69c/64 scope link
valid_lft forever preferred_lft forever
12: enp98s0f0v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether aa:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
26: enp98s0f1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ba:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
31: enp33s0f0v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 42:fa:7e:91:0e:9e brd ff:ff:ff:ff:ff:ff
34: enp98s0f2v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ca:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
42: enp98s0f3v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether da:a7:86:7d:85:e0 brd ff:ff:ff:ff:ff:ff
59: enp33s0f1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 0e:3f:e7:70:4a:6e brd ff:ff:ff:ff:ff:ff

heromode · May 11, 2025

Oh i see, yeah, that might be a mellanox issue. As i said, my old solarflare cards generated random mac's when partitioned, but i could then change those.

My X710 didn't generate random mac's, just all 00

And never had the chance to test on Mellanox

So apparently things are different with Mellanox. That is a google-fu issue, might very well be something you can configure in the ROM, or anything..
I'm sure there is info on this, google 'set custom MAC for mellanox vf's or something.. good luck!

edit: if the case is that Mellanox doesn't accept the MAC's you want it to have, but doesn't produce an error when trying to set them like the X710 did, just silently ignores them, then one option would be to first manually partition it, and document the random MAC's it generates, and then use those in the script to make them permanent.

tillburn · May 11, 2025

Basic command: “ip link set vf mac xx:xx:xx:xx:xx:xx”

i.e.
ip link set enp33s0f0v0 mac ea:a7:86:7d:85:e0

heromode · May 11, 2025

tillburn said:
Basic command: “ip link set vf mac xx:xx:xx:xx:xx:xx”

i.e.
ip link set enp33s0f0v0 mac ea:a7:86:7d:85:e0

no, ip link set enp33s0f0np0 vf 0 mac ea:a7:86:7d:85:e0 i believe..

QNAP QXG-10G2T-X710 discussion thread

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member