Being a freshman on SR-IOV config, I'm desperate and would like to hear some advice from experts.
What I'm trying to do is to setup a 10GE direct link, without involving switches, between my desktop and my home server.
The intended usage is to split LAN and Internet access from my desktop between the new 10GE and the existing 1GE NICs (on a separated subnet).
As a result, I need to setup my home server as a bridge between the new 10GE NIC and its 1GE NICs to forward my desktop traffic between my other LAN devices.
This would sound like a trivial setup, and yes it is trivial until I enabled the SR-IOV mode to the 10GE NIC in my server.
With SR-IOV enabled, it appears that the 10GE NIC does not properly enter promiscuous mode.
It's true that the kernel reports "device enp1s0d1 entered promiscuous mode", the observed behavior is more like it remains in normal mode:
The desktop side can receive traffic from the server side just fine.
But it cannot respond as the 10GE in SR-IOV mode still drops any packet that does not match its MAC.
I'm aware that setting up bridge behind SR-IOV VF ports may need additional setting due to security consideration.
But I didn't find any document saying that the PF port would share the same limitation.
So should I expect a SR-IOV PF could work like normal NIC in Linux bridge?
Assuming the answer is positive, how should I proceed from my current mess?
So far I've updated my NICS to latest firmware version (2.42.5000 for ConnecteX-3) but this does not fix anything.
The driver I'm using is the stock driver coming with the Linux Kernel 5.10.
Didn't try the Mellanox driver release yet. Should I expect any difference from a different driver version?
I was expecting things are mature for such an old model...
What I'm trying to do is to setup a 10GE direct link, without involving switches, between my desktop and my home server.
The intended usage is to split LAN and Internet access from my desktop between the new 10GE and the existing 1GE NICs (on a separated subnet).
As a result, I need to setup my home server as a bridge between the new 10GE NIC and its 1GE NICs to forward my desktop traffic between my other LAN devices.
This would sound like a trivial setup, and yes it is trivial until I enabled the SR-IOV mode to the 10GE NIC in my server.
With SR-IOV enabled, it appears that the 10GE NIC does not properly enter promiscuous mode.
It's true that the kernel reports "device enp1s0d1 entered promiscuous mode", the observed behavior is more like it remains in normal mode:
The desktop side can receive traffic from the server side just fine.
But it cannot respond as the 10GE in SR-IOV mode still drops any packet that does not match its MAC.
I'm aware that setting up bridge behind SR-IOV VF ports may need additional setting due to security consideration.
But I didn't find any document saying that the PF port would share the same limitation.
So should I expect a SR-IOV PF could work like normal NIC in Linux bridge?
Assuming the answer is positive, how should I proceed from my current mess?
So far I've updated my NICS to latest firmware version (2.42.5000 for ConnecteX-3) but this does not fix anything.
The driver I'm using is the stock driver coming with the Linux Kernel 5.10.
Didn't try the Mellanox driver release yet. Should I expect any difference from a different driver version?
I was expecting things are mature for such an old model...