nftables flowtables hardware offload with ConnectX-5

[nexox]

I realized last week that my router's NIC supports hardware offload for bridging, forwarding, and NAT, and while I don't need acceleration it at all, I've got nothing to do next week but break my network trying to see how it works.

I think I read just about everything there is to find on the subject, which isn't very much, and I was curious if anyone here had some experience to share before I jump in.

The most useful source I've found so far is this very detailed blog post: Flowtables - Part 1: A Netfilter/Nftables Fastpath [Thermalcircle.de] and beyond that it's pretty much just the patch description and sources found here: netfilter flowtable hardware offload [LWN.net]. All other references seem too high-level and vague to offer much help.

 

[blunden]

You could perhaps find a bit more information from the documentation for VyOS and OpenWrt.

At least VyOS supports it so if nothing else, looking at the source code for the script that generates the nftables rules from the VyOS configuration might give you some hints.

 

[nexox]

Update: Converting my reasonably-simple iptables script to nftables went smoothly, adding a flow table for software offload was easy enough, but I'm stuck trying to get hardware offload working - every time I try to add flags offload to the flow table definition I get Error: Could not process rule: Operation not supported.

There are a small number of threads out there where people have hit this same issue, none really have a conclusion - at first I thought it was because I was using the in-tree driver, so I built Mellanox's module, and the result is still the same, even after I set hw-tc-offload: on.

Thanks to the VyOS sources I did find the spot in the nftables source where this error comes from, but it's based on a return code from a function called via a nest of pointers and I haven't had time to dive in and figure out what, specifically, is causing the operation to fail. Maybe it doesn't like something with my bridge and VLAN config.