Enabling Intel QuickAssist for IPSec VPN on C2750

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Does anyone have experience using QAT with IPSec? I'm running VyOS (Debian based) on a SuperMicro 5018A-TN4. Getting nowhere finding drivers to enable the QuickAssist functions in the processor. I've managed to (finally) get close to 1gbit throughput on the VPN by loading the pcrypt module, but CPU usage is very high, and I would really like to get QAT working to reduce it.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
pcrypt support for AES-NI should work, but on first glance very sparsely documented. It may be worth trying another IPSec library like LibreSwan.
with AES-NI enabled your CPU usage should be fairly small or so Intel says.
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Yeah, no QAT on my cpu. I saw it referenced in the SuperMicro board manual and thought I had it. It's acceptable now with pcrypt/aes-ni, getting 800mbit+ unidirectional, which is close to wire speed once you consider overhead. Bit it still maxes out at 900 or so combined bidirectional. Switching to another distribution isn't really an option - it'd take a week just to redo the configuration, and I have 2 other sites running VyOS. The remote end is running on 4 cores of a Xeon E5-2665 v1, and doesn't even break a sweat without pcrypt. Is the Atom that much slower than the several generation older Xeon? I'll probably just live with it for now, don't have bidirectional transfers going that often, and bring that box over here once I get it's replacement built (48 core Xeon Scalable).

Code:
top - 21:55:17 up 23:13,  3 users,  load average: 4.14, 2.27, 1.26
Tasks: 181 total,   5 running,  99 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.1 us, 17.4 sy,  0.0 ni, 67.7 id,  0.0 wa,  0.0 hi, 14.7 si,  0.0 st
KiB Mem:  32869980 total,   744408 used, 32125572 free,   121516 buffers
KiB Swap:        0 total,        0 used,        0 free.   289312 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 6251 root      20   0       0      0      0 I  38.5  0.0   0:43.72 kworker/1:0-pde
    9 root      20   0       0      0      0 S  31.9  0.0   1:00.21 ksoftirqd/0
13534 root      20   0       0      0      0 R  30.2  0.0   0:23.76 kworker/4:3-pde
 7097 root      20   0       0      0      0 I  21.2  0.0   0:46.25 kworker/0:1-pen
10582 root      20   0       0      0      0 R  17.9  0.0   0:32.27 kworker/5:2-pde
 7107 root      20   0       0      0      0 I  13.9  0.0   0:50.87 kworker/0:2-eve
13406 root      20   0       0      0      0 R  13.9  0.0   0:20.71 kworker/0:3-pde
12761 vyos      20   0    6516   1768   1636 S  13.6  0.0   0:25.42 iperf3
   32 root      20   0       0      0      0 S  12.0  0.0   1:09.43 ksoftirqd/4
12939 root      20   0       0      0      0 I  10.0  0.0   0:12.29 kworker/5:4-pen
13764 root      20   0       0      0      0 R  10.0  0.0   0:01.07 kworker/3:1-pen
12690 root      20   0       0      0      0 I   9.6  0.0   0:12.88 kworker/6:1-pde
11260 root      20   0       0      0      0 I   9.3  0.0   0:42.57 kworker/5:0-pde
11326 root      20   0       0      0      0 I   9.3  0.0   0:09.92 kworker/7:0-pde
10898 root      20   0       0      0      0 I   8.3  0.0   0:08.84 kworker/2:2-pen
13799 root      20   0       0      0      0 I   3.3  0.0   0:01.17 kworker/2:1-pde
 6736 root      20   0       0      0      0 I   3.0  0.0   0:36.42 kworker/4:0-eve
11829 root      20   0       0      0      0 I   3.0  0.0   1:08.34 kworker/4:1-pde
12686 root      20   0       0      0      0 I   2.3  0.0   0:07.71 kworker/7:2-eve
12708 root      20   0       0      0      0 I   1.7  0.0   0:09.67 kworker/3:3-pde
12874 root      20   0       0      0      0 I   1.7  0.0   0:06.26 kworker/6:2-eve
13549 vyos      20   0    6516   1704   1596 S   1.3  0.0   0:01.40 iperf3
 6192 root      20   0       0      0      0 I   1.0  0.0   0:31.94 kworker/6:3-pde
 6911 root      20   0       0      0      0 I   1.0  0.0   0:39.66 kworker/3:0-eve
 7112 root      20   0       0      0      0 I   1.0  0.0   0:42.54 kworker/1:3-pen
12859 root      20   0       0      0      0 I   1.0  0.0   0:31.17 kworker/1:2-pde
 7136 vyos      20   0   14968   2972   2356 S   0.7  0.0   0:13.62 watch
12705 root      20   0       0      0      0 I   0.7  0.0   0:08.14 kworker/2:3-pen
12889 root      20   0       0      0      0 I   0.7  0.0   0:08.10 kworker/7:3-pde
   10 root      20   0       0      0      0 I   0.3  0.0   0:32.22 rcu_sched
13719 root      20   0   23664   3004   2456 R   0.3  0.0   0:00.15 top
    1 root      20   0  176392   5404   3256 S   0.0  0.0   0:05.20 systemd
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.04 kthreadd
    3 root       0 -20       0      0      0 I   0.0  0.0   0:00.00 rcu_gp
    4 root       0 -20       0      0      0 I   0.0  0.0   0:00.00 rcu_par_gp
    6 root       0 -20       0      0      0 I   0.0  0.0   0:00.00 kworker/0:0H-kb
    8 root       0 -20       0      0      0 I   0.0  0.0   0:00.00 mm_percpu_wq
   11 root      20   0       0      0      0 I   0.0  0.0   0:00.00 rcu_bh
Code:
cat /proc/crypto |more
name         : seqiv(rfc4106(gcm(aes)))
driver       : seqiv(pcrypt(rfc4106-gcm-aesni))
module       : seqiv
priority     : 500
refcnt       : 7
selftest     : passed
internal     : no
type         : aead
async        : yes
blocksize    : 1
ivsize       : 8
maxauthsize  : 16
geniv        : <none>

name         : rfc4106(gcm(aes))
driver       : pcrypt(rfc4106-gcm-aesni)
module       : pcrypt
priority     : 500
refcnt       : 7
selftest     : passed
internal     : no
type         : aead
async        : yes
blocksize    : 1
ivsize       : 8
maxauthsize  : 16
geniv        : <none>

name         : authenc(hmac(sha256),cbc(aes))
driver       : pcrypt(authenc(hmac(sha256-generic),cbc-aes-aesni))
module       : pcrypt
priority     : 4200
refcnt       : 1
selftest     : passed
internal     : no
type         : aead
async        : yes
blocksize    : 16
ivsize       : 16
maxauthsize  : 32
geniv        : <none>