So I finally got some work done on setting up my root partitions using mdadm and lvm partitions instead of zfs and took the dive in rebuilding my proxmox server.
Sadly even with all the configuration I couldn't end up getting QAT to load enough before zfs for it to be used. When I initially only had the 2 disks inserted to the root partition the zfs module didn't get loaded on boot because there were no pools for the import service to pull in. If I started the modules via modprobe I was able to see the qat kstat.
Moving forward I added several more disks the the module was then loading on boot but now I no longer have the kstat object. I even went as far as adding the intel_qat and qat_api modules into /etc/modules.d and updating the initramfs to no avail. I'm running out nof ideas on how to get it to load the modules properly so that zfs can use the QAT card.
Any help would be appreciated.
Info:
root@proxmox2:~# lsmod | grep -i zfs
zfs 3457024 11
zunicode 331776 1 zfs
zavl 16384 1 zfs
icp 258048 1 zfs
zcommon 69632 1 zfs
znvpair 77824 2 zfs,zcommon
spl 106496 4 zfs,icp,znvpair,zcommon
qat_api 536576 1 zfs
root@proxmox2:~# lsmod | grep -i qat
qat_dh895xcc 20480 0
qat_api 536576 1 zfs
intel_qat 229376 3 qat_api,usdm_drv,qat_dh895xcc
authenc 16384 1 intel_qat
uio 20480 1 intel_qat
root@proxmox2:~# modinfo zfs | grep qat
depends: spl,qat_api,znvpair,zcommon,zunicode,zavl,icp
parm: zfs_qat_disable
isable QAT compression (int)
root@proxmox2:~# ls -al /proc/spl/kstat/zfs/
total 0
dr-xr-xr-x 3 root root 0 Feb 13 11:00 .
dr-xr-xr-x 4 root root 0 Feb 13 11:00 ..
-rw-r--r-- 1 root root 0 Feb 13 11:00 abdstats
-rw-r--r-- 1 root root 0 Feb 13 11:00 arcstats
-rw-r--r-- 1 root root 0 Feb 13 11:00 dbgmsg
-rw-r--r-- 1 root root 0 Feb 13 11:00 dbufs
-rw-r--r-- 1 root root 0 Feb 13 11:00 dmu_tx
-rw-r--r-- 1 root root 0 Feb 13 11:00 dnodestats
-rw-r--r-- 1 root root 0 Feb 13 11:00 fletcher_4_bench
-rw-r--r-- 1 root root 0 Feb 13 11:00 fm
dr-xr-xr-x 2 root root 0 Feb 13 11:00 tank
-rw-r--r-- 1 root root 0 Feb 13 11:00 vdev_cache_stats
-rw-r--r-- 1 root root 0 Feb 13 11:00 vdev_raidz_bench
-rw-r--r-- 1 root root 0 Feb 13 11:00 xuio_stats
-rw-r--r-- 1 root root 0 Feb 13 11:00 zfetchstats
-rw-r--r-- 1 root root 0 Feb 13 11:00 zil
root@proxmox2:~# service qat_service status
● qat_service.service - LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters
Loaded: loaded (/etc/init.d/qat_service; generated; vendor preset: enabled)
Active: active (exited) since Tue 2019-02-12 17:30:46 EST; 17h ago
Docs: man:systemd-sysv-generator(8)
Process: 27621 ExecStop=/etc/init.d/qat_service stop (code=exited, status=0/SUCCESS)
Process: 27674 ExecStart=/etc/init.d/qat_service start (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4915)
Memory: 0B
CPU: 0
CGroup: /system.slice/qat_service.service
Feb 12 17:30:44 proxmox2 systemd[1]: Starting LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters...
Feb 12 17:30:45 proxmox2 qat_service[27674]: Restarting all devices.
Feb 12 17:30:45 proxmox2 qat_service[27674]: Processing /etc/dh895xcc_dev0.conf
Feb 12 17:30:46 proxmox2 qat_service[27674]: Checking status of all devices.
Feb 12 17:30:46 proxmox2 qat_service[27674]: There is 1 QAT acceleration device(s) in the system:
Feb 12 17:30:46 proxmox2 qat_service[27674]: qat_dev0 - type: dh895xcc, inst_id: 0, node_id: 0, bsf: 0000:06:00.0, #accel: 6 #engines: 12 state: up
Feb 12 17:30:46 proxmox2 systemd[1]: Started LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters.
root@proxmox2:/proc# cat /sys/module/zfs/parameters/zfs_qat_disable
0