pfSense 23.09 Intel QAT 4xxx passthrough question

Lurick

I'm wondering if I might be missing something on the pfsense side of things with this. I have a proxmox setup and I've tried passing through one of the Intel QAT modules from an Intel Xeon Gold 5416S but no matter what pfsense always shows QAT Crypto: No, even though I have gone under System > Advanced > Misc, and set Crypto Hardware to Intel QAT.

pciconf -lv | grep qat
qat0@pci0:3:0:0:        class=0x0b4000 rev=0x40 hdr=0x00 vendor=0x8086 device=0x4942 subvendor=0x8086 subdevice=0x0000

On the proxmox side of things:

cat /etc/modprobe.d/vfio-pci.conf
options vfio-pci disable_denylist=1

lsmod | grep vfio
vfio_pci               16384  6
vfio_pci_core          86016  1 vfio_pci
irqbypass              12288  223 vfio_pci_core,kvm
vfio_iommu_type1       49152  3
vfio                   57344  25 vfio_pci_core,kvmgt,vfio_iommu_type1,vfio_pci
iommufd                77824  1 vfio

lsmod | grep qat
qat_4xxx               20480  0
intel_qat             258048  1 qat_4xxx
crc8                   12288  1 intel_qat
authenc                12288  1 intel_qat

lspci -knn
f3:00.0 Co-processor [0b40]: Intel Corporation 4xxx Series QAT [8086:4942] (rev 40)
        Subsystem: Intel Corporation 4xxx Series QAT [8086:0000]
        Kernel driver in use: vfio-pci
        Kernel modules: qat_4xxx
f5:00.0 Co-processor [0b40]: Intel Corporation Device [8086:2710]
        Subsystem: Intel Corporation Device [8086:0000]
f7:00.0 Co-processor [0b40]: Intel Corporation 4xxx Series QAT [8086:4942] (rev 40)
        Subsystem: Intel Corporation 4xxx Series QAT [8086:0000]
        Kernel driver in use: vfio-pci
        Kernel modules: qat_4xxx
f9:00.0 Co-processor [0b40]: Intel Corporation Device [8086:2710]
        Subsystem: Intel Corporation Device [8086:0000]
       
cat /etc/modprobe.d/pve-blacklist.conf
# This file contains a list of modules which are not supported by Proxmox VE

# nvidiafb see bugreport https://bugzilla.proxmox.com/show_bug.cgi?id=701
blacklist nvidiafb
blacklist qat_4xxx

Lurick

I see on the docs it mentions intel C2000 and C3000 SoCs, does this mean that perhaps the Intel Xeon 4th gen CPUs with QAT aren't supported yet?
I am running pfsense plus in a VM, I've seen people passthrough the add-in cards but wondering about CPU based accelerators

https://docs.netgate.com/pfsense/en/latest/hardware/cryptographic-accelerators.html

NollipfSense

@Lurick Are you running a PCI QAT device? The method you used describes a PCI device...not a CPU with the ability to perform accelerating encryption. In my setup, I use Qat 8950 that's installed in a PCI slot.

Lurick

@NollipfSense Ok, I feel like an idiot, haha
I'm trying to use the CPU accelerators and assumed I just needed to pass them through somehow.
I guess what threw me off is that QAT says No even though the Intel 4000 series Xeon Scalable does have the support so I'm wondering why pfsense says No

NollipfSense

@Lurick Since you're using the CPU, no need to pass through anything...just select System > Advance . Misc and set crypto...and no, you're not an idiot.

stephenw10

The QAT device still appears as a separate PCIe device that would need to be passed to the guest.

NollipfSense

@stephenw10 said in pfSense 23.09 Intel QAT 4xxx passthrough question:

The QAT device still appears as a separate PCIe device that would need to be passed to the guest.

Even though in his or her case they're using the built-in QAT in the CPU? The OP seems to have done the required configuration for passing through a PCI device with no result.

stephenw10

Yes, it's still a PCIe device so it has to be passed through.

The pfSense GUI doesn't show it because it hasn't yet been updated to support 4xxx devices: https://redmine.pfsense.org/issues/15233

The kernel will still use it for crypto operations if it's loaded though.

NollipfSense

@stephenw10 Good to know...thanks.

Sergei_Shablovsky

@NollipfSense said in pfSense 23.09 Intel QAT 4xxx passthrough question:

@Lurick Are you running a PCI QAT device? The method you used describes a PCI device...not a CPU with the ability to perform accelerating encryption. In my setup, I use Qat 8950 that's installed in a PCI slot.

@NollipfSense Please, is Qat 8970 much (30-40%+) faster than Qat 8950 ?

Do You have some issues with stability? Overheating?

NollipfSense

@Sergei_Shablovsky said in pfSense 23.09 Intel QAT 4xxx passthrough question:

Please, is Qat 8970 much (30-40%+) faster than Qat 8950 ?

Do You have some issues with stability? Overheating?

I have never compared however, it should...I got mine from a Chinese seller on eBay as price was very reasonable...QAT 8970 is just too expensive. Mine has never showed any instabilities since installed. It does output heat...never overheated though.

I had hope that QAT was enabled in pfSense for any encrypted browsing but it's not the case...bummer...now that an expensive price on the table, it would be nice of pfSense to enable QAT for all encryption need...waiting.

Sergei_Shablovsky

@NollipfSense said in pfSense 23.09 Intel QAT 4xxx passthrough question:

@Sergei_Shablovsky said in pfSense 23.09 Intel QAT 4xxx passthrough question:

Please, is Qat 8970 much (30-40%+) faster than Qat 8950 ?

Do You have some issues with stability? Overheating?

I have never compared however, it should...I got mine from a Chinese seller on eBay as price was very reasonable...

Right now on USA eBay (where PP able to make chargeback, opposite to many DOA from Chinas sellers) QAT8950 cost USD$50-80

QAT 8970 is just too expensive.
Right now on USA eBay price are USD$150-220

Because that difference in price I’m asking about speed..:)

Mine has never showed any instabilities since installed. It does output heat...never overheated though.

Is it installed in rack server or just desktop tower?

I had hope that QAT was enabled in pfSense for any encrypted browsing but it's not the case...bummer...now that an expensive price on the table, it would be nice of pfSense to enable QAT for all encryption need...waiting.

But QAT are presented many years ago… I also hate LOOONG pfSense update cycles. ;)

Good luck with QAT using!

NollipfSense

@Sergei_Shablovsky said in pfSense 23.09 Intel QAT 4xxx passthrough question:

Right now on USA eBay (where PP able to make chargeback, opposite to many DOA from Chinas sellers) QAT8950 cost USD$50-80

I bought mine over a year ago from a Chinese seller for $37, in fact I bought two because at that price each, why not, while at the time, the QAT 8970 by US sellers priced at $800+ So, priced had fallen...that's good. Both mine are installed in work stations, a Lenovo and a Dell...see sig.