@stephenw10 said in Intel Xeon D-2796NT and QAT:
So before you enable SR-IOV you can't pass through the PCIe devices directly?
Yes, it was, thanks for the train of thought. :)
(control plane is needed)
Only the pass-thr. the Virtual Functions does not work.
I updated Intel FreeBSD 1.x VIB to the latest version on ESXi.
I revoked all pass-thr. and PCI assignments in ESXi and pfS VM -- & SR-IOV off in ESXi on C62X
After updating VIB, I was able to enable only the pass-thr. of C62x chipset device, which I transferred to the VM as a complete PCI device.
Seeing the complete device, pfSense was able to assign a driver to the QAT C62X.
[image: 1762511051996-d4ea54e4-ac73-4d9a-8202-02502dc9c22c-image.png]
[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl -a | grep 'qat'
qat0: <Intel c6xx QuickAssist> mem 0xffb40000-0xffb7ffff,0xffb00000-0xffb3ffff at device 8.0 numa-domain 0 on pci4
qat0: qat_dev0 started 8 acceleration engines
qat0: FW version: 4.18.0
qat0: Excessive clock measure delay
qat_ocf0: <QAT engine>
irq135: qat0:b0:277 @cpu0(domain0): 0
irq136: qat0:b1:279 @cpu0(domain0): 0
irq137: qat0:b2:281 @cpu0(domain0): 0
irq138: qat0:b3:283 @cpu0(domain0): 0
irq139: qat0:b4:285 @cpu0(domain0): 0
irq140: qat0:b5:287 @cpu0(domain0): 0
irq141: qat0:b6:289 @cpu0(domain0): 0
irq142: qat0:b7:291 @cpu0(domain0): 0
irq143: qat0:b8:293 @cpu0(domain0): 0
irq144: qat0:b9:295 @cpu0(domain0): 0
irq145: qat0:b10:297 @cpu0(domain0): 0
irq146: qat0:b11:299 @cpu0(domain0): 0
irq147: qat0:b12:301 @cpu0(domain0): 0
irq148: qat0:b13:303 @cpu0(domain0): 0
irq149: qat0:b14:305 @cpu0(domain0): 0
irq150: qat0:b15:307 @cpu0(domain0): 0
irq151: qat0:ae:309 @cpu0(domain0): 0
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%iommu:
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:
dev.qat.0.frequency: 685000000
dev.qat.0.cnv_error:
dev.qat.0.fw_counters:
dev.qat.0.mmp_version: 6.0.0
dev.qat.0.hw_version: 4
dev.qat.0.fw_version: 4.18.0
dev.qat.0.heartbeat: 1
dev.qat.0.heartbeat_failed: 0
dev.qat.0.heartbeat_sent: 1
dev.qat.0.dev_cfg: [GENERAL]
dev.qat.0.num_user_processes: 0
dev.qat.0.cfg_mode: ks
dev.qat.0.cfg_services: sym;dc
dev.qat.0.state: up
dev.qat.0.%domain: 0
dev.qat.0.%iommu: rid=0x440
dev.qat.0.%parent: pci4
dev.qat.0.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0000 class=0x0b4000
dev.qat.0.%location: slot=8 function=0 dbsf=pci0:4:8:0 handle=\_SB_.PC0G.S9F0
dev.qat.0.%driver: qat
dev.qat.0.%desc: Intel c6xx QuickAssist
dev.qat.%parent:
D-2145NT CPU only 1 "ch" QAT implemented
[image: 1762512285003-18cdec9a-fa83-484f-acd9-5e7207d29906-image.png]
The original goal was not to assign all VFs to pfSense, but with this CPU, it is necessary because there is only one "ch" QAT.
For example, D-2187NT CPU has 3 channels (3x16VFs), where other VMs can also get a QAT device if needed.
The lesson is that cannot use SR-IOV first, but must need pass-thr. the entire device to the pfS guest.
On Linux, for example handles this and explicitly recommends that in case multiple QAT "CHs", - VFs should be symmetrically distributed to the guest(s).
D-2187NT offers greater flexibility if you want to run more than just pfSense guests on ESXi, as QAT capability remains available for other VMs.
I should note that I am STILL a fan of bare metal pfSense, but there are cases where paravirtualization can help. This is now the case.
And honestly, after so many years away, it was good to return to pfSense. It never disappoints. I did skip the PLUS licensing thing (but the TAC was fair), but Netgate is right: counterfeiters MUST be stopped somehow!
@stephenw10 - THX