Intel Xeon D-2796NT and QAT

stephenw10

Mmm, the driver itself is loaded and has attached to the hardware. It should have registered the supported ciphers with the cryptodev framework.

The device lists in pfSense just need to be updated to include it. Similarly to this: https://redmine.pfsense.org/issues/14844

Let me see....

stephenw10

Ok try the attached patch if you can:

180.diff

I don't actually have a device with that QAT hardware so I can't test it unfortunately. But it's pretty simple.

MarinSNB

@stephenw10

It works! Thank you so much!

Screenshot 2025-10-24 at 2.53.13 PM.png

So will this patch make it into the next Plus release then?

Thanks again!

stephenw10

It may be too late for 25.11 but it's very simple so we'll see.....

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

It may be too late for 25.11 but it's very simple so we'll see.....

@stephenw10 -- Steve, since I'm already here on the forum with another question, I'll ask if this also applies to the D-2145NT and D-2187NT CPUs?

QAT is assigned to these CPUs with ESXi.

Intel (R) C62x Chipset with
Intel QuickAssist Technology (Intel QAT) Driver for VMware* for Customer Enabling (CE) Release

tinfoilmatt

@stephenw10 Any plans to open source this firmware/driver so CE can take advantage?! Got any insider 'predictions' you're free to share at this time???

stephenw10

@DaddyGo said in Intel Xeon D-2796NT and QAT:

if this also applies to the D-2145NT and D-2187NT CPUs

I'm not sure what QAT device is on those so I can't tell you. It seems quite likely though since they're from the same family of CPUs.

@tinfoilmatt said in Intel Xeon D-2796NT and QAT:

Got any insider 'predictions' you're free to share at this time?

Nope, sorry. I have no insight there.

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

I'm not sure what QAT device is on those so I can't tell you. It seems quite likely though since they're from the same family of CPUs.

Intel C62x

none1@pci0:4:8:0: class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
vendor = 'Intel Corporation'
device = 'C62x Chipset QuickAssist Technology Virtual Function'
class = processor

So should I install the given - 1761331340268-180.diff ?

stephenw10

That device is already supported in 25.07.1. Check the IDs in the diff.

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

That device is already supported in 25.07.1. Check the IDs in the diff.

I still can't get it to work, I'll show you :)-------
(now I was able to focus on this more deeply, - this pfs box was completed.)

-these are PCI QAT VFs /w pass-thr. from ESXi - no qat0

none1@pci0:4:8:0: to none10@pci0:4:17:0:

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: vmstat -i | grep qat


[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl -a | grep 'qat'
qat_ocf0: <QAT engine>
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%iommu:
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl dev.qat_ocf.0.enable
dev.qat_ocf.0.enable: 1

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: kldstat | egrep 'qat|ocf|crypto'
 9    1 0xffffffff8412b000     4b18 cryptodev.ko
14    1 0xffffffff84142000     4390 qat.ko
15    6 0xffffffff84147000    14dd0 qat_hw.ko
16    9 0xffffffff8415c000    2ffe0 qat_common.ko
17    8 0xffffffff8418c000    68cd8 qat_api.ko
18    1 0xffffffff847d7000    11258 qat_c2xxx.ko



[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: kldstat -v | grep qat
14    1 0xffffffff84142000     4390 qat.ko (/boot/kernel/qat.ko)
                698 nexus/qat
15    6 0xffffffff84147000    14dd0 qat_hw.ko (/boot/kernel/qat_hw.ko)
                697 pci/qat_c4xxx
                692 pci/qat_200xx
                696 pci/qat_dh895xcc
                693 pci/qat_4xxx
                695 pci/qat_c3xxx
                691 pci/qat_c62x
                694 pci/qat_4xxxvf
16    9 0xffffffff8415c000    2ffe0 qat_common.ko (/boot/kernel/qat_common.ko)
                689 qat_common
17    8 0xffffffff8418c000    68cd8 qat_api.ko (/boot/kernel/qat_api.ko)
                690 qat_api
18    1 0xffffffff847d7000    11258 qat_c2xxx.ko (/boot/kernel/qat_c2xxx.ko)
                699 pci/qat_c2xxx
				

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: dmesg | grep -i qat
qat_ocf0: <QAT engine>


[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: pciconf -lv



none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor



[root@vepesxi:~] vsish -e get /hardware/ipmiSupport
 1 -> IPMI supported
 
[root@vepesxi:~] lspci -vn | grep -E '37c9|18ef|18a1.*_VF_'
0000:b5:01.0 Class 0b40: 8086:37c9
0000:b5:01.1 Class 0b40: 8086:37c9
0000:b5:01.2 Class 0b40: 8086:37c9
0000:b5:01.3 Class 0b40: 8086:37c9
0000:b5:01.4 Class 0b40: 8086:37c9
0000:b5:01.5 Class 0b40: 8086:37c9
0000:b5:01.6 Class 0b40: 8086:37c9
0000:b5:01.7 Class 0b40: 8086:37c9
0000:b5:02.0 Class 0b40: 8086:37c9
0000:b5:02.1 Class 0b40: 8086:37c9
0000:b5:02.2 Class 0b40: 8086:37c9
0000:b5:02.3 Class 0b40: 8086:37c9
0000:b5:02.4 Class 0b40: 8086:37c9
0000:b5:02.5 Class 0b40: 8086:37c9
0000:b5:02.6 Class 0b40: 8086:37c9
0000:b5:02.7 Class 0b40: 8086:37c9


[root@vepesxi:~] lspci -vvv

0000:b5:00.0 Processor Co-processor: Intel Corporation c6xx QAT
         Class 0b40: 8086:37c8

0000:b5:01.0 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.1 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.2 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.3 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.4 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.5 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.6 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.7 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.0 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.1 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.2 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.3 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.4 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.5 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.6 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.7 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

Cannot be (active), but is detected as a QAT device.

@stephenw10 - Any ideas?
As far as I know, Netgate pushed for this development in FreeBSD.
I can't really find anything about this on FreeBSD channels.

PS:

Broadcom:
https://knowledge.broadcom.com/external/article/315470/intel-quickassist-technology-driver-for.html

stephenw10

Hmm, no logs showing the driver trying to attach?

It could be the driver itself doesn't support the virtual function for some reason. That would be unusual though.

stephenw10

Nope it is there: https://github.com/pfsense/FreeBSD-src/blob/devel-main/sys/dev/qat/include/common/adf_accel_devices.h#L29

And it's in 25.07.1. So I'd expect to see it attach or attempt to attach.

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

Hmm, no logs showing the driver trying to attach?

I think I got it - I can see it in - dmesg.boot

pci4: <processor> at device 8.0 (no driver attached)
pci4: <processor> at device 9.0 (no driver attached)
pci4: <processor> at device 10.0 (no driver attached)
pci4: <processor> at device 11.0 (no driver attached)
pci4: <processor> at device 12.0 (no driver attached)
pci4: <processor> at device 13.0 (no driver attached)
pci4: <processor> at device 14.0 (no driver attached)
pci4: <processor> at device 15.0 (no driver attached)
pci4: <processor> at device 16.0 (no driver attached)
pci4: <processor> at device 17.0 (no driver attached)

& pciconf -lv --- pci4 --- 8 to 17 no driver attached

none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor

Dash claims to see QAT capabilities, but no driver is attached. Why?

stephenw10

@DaddyGo said in Intel Xeon D-2796NT and QAT:

Dash claims to see QAT capabilities, but no driver is attached. Why?

Because the webgui code looks for the device IDs and the C62x is in that list. For some reason the driver isn't attaching to the devices on your hardware, or even trying to attach, so whilst pfSense thinks you have a QAT device there isn't actually one present.

I've never tried passing through a VF QAT device like that and I suspect that could be causing it. I also don't have any hardware to test it with though.

Can you pass-through the real PCIe device?

Can you test bare meta?

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

Can you pass-through the real PCIe device?

Can you test bare meta?

A little background on the case, this is exactly what this device is:

https://i.dell.com/sites/csdocuments/Shared-Content_data-Sheets_Documents/en/virtual-edge-platform-4600-spec-sheet.pdf

We have 15 of these in our system, with VersaOS, but the license is extremely expensive and we want to switch to pfSense.

What I am currently testing with pfS Plus is equipped with an 8C16T Intel D-2145NT CPU.
QAT on these not real PCI devices; they are C62x implemented in the CPU as chipset device(s).

Since Dell also sells these with ESXi as base OS opc., we still have a license valid for four years, so I used that.

--and the other thing I would like to test before the mature version of "if_pppoe" is this - multi-threaded PPPoE in pfSense
https://www.neelc.org/posts/opnsense-pppoe-kvm/

Also have a 16C32T D-2187NT version "behind me", which I have already started with a light mfsBSD. So I think pfSense can also run on it as bare metal. I'll try it out soon.

This is how it looks in ESXi before you enable SR-IOV - only PCI bridges and the base Chipset device

EDIT1: This is a screenshot of the D-2187NT CPU where 3 "ch" QAT is implemented.

After enabling SR-IOV, it creates 16 VFs for each Chipset device (C62x), and only these can be passed to the guest using "pass-thr".

What you see in the VM PCI list...

pfSense also sees this correctly

none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor

just doesn't load drivers for them for some reason (?)
Perhaps because they are VFs, but they are also listed as PCI devices, so they are present.

stephenw10

So before you enable SR-IOV you can't pass through the PCIe devices directly?

DaddyGo

@stephenw10 said in Intel Xeon D-2796NT and QAT:

So before you enable SR-IOV you can't pass through the PCIe devices directly?

Yes, it was, thanks for the train of thought. :)
(control plane is needed)

Only the pass-thr. the Virtual Functions does not work.

I updated Intel FreeBSD 1.x VIB to the latest version on ESXi.
I revoked all pass-thr. and PCI assignments in ESXi and pfS VM -- & SR-IOV off in ESXi on C62X
After updating VIB, I was able to enable only the pass-thr. of C62x chipset device, which I transferred to the VM as a complete PCI device.

Seeing the complete device, pfSense was able to assign a driver to the QAT C62X.

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl -a | grep 'qat'
qat0: <Intel c6xx QuickAssist> mem 0xffb40000-0xffb7ffff,0xffb00000-0xffb3ffff at device 8.0 numa-domain 0 on pci4
qat0: qat_dev0 started 8 acceleration engines
qat0: FW version: 4.18.0
qat0: Excessive clock measure delay
qat_ocf0: <QAT engine>
irq135: qat0:b0:277 @cpu0(domain0): 0
irq136: qat0:b1:279 @cpu0(domain0): 0
irq137: qat0:b2:281 @cpu0(domain0): 0
irq138: qat0:b3:283 @cpu0(domain0): 0
irq139: qat0:b4:285 @cpu0(domain0): 0
irq140: qat0:b5:287 @cpu0(domain0): 0
irq141: qat0:b6:289 @cpu0(domain0): 0
irq142: qat0:b7:291 @cpu0(domain0): 0
irq143: qat0:b8:293 @cpu0(domain0): 0
irq144: qat0:b9:295 @cpu0(domain0): 0
irq145: qat0:b10:297 @cpu0(domain0): 0
irq146: qat0:b11:299 @cpu0(domain0): 0
irq147: qat0:b12:301 @cpu0(domain0): 0
irq148: qat0:b13:303 @cpu0(domain0): 0
irq149: qat0:b14:305 @cpu0(domain0): 0
irq150: qat0:b15:307 @cpu0(domain0): 0
irq151: qat0:ae:309 @cpu0(domain0): 0
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%iommu:
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:
dev.qat.0.frequency: 685000000
dev.qat.0.cnv_error:
dev.qat.0.fw_counters:
dev.qat.0.mmp_version: 6.0.0
dev.qat.0.hw_version: 4
dev.qat.0.fw_version: 4.18.0
dev.qat.0.heartbeat: 1
dev.qat.0.heartbeat_failed: 0
dev.qat.0.heartbeat_sent: 1
dev.qat.0.dev_cfg: [GENERAL]
dev.qat.0.num_user_processes: 0
dev.qat.0.cfg_mode: ks
dev.qat.0.cfg_services: sym;dc
dev.qat.0.state: up
dev.qat.0.%domain: 0
dev.qat.0.%iommu: rid=0x440
dev.qat.0.%parent: pci4
dev.qat.0.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0000 class=0x0b4000
dev.qat.0.%location: slot=8 function=0 dbsf=pci0:4:8:0 handle=\_SB_.PC0G.S9F0
dev.qat.0.%driver: qat
dev.qat.0.%desc: Intel c6xx QuickAssist
dev.qat.%parent:

D-2145NT CPU only 1 "ch" QAT implemented

The original goal was not to assign all VFs to pfSense, but with this CPU, it is necessary because there is only one "ch" QAT.

For example, D-2187NT CPU has 3 channels (3x16VFs), where other VMs can also get a QAT device if needed.

The lesson is that cannot use SR-IOV first, but must need pass-thr. the entire device to the pfS guest.

On Linux, for example handles this and explicitly recommends that in case multiple QAT "CHs", - VFs should be symmetrically distributed to the guest(s).

D-2187NT offers greater flexibility if you want to run more than just pfSense guests on ESXi, as QAT capability remains available for other VMs.

I should note that I am STILL a fan of bare metal pfSense, but there are cases where paravirtualization can help. This is now the case.

And honestly, after so many years away, it was good to return to pfSense. It never disappoints. I did skip the PLUS licensing thing (but the TAC was fair), but Netgate is right: counterfeiters MUST be stopped somehow!

@stephenw10 - THX

stephenw10

Aha, nice!