Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Intel Xeon D-2796NT and QAT

    Scheduled Pinned Locked Moved Development
    19 Posts 4 Posters 320 Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S Offline
      stephenw10 Netgate Administrator @DaddyGo
      last edited by

      @DaddyGo said in Intel Xeon D-2796NT and QAT:

      if this also applies to the D-2145NT and D-2187NT CPUs

      I'm not sure what QAT device is on those so I can't tell you. It seems quite likely though since they're from the same family of CPUs.

      @tinfoilmatt said in Intel Xeon D-2796NT and QAT:

      Got any insider 'predictions' you're free to share at this time?

      Nope, sorry. I have no insight there. 😁

      DaddyGoD 1 Reply Last reply Reply Quote 1
      • DaddyGoD Offline
        DaddyGo @stephenw10
        last edited by

        @stephenw10 said in Intel Xeon D-2796NT and QAT:

        I'm not sure what QAT device is on those so I can't tell you. It seems quite likely though since they're from the same family of CPUs.

        Intel C62x

        none1@pci0:4:8:0: class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
        vendor = 'Intel Corporation'
        device = 'C62x Chipset QuickAssist Technology Virtual Function'
        class = processor

        So should I install the given - 1761331340268-180.diff ?

        Cats bury it so they can't see it!
        (You know what I mean if you have a cat)

        1 Reply Last reply Reply Quote 0
        • stephenw10S Offline
          stephenw10 Netgate Administrator
          last edited by

          That device is already supported in 25.07.1. Check the IDs in the diff.

          DaddyGoD 1 Reply Last reply Reply Quote 0
          • DaddyGoD Offline
            DaddyGo @stephenw10
            last edited by DaddyGo

            @stephenw10 said in Intel Xeon D-2796NT and QAT:

            That device is already supported in 25.07.1. Check the IDs in the diff.

            I still can't get it to work, I'll show you :)-------
            (now I was able to focus on this more deeply, - this pfs box was completed.)

            -these are PCI QAT VFs /w pass-thr. from ESXi - no qat0

            none1@pci0:4:8:0: to none10@pci0:4:17:0:

            [25.07.1-RELEASE][root@ngfw.rm.arpa]/root: vmstat -i | grep qat


[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl -a | grep 'qat'
qat_ocf0: <QAT engine>
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%iommu:
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl dev.qat_ocf.0.enable
dev.qat_ocf.0.enable: 1

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: kldstat | egrep 'qat|ocf|crypto'
 9    1 0xffffffff8412b000     4b18 cryptodev.ko
14    1 0xffffffff84142000     4390 qat.ko
15    6 0xffffffff84147000    14dd0 qat_hw.ko
16    9 0xffffffff8415c000    2ffe0 qat_common.ko
17    8 0xffffffff8418c000    68cd8 qat_api.ko
18    1 0xffffffff847d7000    11258 qat_c2xxx.ko



[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: kldstat -v | grep qat
14    1 0xffffffff84142000     4390 qat.ko (/boot/kernel/qat.ko)
                698 nexus/qat
15    6 0xffffffff84147000    14dd0 qat_hw.ko (/boot/kernel/qat_hw.ko)
                697 pci/qat_c4xxx
                692 pci/qat_200xx
                696 pci/qat_dh895xcc
                693 pci/qat_4xxx
                695 pci/qat_c3xxx
                691 pci/qat_c62x
                694 pci/qat_4xxxvf
16    9 0xffffffff8415c000    2ffe0 qat_common.ko (/boot/kernel/qat_common.ko)
                689 qat_common
17    8 0xffffffff8418c000    68cd8 qat_api.ko (/boot/kernel/qat_api.ko)
                690 qat_api
18    1 0xffffffff847d7000    11258 qat_c2xxx.ko (/boot/kernel/qat_c2xxx.ko)
                699 pci/qat_c2xxx
				

[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: dmesg | grep -i qat
qat_ocf0: <QAT engine>


[25.07.1-RELEASE][root@ngfw.rm.arpa]/root: pciconf -lv



none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor



[root@vepesxi:~] vsish -e get /hardware/ipmiSupport
 1 -> IPMI supported
 
[root@vepesxi:~] lspci -vn | grep -E '37c9|18ef|18a1.*_VF_'
0000:b5:01.0 Class 0b40: 8086:37c9
0000:b5:01.1 Class 0b40: 8086:37c9
0000:b5:01.2 Class 0b40: 8086:37c9
0000:b5:01.3 Class 0b40: 8086:37c9
0000:b5:01.4 Class 0b40: 8086:37c9
0000:b5:01.5 Class 0b40: 8086:37c9
0000:b5:01.6 Class 0b40: 8086:37c9
0000:b5:01.7 Class 0b40: 8086:37c9
0000:b5:02.0 Class 0b40: 8086:37c9
0000:b5:02.1 Class 0b40: 8086:37c9
0000:b5:02.2 Class 0b40: 8086:37c9
0000:b5:02.3 Class 0b40: 8086:37c9
0000:b5:02.4 Class 0b40: 8086:37c9
0000:b5:02.5 Class 0b40: 8086:37c9
0000:b5:02.6 Class 0b40: 8086:37c9
0000:b5:02.7 Class 0b40: 8086:37c9


[root@vepesxi:~] lspci -vvv

0000:b5:00.0 Processor Co-processor: Intel Corporation c6xx QAT
         Class 0b40: 8086:37c8

0000:b5:01.0 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.1 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.2 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.3 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.4 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.5 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.6 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:01.7 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.0 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.1 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.2 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.3 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.4 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.5 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.6 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

0000:b5:02.7 Processor Co-processor: Intel Corporation c6xx QAT VF
         Class 0b40: 8086:37c9

            Cannot be (active), but is detected as a QAT device.

            a6d0a0b2-f4f6-4924-91bd-355251576a3d-image.png

            @stephenw10 - Any ideas?
            As far as I know, Netgate pushed for this development in FreeBSD.
            I can't really find anything about this on FreeBSD channels.

            PS:

            Broadcom:
            https://knowledge.broadcom.com/external/article/315470/intel-quickassist-technology-driver-for.html

            Cats bury it so they can't see it!
            (You know what I mean if you have a cat)

            1 Reply Last reply Reply Quote 0
            • stephenw10S Offline
              stephenw10 Netgate Administrator
              last edited by

              Hmm, no logs showing the driver trying to attach?

              It could be the driver itself doesn't support the virtual function for some reason. That would be unusual though. 🤔

              DaddyGoD 1 Reply Last reply Reply Quote 0
              • stephenw10S Offline
                stephenw10 Netgate Administrator
                last edited by

                Nope it is there: https://github.com/pfsense/FreeBSD-src/blob/devel-main/sys/dev/qat/include/common/adf_accel_devices.h#L29

                And it's in 25.07.1. So I'd expect to see it attach or attempt to attach.

                1 Reply Last reply Reply Quote 0
                • DaddyGoD Offline
                  DaddyGo @stephenw10
                  last edited by

                  @stephenw10 said in Intel Xeon D-2796NT and QAT:

                  Hmm, no logs showing the driver trying to attach?

                  I think I got it - I can see it in - dmesg.boot

                  pci4: <processor> at device 8.0 (no driver attached)
pci4: <processor> at device 9.0 (no driver attached)
pci4: <processor> at device 10.0 (no driver attached)
pci4: <processor> at device 11.0 (no driver attached)
pci4: <processor> at device 12.0 (no driver attached)
pci4: <processor> at device 13.0 (no driver attached)
pci4: <processor> at device 14.0 (no driver attached)
pci4: <processor> at device 15.0 (no driver attached)
pci4: <processor> at device 16.0 (no driver attached)
pci4: <processor> at device 17.0 (no driver attached)

                  & pciconf -lv --- pci4 --- 8 to 17 no driver attached

                  none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor

                  Dash claims to see QAT capabilities, but no driver is attached. Why?

                  Cats bury it so they can't see it!
                  (You know what I mean if you have a cat)

                  stephenw10S 1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator @DaddyGo
                    last edited by

                    @DaddyGo said in Intel Xeon D-2796NT and QAT:

                    Dash claims to see QAT capabilities, but no driver is attached. Why?

                    Because the webgui code looks for the device IDs and the C62x is in that list. For some reason the driver isn't attaching to the devices on your hardware, or even trying to attach, so whilst pfSense thinks you have a QAT device there isn't actually one present.

                    I've never tried passing through a VF QAT device like that and I suspect that could be causing it. I also don't have any hardware to test it with though.

                    Can you pass-through the real PCIe device?

                    Can you test bare meta?

                    DaddyGoD 1 Reply Last reply Reply Quote 0
                    • DaddyGoD Offline
                      DaddyGo @stephenw10
                      last edited by DaddyGo

                      @stephenw10 said in Intel Xeon D-2796NT and QAT:

                      Can you pass-through the real PCIe device?

                      Can you test bare meta?

                      A little background on the case, this is exactly what this device is:

                      https://i.dell.com/sites/csdocuments/Shared-Content_data-Sheets_Documents/en/virtual-edge-platform-4600-spec-sheet.pdf

                      We have 15 of these in our system, with VersaOS, but the license is extremely expensive and we want to switch to pfSense.

                      What I am currently testing with pfS Plus is equipped with an 8C16T Intel D-2145NT CPU.
                      QAT on these not real PCI devices; they are C62x implemented in the CPU as chipset device(s).

                      Since Dell also sells these with ESXi as base OS opc., we still have a license valid for four years, so I used that.

                      --and the other thing I would like to test before the mature version of "if_pppoe" is this - multi-threaded PPPoE in pfSense
                      https://www.neelc.org/posts/opnsense-pppoe-kvm/

                      Also have a 16C32T D-2187NT version "behind me", which I have already started with a light mfsBSD. So I think pfSense can also run on it as bare metal. I'll try it out soon.

                      This is how it looks in ESXi before you enable SR-IOV - only PCI bridges and the base Chipset device

                      EDIT1: This is a screenshot of the D-2187NT CPU where 3 "ch" QAT is implemented.

                      61f61210-0e09-4c5c-a7c8-cd4aebdcb7ac-image.png

                      After enabling SR-IOV, it creates 16 VFs for each Chipset device (C62x), and only these can be passed to the guest using "pass-thr".

                      9cce2c4d-f1fc-4ef8-94c5-07dd6ad805f4-image.png

                      What you see in the VM PCI list...

                      9401334c-d5fc-420e-944c-d6b1060bb2cd-image.png

                      pfSense also sees this correctly

                      none1@pci0:4:8:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none2@pci0:4:9:0:       class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none3@pci0:4:10:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none4@pci0:4:11:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none5@pci0:4:12:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none6@pci0:4:13:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none7@pci0:4:14:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none8@pci0:4:15:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none9@pci0:4:16:0:      class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
none10@pci0:4:17:0:     class=0x0b4000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x37c9 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'C62x Chipset QuickAssist Technology Virtual Function'
    class      = processor
                      • just doesn't load drivers for them for some reason (?)
                        Perhaps because they are VFs, but they are also listed as PCI devices, so they are present.

                      Cats bury it so they can't see it!
                      (You know what I mean if you have a cat)

                      stephenw10S 1 Reply Last reply Reply Quote 0
                      • stephenw10S Offline
                        stephenw10 Netgate Administrator @DaddyGo
                        last edited by

                        So before you enable SR-IOV you can't pass through the PCIe devices directly?

                        DaddyGoD 1 Reply Last reply Reply Quote 1
                        • DaddyGoD Offline
                          DaddyGo @stephenw10
                          last edited by DaddyGo

                          @stephenw10 said in Intel Xeon D-2796NT and QAT:

                          So before you enable SR-IOV you can't pass through the PCIe devices directly?

                          Yes, it was, thanks for the train of thought. :)
                          (control plane is needed)

                          Only the pass-thr. the Virtual Functions does not work.

                          • I updated Intel FreeBSD 1.x VIB to the latest version on ESXi.
                          • I revoked all pass-thr. and PCI assignments in ESXi and pfS VM -- & SR-IOV off in ESXi on C62X
                          • After updating VIB, I was able to enable only the pass-thr. of C62x chipset device, which I transferred to the VM as a complete PCI device.

                          Seeing the complete device, pfSense was able to assign a driver to the QAT C62X.

                          d4ea54e4-ac73-4d9a-8202-02502dc9c22c-image.png

                          [25.07.1-RELEASE][root@ngfw.rm.arpa]/root: sysctl -a | grep 'qat'
qat0: <Intel c6xx QuickAssist> mem 0xffb40000-0xffb7ffff,0xffb00000-0xffb3ffff at device 8.0 numa-domain 0 on pci4
qat0: qat_dev0 started 8 acceleration engines
qat0: FW version: 4.18.0
qat0: Excessive clock measure delay
qat_ocf0: <QAT engine>
irq135: qat0:b0:277 @cpu0(domain0): 0
irq136: qat0:b1:279 @cpu0(domain0): 0
irq137: qat0:b2:281 @cpu0(domain0): 0
irq138: qat0:b3:283 @cpu0(domain0): 0
irq139: qat0:b4:285 @cpu0(domain0): 0
irq140: qat0:b5:287 @cpu0(domain0): 0
irq141: qat0:b6:289 @cpu0(domain0): 0
irq142: qat0:b7:291 @cpu0(domain0): 0
irq143: qat0:b8:293 @cpu0(domain0): 0
irq144: qat0:b9:295 @cpu0(domain0): 0
irq145: qat0:b10:297 @cpu0(domain0): 0
irq146: qat0:b11:299 @cpu0(domain0): 0
irq147: qat0:b12:301 @cpu0(domain0): 0
irq148: qat0:b13:303 @cpu0(domain0): 0
irq149: qat0:b14:305 @cpu0(domain0): 0
irq150: qat0:b15:307 @cpu0(domain0): 0
irq151: qat0:ae:309 @cpu0(domain0): 0
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%iommu:
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:
dev.qat.0.frequency: 685000000
dev.qat.0.cnv_error:
dev.qat.0.fw_counters:
dev.qat.0.mmp_version: 6.0.0
dev.qat.0.hw_version: 4
dev.qat.0.fw_version: 4.18.0
dev.qat.0.heartbeat: 1
dev.qat.0.heartbeat_failed: 0
dev.qat.0.heartbeat_sent: 1
dev.qat.0.dev_cfg: [GENERAL]
dev.qat.0.num_user_processes: 0
dev.qat.0.cfg_mode: ks
dev.qat.0.cfg_services: sym;dc
dev.qat.0.state: up
dev.qat.0.%domain: 0
dev.qat.0.%iommu: rid=0x440
dev.qat.0.%parent: pci4
dev.qat.0.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0000 class=0x0b4000
dev.qat.0.%location: slot=8 function=0 dbsf=pci0:4:8:0 handle=\_SB_.PC0G.S9F0
dev.qat.0.%driver: qat
dev.qat.0.%desc: Intel c6xx QuickAssist
dev.qat.%parent:

                          D-2145NT CPU only 1 "ch" QAT implemented

                          18cdec9a-fa83-484f-acd9-5e7207d29906-image.png

                          The original goal was not to assign all VFs to pfSense, but with this CPU, it is necessary because there is only one "ch" QAT.

                          For example, D-2187NT CPU has 3 channels (3x16VFs), where other VMs can also get a QAT device if needed.

                          The lesson is that cannot use SR-IOV first, but must need pass-thr. the entire device to the pfS guest.

                          On Linux, for example handles this and explicitly recommends that in case multiple QAT "CHs", - VFs should be symmetrically distributed to the guest(s).

                          D-2187NT offers greater flexibility if you want to run more than just pfSense guests on ESXi, as QAT capability remains available for other VMs.

                          I should note that I am STILL a fan of bare metal pfSense, but there are cases where paravirtualization can help. This is now the case.

                          And honestly, after so many years away, it was good to return to pfSense. It never disappoints. I did skip the PLUS licensing thing (but the TAC was fair), but Netgate is right: counterfeiters MUST be stopped somehow!

                          @stephenw10 - THX 😉

                          Cats bury it so they can't see it!
                          (You know what I mean if you have a cat)

                          1 Reply Last reply Reply Quote 1
                          • stephenw10S Offline
                            stephenw10 Netgate Administrator
                            last edited by

                            Aha, nice!

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.