Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL

    Scheduled Pinned Locked Moved pfBlockerNG
    18 Posts 7 Posters 1.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      iTestAndroid
      last edited by

      When I load all the ad servers lists I found into DNSBL, it shows total entries as "DNSBL:1425095", so roughly 1.4M

      But I have 128GB RAM and i7 CPU dedicated solely to my pfSense.

      So, when I try to resolve a new domain (not cached), it takes a long time. Sometimes my request times out in Chrome and I have to hit refresh and then it works.

      What can I do to improve performance of my DNSBL? Is there any settings to load entire list in memory or do faster regex or other types of scans to check if domain is in list or not? Any optimizations?

      BBcan177B NollipfSenseN 2 Replies Last reply Reply Quote 0
      • NollipfSenseN
        NollipfSense
        last edited by

        https://docs.netgate.com/pfsense/en/latest/dns/blocking-dns-queries-to-external-resolvers.html
        OR
        https://docs.netgate.com/pfsense/en/latest/dns/redirecting-all-dns-requests-to-pfsense.html

        pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
        pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

        1 Reply Last reply Reply Quote 0
        • I
          iTestAndroid
          last edited by

          Yeah... I don't think you read my question. Links are irrelevant to what I asked/said

          1 Reply Last reply Reply Quote 0
          • BBcan177B
            BBcan177 Moderator @iTestAndroid
            last edited by

            @iTestAndroid
            Is it just chrome? or all browsers? In pfSense Resolver, increase the "Log Verbosity" to "2" and see if you can get some more debugging information to help diagnose...

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            1 Reply Last reply Reply Quote 0
            • NollipfSenseN
              NollipfSense @iTestAndroid
              last edited by

              @iTestAndroid said in i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL:

              So, when I try to resolve a new domain (not cached), it takes a long time.

              That's why I gave you the links.

              pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
              pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

              1 Reply Last reply Reply Quote 0
              • I
                iTestAndroid
                last edited by iTestAndroid

                @BBcan177
                It happens on all devices, I just gave Chrome as and example. It happens in all devices and I had logging on level 2 for a while, didn't see anything special/interesting in there. Should I increase level or look for something in particular in the logs?

                I put log level to 2 again, let me see what I find

                1 Reply Last reply Reply Quote 0
                • I
                  iTestAndroid
                  last edited by

                  @BBcan177

                  Oct 10 21:35:00 	unbound 	29825:1 	info: Verified that unsigned response is INSECURE
Oct 10 21:35:00 	unbound 	29825:1 	info: NSEC3s for the referral proved no DS.
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was nodata ANSWER
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for amazonaws.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving amazonaws.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: Verified that unsigned response is INSECURE
Oct 10 21:35:00 	unbound 	29825:1 	info: NSEC3s for the referral proved no DS.
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving yahoo.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was CNAME
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.1.1.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for c2s-resources-global.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving c2s-resources-global.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for beacons-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:1 	info: response for dicts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving beacons-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:1 	info: resolving dicts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:4 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving google.com. DS IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:4 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:4 	info: response for clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:4 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:4 	info: response for scripts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving scripts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for oath.cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving oath.cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: validated DNSKEY cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: validated DS cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yahoo.com. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for psicdn.net. DS IN
Oct 10 21:34:59 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:1 	info: response for edge.gycpi.b.yahoodns.net. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving psicdn.net. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for oath.com. DS IN
Oct 10 21:34:59 	unbound 	29825:1 	info: resolving edge.gycpi.b.yahoodns.net. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for yimg.com. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving oath.com. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yimg.com. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving s.yimg.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for google.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving google.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was NXDOMAIN ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was NXDOMAIN ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: generate keytag query _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for msh.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: generate keytag query _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for api.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 		[29825:a] error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:6 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:5 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 		[29825:b] error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:0 	info: start of service (unbound 1.9.1).
Oct 10 21:34:58 	unbound 	29825:0 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:2 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:1 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:3 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:9 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:4 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving api.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving answers.yahoo.com. A IN

                  This is what I found in logs (level 2), answers.yahoo.com took very long time to resolve. I don't think it took long time for answer.yahoo.com in the DNS resolver, I think the pfBlockerNG test took long time.

                  BBcan177B 1 Reply Last reply Reply Quote 0
                  • BBcan177B
                    BBcan177 Moderator @iTestAndroid
                    last edited by

                    @iTestAndroid
                    Try without DNS over TLS. Seems to be the Resolver issue and not pfBlockerNG.

                    "Experience is something you don't get until just after you need it."

                    Website: http://pfBlockerNG.com
                    Twitter: @BBcan177  #pfBlockerNG
                    Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                    I 1 Reply Last reply Reply Quote 0
                    • I
                      iTestAndroid
                      last edited by iTestAndroid

                      @BBcan177
                      For testing, sure, I'm turning it off now and will try again. But generally speaking, I have to keep the DNS over TLS. Will test and let you know soon.

                      A 1 Reply Last reply Reply Quote 0
                      • A
                        akuma1x @iTestAndroid
                        last edited by

                        @iTestAndroid Why do you "have to keep" DNS over TLS? Don't trust your ISP with your DNS traffic?

                        Just wondering...

                        Jeff

                        I 1 Reply Last reply Reply Quote 0
                        • I
                          iTestAndroid @akuma1x
                          last edited by

                          @akuma1x
                          Yep, I'd rather hand all my DNS data to CloudFlare than to my ISP.

                          1 Reply Last reply Reply Quote 0
                          • NollipfSenseN
                            NollipfSense
                            last edited by

                            This is off topic; however, curious about how much of your 128GB RAM do you use? I got 16GB and using 17% of it.

                            pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
                            pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

                            1 Reply Last reply Reply Quote 0
                            • I
                              iTestAndroid @BBcan177
                              last edited by iTestAndroid

                              @BBcan177
                              I tried without DNS over TLS, used 8.8.8.8 and 4.4.4.4 DNS servers and had same results. It goes smoothly most of the time, but every here and there it gets stuck

                              @NollipfSense Same, pfSense barely uses 16GB. I also enable/disable ntopng here and there to analyze and monitor activities in network. I just turned an old PC I had into firewall for my home net, so didn't really take out anything.

                              1 Reply Last reply Reply Quote 0
                              • T
                                tman222
                                last edited by

                                Hi @iTestAndroid - I have a couple clarifying questions:

                                1. Is DNS performance acceptable if you temporarily disable pfBlockerNG?
                                2. Can you confirm your DNS settings for us? What do you have checked for DNS Query Forwarding under the DNS Resolver Settings? Do you have DNSSEC checked or unchecked?
                                3. What DNS Servers listed in the Dashboard System Information widget?
                                4. If you ping Google's DNS or Cloudflare's DNS servers, is the packet RTT acceptable?

                                Thanks in advance.

                                1 Reply Last reply Reply Quote 0
                                • provelsP
                                  provels
                                  last edited by provels

                                  Are you using ramdisks for /var and /tmp?
                                  System/Advanced/Miscellaneous

                                  Peder

                                  MAIN - pfSense+ 24.11-RELEASE - Adlink MXE-5401, i7, 16 GB RAM, 64 GB SSD. 500 GB HDD for SyslogNG
                                  BACKUP - pfSense+ 23.01-RELEASE - Hyper-V Virtual Machine, Gen 1, 2 v-CPUs, 3 GB RAM, 8GB VHDX (Dynamic)

                                  1 Reply Last reply Reply Quote 0
                                  • I
                                    iTestAndroid
                                    last edited by iTestAndroid

                                    @tman222

                                    1. Yes. Actually, with pfBlocker and list of around 400-500k, it works flawlessly, its super fast and blocks most ads. When I turn on all lists I have, the total count adds up to ~1.4M and that's when things start to go south. Otherwise both DNS resolver and pfBlocker works fine. So when 1.4M entry is added to pfBlocker, things get slow and some DNS requests hangs.

                                    2. DNS Query Forwarding -> Enabled
                                      Use SSL/TLS for outgoing DNS queries -> Enabled
                                      Custom Options:
                                      server:
                                      forward-zone:
                                      name: "."
                                      forward-ssl-upstream: yes
                                      forward-addr: 1.1.1.1@853
                                      forward-addr: 1.0.0.1@853
                                      forward-addr: 9.9.9.9@853

                                    server:include: /var/unbound/pfb_dnsbl.*conf

                                    1. These are the DNS server addresses listed there
                                      1.1.1.1
                                      1.0.0.1
                                      9.9.9.9

                                    2. I have gigabit internet, RTT is acceptable:
                                      ping cloudflare.net
                                      PING cloudflare.net (104.16.208.90) 56(84) bytes of data.
                                      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=1 ttl=57 time=2.73 ms
                                      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=2 ttl=57 time=2.60 ms
                                      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=3 ttl=57 time=2.43 ms
                                      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=4 ttl=57 time=2.37 ms
                                      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=5 ttl=57 time=2.53 ms

                                    @provels
                                    Yes, I have both enabled and each of them have size of 4096MB (4GB)

                                    T 1 Reply Last reply Reply Quote 0
                                    • T
                                      tman222 @iTestAndroid
                                      last edited by

                                      @iTestAndroid said in i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL:

                                      @tman222

                                      1. Yes. Actually, with pfBlocker and list of around 400-500k, it works flawlessly, its super fast and blocks most ads. When I turn on all lists I have, the total count adds up to ~1.4M and that's when things start to go south. Otherwise both DNS resolver and pfBlocker works fine. So when 1.4M entry is added to pfBlocker, things get slow and some DNS requests hangs.

                                      2. DNS Query Forwarding -> Enabled
                                        Use SSL/TLS for outgoing DNS queries -> Enabled
                                        Custom Options:
                                        server:
                                        forward-zone:
                                        name: "."
                                        forward-ssl-upstream: yes
                                        forward-addr: 1.1.1.1@853
                                        forward-addr: 1.0.0.1@853
                                        forward-addr: 9.9.9.9@853

                                      server:include: /var/unbound/pfb_dnsbl.*conf

                                      1. These are the DNS server addresses listed there
                                        1.1.1.1
                                        1.0.0.1
                                        9.9.9.9

                                      2. I have gigabit internet, RTT is acceptable:
                                        ping cloudflare.net
                                        PING cloudflare.net (104.16.208.90) 56(84) bytes of data.
                                        64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=1 ttl=57 time=2.73 ms
                                        64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=2 ttl=57 time=2.60 ms
                                        64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=3 ttl=57 time=2.43 ms
                                        64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=4 ttl=57 time=2.37 ms
                                        64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=5 ttl=57 time=2.53 ms

                                      @provels
                                      Yes, I have both enabled and each of them have size of 4096MB (4GB)

                                      Hi @iTestAndroid - do you see any difference if you take out 9.9.9.9 and just use Cloudflare's 1.1.1.1 and 1.0.0.1 servers? Do you have DNSSEC checked or unchecked? I'm still not quite convinced this is a pfBlockerNG issue -- 1.4M is really not that big and you have got some pretty powerful hardware too.

                                      Hope this helps.

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        durianbusuk
                                        last edited by

                                        This post is deleted!
                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.