4. i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL

i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL

  • I

    When I load all the ad servers lists I found into DNSBL, it shows total entries as "DNSBL:1425095", so roughly 1.4M

    But I have 128GB RAM and i7 CPU dedicated solely to my pfSense.

    So, when I try to resolve a new domain (not cached), it takes a long time. Sometimes my request times out in Chrome and I have to hit refresh and then it works.

    What can I do to improve performance of my DNSBL? Is there any settings to load entire list in memory or do faster regex or other types of scans to check if domain is in list or not? Any optimizations?

    0 2 Replies

  • https://docs.netgate.com/pfsense/en/latest/dns/blocking-dns-queries-to-external-resolvers.html
    OR
    https://docs.netgate.com/pfsense/en/latest/dns/redirecting-all-dns-requests-to-pfsense.html

    0
  • I

    Yeah... I don't think you read my question. Links are irrelevant to what I asked/said

    0
  • Moderator

    @iTestAndroid
    Is it just chrome? or all browsers? In pfSense Resolver, increase the "Log Verbosity" to "2" and see if you can get some more debugging information to help diagnose...

    0

  • @iTestAndroid said in i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL:

    So, when I try to resolve a new domain (not cached), it takes a long time.

    That's why I gave you the links.

    0
  • I

    @BBcan177
    It happens on all devices, I just gave Chrome as and example. It happens in all devices and I had logging on level 2 for a while, didn't see anything special/interesting in there. Should I increase level or look for something in particular in the logs?

    I put log level to 2 again, let me see what I find

    0
  • I

    @BBcan177

    Oct 10 21:35:00 	unbound 	29825:1 	info: Verified that unsigned response is INSECURE
Oct 10 21:35:00 	unbound 	29825:1 	info: NSEC3s for the referral proved no DS.
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was nodata ANSWER
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for amazonaws.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving amazonaws.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: Verified that unsigned response is INSECURE
Oct 10 21:35:00 	unbound 	29825:1 	info: NSEC3s for the referral proved no DS.
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving yahoo.com. DS IN
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: query response was CNAME
Oct 10 21:35:00 	unbound 	29825:1 	info: reply from <.> 1.1.1.1#853
Oct 10 21:35:00 	unbound 	29825:1 	info: response for us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:35:00 	unbound 	29825:1 	info: resolving us-east-1.onemobile.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for c2s-resources-global.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving c2s-resources-global.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for beacons-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:1 	info: response for dicts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving beacons-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:1 	info: resolving dicts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:4 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving google.com. DS IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:4 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:4 	info: response for clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving clients1.google.com. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:4 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:4 	info: response for scripts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:4 	info: resolving scripts-prodga.c2s-sa-production.aws.oath.cloud. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for oath.cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving oath.cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: validated DNSKEY cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving cloud. DNSKEY IN
Oct 10 21:34:59 	unbound 	29825:9 	info: validated DS cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving cloud. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yahoo.com. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving guce.yahoo.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for psicdn.net. DS IN
Oct 10 21:34:59 	unbound 	29825:1 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:1 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:1 	info: response for edge.gycpi.b.yahoodns.net. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving psicdn.net. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for oath.com. DS IN
Oct 10 21:34:59 	unbound 	29825:1 	info: resolving edge.gycpi.b.yahoodns.net. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:59 	unbound 	29825:9 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was nodata ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for yimg.com. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving oath.com. DS IN
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving yimg.com. DS IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was ANSWER
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:59 	unbound 	29825:8 	info: response for consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: query response was CNAME
Oct 10 21:34:59 	unbound 	29825:9 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:59 	unbound 	29825:9 	info: response for s.yimg.com. A IN
Oct 10 21:34:59 	unbound 	29825:8 	info: resolving consent.cmp.oath.com. A IN
Oct 10 21:34:59 	unbound 	29825:9 	info: resolving s.yimg.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving yahoodns.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:7 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for google.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving akamai.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Verified that unsigned response is INSECURE
Oct 10 21:34:58 	unbound 	29825:8 	info: NSEC3s for the referral proved no DS.
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was nodata ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving google.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving yahoo.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving amazon.com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DNSKEY com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving edgesuite.net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DNSKEY net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving com. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validated DS com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was NXDOMAIN ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving net. DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validated DS net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving com. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving com. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving net. DS IN
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving net. DS IN
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:7 	info: Successfully primed trust anchor . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: validate keys with anchor(DS): sec_status_secure
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was NXDOMAIN ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:7 	info: generate keytag query _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:7 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for msh.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving . DNSKEY IN
Oct 10 21:34:58 	unbound 	29825:8 	info: generate keytag query _ta-4f66. NULL IN
Oct 10 21:34:58 	unbound 	29825:8 	info: prime trust anchor
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was ANSWER
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for api.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.1.1.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:7 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:7 	info: response for clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: query response was CNAME
Oct 10 21:34:58 	unbound 	29825:8 	info: reply from <.> 1.0.0.1#853
Oct 10 21:34:58 	unbound 	29825:8 	info: response for answers.yahoo.com. A IN
Oct 10 21:34:58 	unbound 		[29825:a] error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:6 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:5 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 		[29825:b] error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:0 	info: start of service (unbound 1.9.1).
Oct 10 21:34:58 	unbound 	29825:0 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:2 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:1 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:3 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:9 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:4 	error: duplicate forward zone . ignored.
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving api.amazon.com. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving clients4.google.com. A IN
Oct 10 21:34:58 	unbound 	29825:8 	info: resolving www.msftncsi.com.edgesuite.net. A IN
Oct 10 21:34:58 	unbound 	29825:7 	info: resolving answers.yahoo.com. A IN

    This is what I found in logs (level 2), answers.yahoo.com took very long time to resolve. I don't think it took long time for answer.yahoo.com in the DNS resolver, I think the pfBlockerNG test took long time.

    0 1 Reply
  • Moderator

    @iTestAndroid
    Try without DNS over TLS. Seems to be the Resolver issue and not pfBlockerNG.

    0
    I
     1 Reply
  • I

    @BBcan177
    For testing, sure, I'm turning it off now and will try again. But generally speaking, I have to keep the DNS over TLS. Will test and let you know soon.

    0
    A
     1 Reply
  • A

    @iTestAndroid Why do you "have to keep" DNS over TLS? Don't trust your ISP with your DNS traffic?

    Just wondering...

    Jeff

    0
    I
     1 Reply
  • I

    @akuma1x
    Yep, I'd rather hand all my DNS data to CloudFlare than to my ISP.

    0

  • This is off topic; however, curious about how much of your 128GB RAM do you use? I got 16GB and using 17% of it.

    0
  • I

    @BBcan177
    I tried without DNS over TLS, used 8.8.8.8 and 4.4.4.4 DNS servers and had same results. It goes smoothly most of the time, but every here and there it gets stuck

    @NollipfSense Same, pfSense barely uses 16GB. I also enable/disable ntopng here and there to analyze and monitor activities in network. I just turned an old PC I had into firewall for my home net, so didn't really take out anything.

    0
  • T

    Hi @iTestAndroid - I have a couple clarifying questions:

    1. Is DNS performance acceptable if you temporarily disable pfBlockerNG?
    2. Can you confirm your DNS settings for us? What do you have checked for DNS Query Forwarding under the DNS Resolver Settings? Do you have DNSSEC checked or unchecked?
    3. What DNS Servers listed in the Dashboard System Information widget?
    4. If you ping Google's DNS or Cloudflare's DNS servers, is the packet RTT acceptable?

    Thanks in advance.

    0
  • P

    Are you using ramdisks for /var and /tmp?
    System/Advanced/Miscellaneous

    0
  • I

    @tman222

    1. Yes. Actually, with pfBlocker and list of around 400-500k, it works flawlessly, its super fast and blocks most ads. When I turn on all lists I have, the total count adds up to ~1.4M and that's when things start to go south. Otherwise both DNS resolver and pfBlocker works fine. So when 1.4M entry is added to pfBlocker, things get slow and some DNS requests hangs.

    2. DNS Query Forwarding -> Enabled
      Use SSL/TLS for outgoing DNS queries -> Enabled
      Custom Options:
      server:
      forward-zone:
      name: "."
      forward-ssl-upstream: yes
      forward-addr: 1.1.1.1@853
      forward-addr: 1.0.0.1@853
      forward-addr: 9.9.9.9@853

    server:include: /var/unbound/pfb_dnsbl.*conf

    1. These are the DNS server addresses listed there
      1.1.1.1
      1.0.0.1
      9.9.9.9

    2. I have gigabit internet, RTT is acceptable:
      ping cloudflare.net
      PING cloudflare.net (104.16.208.90) 56(84) bytes of data.
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=1 ttl=57 time=2.73 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=2 ttl=57 time=2.60 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=3 ttl=57 time=2.43 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=4 ttl=57 time=2.37 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=5 ttl=57 time=2.53 ms

    @provels
    Yes, I have both enabled and each of them have size of 4096MB (4GB)

    0
    T
     1 Reply
  • T

    @iTestAndroid said in i7 4GHz CPU + 128GB RAM, still slow when I load a large list in DNSBL:

    @tman222

    1. Yes. Actually, with pfBlocker and list of around 400-500k, it works flawlessly, its super fast and blocks most ads. When I turn on all lists I have, the total count adds up to ~1.4M and that's when things start to go south. Otherwise both DNS resolver and pfBlocker works fine. So when 1.4M entry is added to pfBlocker, things get slow and some DNS requests hangs.

    2. DNS Query Forwarding -> Enabled
      Use SSL/TLS for outgoing DNS queries -> Enabled
      Custom Options:
      server:
      forward-zone:
      name: "."
      forward-ssl-upstream: yes
      forward-addr: 1.1.1.1@853
      forward-addr: 1.0.0.1@853
      forward-addr: 9.9.9.9@853

    server:include: /var/unbound/pfb_dnsbl.*conf

    1. These are the DNS server addresses listed there
      1.1.1.1
      1.0.0.1
      9.9.9.9

    2. I have gigabit internet, RTT is acceptable:
      ping cloudflare.net
      PING cloudflare.net (104.16.208.90) 56(84) bytes of data.
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=1 ttl=57 time=2.73 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=2 ttl=57 time=2.60 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=3 ttl=57 time=2.43 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=4 ttl=57 time=2.37 ms
      64 bytes from 104.16.208.90 (104.16.208.90): icmp_seq=5 ttl=57 time=2.53 ms

    @provels
    Yes, I have both enabled and each of them have size of 4096MB (4GB)

    Hi @iTestAndroid - do you see any difference if you take out 9.9.9.9 and just use Cloudflare's 1.1.1.1 and 1.0.0.1 servers? Do you have DNSSEC checked or unchecked? I'm still not quite convinced this is a pfBlockerNG issue -- 1.4M is really not that big and you have got some pretty powerful hardware too.

    Hope this helps.

    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy