Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfBlockerNG not blocking everything in blocklists

    Scheduled Pinned Locked Moved pfBlockerNG
    21 Posts 3 Posters 4.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • GertjanG
      Gertjan @vjizzle
      last edited by

      @vjizzle :

      C:\Users\gwkro>nslookup www.ku6.com
Serveur :   pfsense.brit-hotel-fumel.net
Address:  2001:470:1f13:5c0:2::1

Nom :    www.ku6.com
Address:  0.0.0.0

      @vjizzle said in pfBlockerNG not blocking everything in blocklists:

      It is not that I am willingly visiting BS sites ^^.

      I know ^^.

      Btw : the feed "oisd.nl" works fine for me.
      One of your settings is doing what you see.
      As, if we use the same code (pfBlockerNG) and the same feed (oisd.nl) only our local setting differ.

      These are my "Firewall > pfBlockerNG > DNSBL" settings :

      879efb74-c7ae-479e-a869-4c0309533252-image.png

      You already saw "Firewall > pfBlockerNG > DNSBL > DNSBL Groups".

      Here are the settings for the feed :

      fdb73a8e-ee32-42f0-9192-f97b23222adc-image.png

      When you force update, all looks well ?

      192.168.100.1 == pfSense (== resolver), right ?

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      V 1 Reply Last reply Reply Quote 0
      • V
        vjizzle @Gertjan
        last edited by vjizzle

        @gertjan
        Thank you again for your help! Thanks for sharing your settings for DNSBL, mine are same without the IPV6 options and I am not using the TOP1M lists.

        Yes 192.168.100.1 is my pfSense with Unbound running in resolver mode. I think I found something.

        When I load the lists in pfBlockerNG I can see that it is not blocking the domain www.ku6.com, www.conduit.com or www.vmn.net. Those are domains which are being triggered when using GRC's DNS testing tool. BUT I got pfSense working briefly as I wanted when I restarted Unbound manually after a force reload of pfBlockerNG! But then after cron of pfBlockerNG runs I and I check those domains again...pfBlockerNG is allowing them again.

        So I think that there is something wrong with my unbound - pfBlockerNG config.

        See:

        Before Ubound restart:

        www.conduit.com
        Server: odin.vXXXXXXX.lan
        Address: 192.168.100.1

        Non-authoritative answer:
        Name: www.va.conduit.com
        Address: 198.49.162.233
        Aliases: www.conduit.com

        www.vmn.net
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Non-authoritative answer:
        Name: www.vmn.net
        Address: 54.146.142.3

        And then I do the restart of Unbound:

        www.conduit.com
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Name: www.conduit.com
        Address: 0.0.0.0

        www.vmn.net
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Name: www.vmn.net
        Address: 0.0.0.0

        www.ku6.com
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Name: www.ku6.com
        Address: 0.0.0.0

        After a reload of pfBlockerNG:

        www.ku6.com
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Non-authoritative answer:
        Name: 3a1d905b3d90ef4e-cl2.qcloudcjgj.com
        Address: 118.89.206.86
        Aliases: www.ku6.com
        fa2a686c758fa3d110164df993a0b8c4.qcloudzygj.com

        www.vmn.net
        Server: odin.vXXXXXX.lan
        Address: 192.168.100.1

        Non-authoritative answer:
        Name: www.vmn.net
        Address: 54.146.142.3

        www.conduit.com
        Server: odin.vXXXXXXX.lan
        Address: 192.168.100.1

        Non-authoritative answer:
        Name: www.va.conduit.com
        Address: 198.49.162.233
        Aliases: www.conduit.com

        It looks like something is broken on my end of pfSense and pfBlockerNG but can't imagine what. Maybe it is time to try a clean install. Thank you for troubleshooting with me :)

        GertjanG 1 Reply Last reply Reply Quote 0
        • GertjanG
          Gertjan @vjizzle
          last edited by

          You can do some more checking.

          Locate the file /var/unbound/pfb_py_data.txt - load it into, for example, notepad++.
          (a SSH access and SFTP with WinSCP comes in handy here)

          rw-r--r--  1 root  unbound  63782900 Apr  5 10:29 /var/unbound/pfb_py_data.txt

          A 63 Megabytes file (and I'm just using a couple of feeds).

          When the oisd.nl feed file is loaded and parsed in, you'll find lines in this file like

          ....
,ku6.com,,0,OISD,DNSBL_Compilation
...
,www.ku6.com,,0,OISD,DNSBL_Compilation
....
etc.

....

          unbound, when it receive a DNS lookup request, first looks in the local cache. Part of the lookup is a call to the python script written for pfblockerNG. And this script uses /var/unbound/pfb_py_data.txt as it's local lookup table.
          If a host name is found in that file, it will be blocked.

          I tend to say that when "ku6.com" resolves, it isn't present in var/unbound/pfb_py_data.txt or : the feed didn't get loaded.

          No "help me" PM's please. Use the forum, the community will thank you.
          Edit : and where are the logs ??

          V 1 Reply Last reply Reply Quote 0
          • V
            vjizzle @Gertjan
            last edited by

            @gertjan

            I have checked the files and see the screenshots below:

            screen01.jpg

            Then I checked the entries for the domains in the file. They are there:

            screen02.jpg

            screen03.jpg

            And then when I check with nslookup pfBlockerNG is not blocking them:

            screen04.jpg

            I think something seriously is broken and I hope that it is only on my end. I am planning to do a clean install tomorrow and restore my backup and hopefully that will fix this.

            V 1 Reply Last reply Reply Quote 0
            • V
              vjizzle @vjizzle
              last edited by vjizzle

              @vjizzle

              I can confirm that reloading (force or normal) of pfBlockerNG does not solve this problem. However (like said before) when I restart unbound service it start working properly and pfBlockerNG blocks those domains.

              After restarting Unbound I go back to force reload pfblockerNG (to test) and it keeps working.

              After that I hit pfSense again with GRC DNS benchmark tool. This tool generates a LOT of DNS requests. After running the benchmark tool pfBlockerNG is broken and pfSense unbound resolves those blocked domains. So it look like when you hit pfSense Unbound with a lot of DNS requests something breaks under the load and pfBlockerNG is not working anymore with unbound. The pfb_py_data.txt is still there containing the entries from the blocked feeds. Force reloading pfBlockerNG does nog fix the problem at this point. I have to restart Unbound service manually again fixes the problem :|

              @Gertjan : can you test your side also with the GRC DNS Benchmark tool and see of pfsense with pfblocker hold up? I would appreciate it!

              GertjanG 1 Reply Last reply Reply Quote 0
              • GertjanG
                Gertjan @vjizzle
                last edited by

                @vjizzle

                Actually, there is nothing to restart.
                Check the unbound config :

                [2.5.0-RELEASE][root@mypfsense.net]/root: tail -n 3 /var/unbound/unbound.conf
# Python Module
python:
python-script: pfb_unbound.py

                unbound works - and uses the pfb_unbound.py script file to do some extra work.

                Btw :

                tail -f /var/unbound/var/log/pfblockerng/dns_reply.log

                shows the progress during DNS benchmark test. There are not that many DNS requests, a couple per seconds, and that is not much.

                I've got another test for you :

                cat /var/log/resolver.log | grep 'start'

                How often does your unbound restart ? per hour / per day ?

                No "help me" PM's please. Use the forum, the community will thank you.
                Edit : and where are the logs ??

                V 1 Reply Last reply Reply Quote 0
                • V
                  vjizzle @Gertjan
                  last edited by vjizzle

                  @gertjan

                  Here are the restarts from Unbound. I did some testing and restarting but nothing I think it looks good:

                  Apr  5 16:58:59 odin unbound[81926]: [81926:0] info: start of service (unbound 1.13.1).
Apr  5 17:15:14 odin unbound[70024]: [70024:0] info: start of service (unbound 1.13.1).
Apr  5 17:16:22 odin unbound[57071]: [57071:0] info: start of service (unbound 1.13.1).
Apr  6 11:38:11 odin unbound[11398]: [11398:0] info: start of service (unbound 1.13.1).
Apr  6 11:43:22 odin unbound[51771]: [51771:0] info: start of service (unbound 1.13.1).
Apr  6 11:57:55 odin unbound[9894]: [9894:0] info: start of service (unbound 1.13.1).
Apr  6 11:58:37 odin unbound[78660]: [78660:0] info: start of service (unbound 1.13.1).
Apr  6 12:01:13 odin unbound[7024]: [7024:0] info: start of service (unbound 1.13.1).
Apr  6 12:15:23 odin unbound[78621]: [78621:0] info: start of service (unbound 1.13.1).
Apr  6 12:23:14 odin unbound[93856]: [93856:0] info: start of service (unbound 1.13.1).

                  pfBlockerNG is not blocking those domains now and when I tail the unbound.conf I see this:

                  [2.5.0-RELEASE][admin@odin.vXXXXXX.lan]/root: tail -n 3 /var/unbound/unbound.conf
# Python Module
python:
python-script: pfb_unbound.py
[2.5.0-RELEASE][admin@odin.vXXXXXX.lan]/root:

                  In the dns_reply.log I can see all the queries when using the GRC tool. Looks to me like a lot. I restart Unbound service to make sure that pfBlockerNG is blocking those domains. Here is the output when everything is working fine and the domain www.conduit.com is being blocked:

                  DNS-reply,Apr 6 12:46:02,reply,A,SOA,3600,www.conduit.com.vXXXXX.lan,192.168.100.25,SOA,unk
DNS-reply,Apr 6 12:46:02,reply,AAAA,SOA,3600,www.conduit.com.vXXXXX.lan,192.168.100.25,SOA,unk
DNS-reply,Apr 6 12:46:02,servfail,AAAA,AAAA,Unk,www.conduit.com,192.168.100.25,ServFail,unk

                  Then I run GRC tool and see a lot of requests going by. Now when I query www.conduit.com I get an IP in nslookup and I see the following output and pfBlockerNG is broken:

                  DNS-reply,Apr 6 12:48:40,cache,A,A,3442,www.conduit.com.vXXXXXX.lan,192.168.100.25,NXDOMAIN,unk
DNS-reply,Apr 6 12:48:40,reply,AAAA,SOA,3442,www.conduit.com.vXXXXXX.lan,192.168.100.25,SOA,unk
DNS-reply,Apr 6 12:48:40,cache,A,A,86385,www.conduit.com,192.168.100.25,198.49.162.233,IL
DNS-reply,Apr 6 12:48:40,servfail,AAAA,AAAA,Unk,www.conduit.com,192.168.100.25,ServFail,unk
                  GertjanG 1 Reply Last reply Reply Quote 0
                  • GertjanG
                    Gertjan @vjizzle
                    last edited by

                    I see what you mean :

                    @vjizzle said in pfBlockerNG not blocking everything in blocklists:

                    DNS-reply,Apr 6 12:48:40,cache,A,A,86385,www.conduit.com,192.168.100.25,198.49.162.233,IL

                    dono where and when unbound (?) resolves "www.conduit.com".

                    Check these files :
                    dnsbl_error.log (mine is empty)
                    py_error.log (mine is empty)
                    dnsbl_parsed_error.log (some data, not related to this issue)

                    No "help me" PM's please. Use the forum, the community will thank you.
                    Edit : and where are the logs ??

                    V 1 Reply Last reply Reply Quote 0
                    • V
                      vjizzle @Gertjan
                      last edited by vjizzle

                      @gertjan
                      Yes for some reason every time I run the GRC lookup tool and query my pfSense, pfblockerng is broken afterwards. I can reproduce it every time and when I restart unbound server it is fixed.

                      I am using unbound on pfSense in resolver mode (forwarding option is disabled) and I have no DNS server entries in General Setup tab in pfSense. I have set the option DNS Resolution Behavior to Use local DNS (ignore remote DNS servers).

                      py_error.log is empty.
                      dnsbl_error.log shows a few entries

                      [ DNSBL_Abuse_CH - Abuse_ch_hostfile ] Download Fail
 Firewall and/or IDS (Legacy mode only) are not blocking download.

 Restoring previously downloaded file


[ DNSBL_Abuse_CH - Abuse_ch_hostfile ] Download Fail
 Firewall and/or IDS (Legacy mode only) are not blocking download.

 Restoring previously downloaded file

                      dnsbl_parsed_error.log has some entries like these:

                      04/6/21 14:16:12,No_Tracking_Hostnames,from,:: from
04/6/21 14:16:12,No_Tracking_Hostnames,hostwinds,0.0.0.0 hostwinds
04/6/21 14:16:12,No_Tracking_Hostnames,hostwinds,:: hostwinds
04/6/21 14:16:30,MS_2,beritanowtk,beritanowtk
                      GertjanG 1 Reply Last reply Reply Quote 0
                      • GertjanG
                        Gertjan @vjizzle
                        last edited by

                        Hummm.

                        Everything looks fine to me / identical to me.

                        Except : I can run that GRC tool as hard as I can, it won't break 'unbound'.

                        Did you try the old method "unbound mode" ?

                        No "help me" PM's please. Use the forum, the community will thank you.
                        Edit : and where are the logs ??

                        V 1 Reply Last reply Reply Quote 0
                        • V
                          vjizzle @Gertjan
                          last edited by

                          @gertjan
                          Good tip @Gertjan. I immediately tested it and low-and-behold. Everything keeps working even if I bombard pfsense with the GRC benchmarking tool. Here the output before and after the GRC testing:

                          > server 192.168.100.1
Default Server:  odin.vXXXX.lan
Address:  192.168.100.1

> www.conduit.com
Server:  odin.vXXXXX.lan
Address:  192.168.100.1

Name:    www.conduit.com
Address:  10.10.10.1

> www.vmn.net
Server:  odin.vXXXXX.lan
Address:  192.168.100.1

Name:    www.vmn.net
Address:  10.10.10.1

> www.ku6.com
Server:  odin.vXXXXX.lan
Address:  192.168.100.1

Name:    www.ku6.com
Address:  10.10.10.1

>

                          Looks like there is something the matter with the python integration here. Also as an added bonus in Unbound mode I can see that pfSense is now faster in answering queries compared to the AdGuard home server I have running here. In python mode pfSense was always slower:

                          screen05.jpg

                          Not that the speed difference is that big but still ^^. So in the end something is broken here. I will initiate a reinstall tomorrow unless someone else has another good idea. Thank you very. much for helping Gertjan!

                          1 Reply Last reply Reply Quote 0
                          • V
                            vjizzle
                            last edited by

                            Ok I just did a clean install of pfsense and then restored a backup. I have the exact same behavior again. Python mode pfblockerng stops working after I hammer it with GRC DNS benchmark.

                            For now I am switching over to AdGuard Home. Maybe better luck with pfSense in the next release :(

                            GertjanG 1 Reply Last reply Reply Quote 0
                            • GertjanG
                              Gertjan @vjizzle
                              last edited by Gertjan

                              I'm at home now.

                              Did some testing.
                              The dns benchmark running, doing it thing.

                              In a console/SSH, I ran this :

                              tail -f /var/unbound/var/log/pfblockerng/dns_reply.log | grep 'conduit'

                              According the https://dbl.oisd.nl/ feed ( the oisd.nl feed) :

                              ,www.conduit.com,,0,OISD,DNSBL_Compilation

                              and also :

                              ,conduit.com,,0,MS_2,DNSBL_BBcan177

                              I was spamming

                              www.conduit.com

                              in a nslookup (Windows 10) session.

                              Suddenly, during the test :

                              ......
> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

> www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1

Nom :    www.conduit.com
Address:  0.0.0.0

                              When the 0.0.0.0 came back, I saw this in the 'tail' console :

                              DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk
DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk

                              and then :

                              > www.conduit.com.
Serveur :   priv.my-domain.net
Address:  2001:470:xxxx:65c::1
Réponse ne faisant pas autorité :
Nom :    www.va.conduit.com
Address:  198.49.162.233
Aliases:  www.conduit.com

                              DNS-reply,Apr 6 23:59:12,cache,A,A,86399,www.conduit.com,2001:470:xxxx:65c::2001,198.49.162.233,IL

                              198.49.162.233 popped up.

                              I guess I can confirm what you saw.

                              Btw :

                              I saw :

                              Assembling DNSBL database...... completed [ 04/7/21 00:05:47 ]
TLD:
TLD analysis..xxxxxxxxxxx completed [ 04/7/21 00:05:51 ]

  ** TLD Domain count exceeded. [ 200000 ] All subsequent Domains listed as-is **

                              during the force update. The OISD feed is BIG : 1,2 million domain names.

                              I'll test tommorow agagin without the OISD feed, as the base domain name itself "conduit.com" is already in the BBCan 77 feed - as shown above.

                              edit : without the oisd.nl feed, and thus a rather (for me) small set of blocked domains, env 16962, the "0.0.0.0" blocking of "conduit.com" works.
                              www.conduit.com is resolved.....

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              1 Reply Last reply Reply Quote 0
                              • V
                                vjizzle
                                last edited by vjizzle

                                Interesting! So maybe I am on to something :p.

                                I got curious and did the test with the BBCan77 feed. The domain conduit.com is indeed there. So I decided to disable the oisd.nl feed and force reload pfblockerNG. After that I bombarded pfSense with the DNS benchmarking tool. Guess what...everything keeps running fine for conduit.com. Multiple reloads of pfBlockerNG, multiple restart of Unbound but everything kept working as expected:

                                > conduit.com
Server:  odin.vXXXXX.lan
Address:  192.168.100.1

Name:    conduit.com
Address:  0.0.0.0

                                Then I enabled the https://dbl.oisd.nl feed did a force reload of pfBlockerNG. I don't have TLD blocking enabled so I don't get that message you have. You can see the log about loading the oisd.nl list:

                                [ dbl_list ]			 Downloading update .. 200 OK.
  ----------------------------------------------------------------------
  Orig.    Unique     # Dups     # White    # TOP1M    Final                
  ----------------------------------------------------------------------
  1251871  1251871    154015     0          0          1097856              
  ----------------------------------------------------------------------

Saving DNSBL statistics... completed [ 04/7/21 00:31:48 ]
------------------------------------------------------------------------
Assembling DNSBL database...... completed [ 04/7/21 00:31:49 ]
Reloading Unbound Resolver (DNSBL python).
Stopping Unbound Resolver.
Unbound stopped in 2 sec.
Additional mounts (DNSBL python):
  No changes required.
Starting Unbound Resolver... completed [ 04/7/21 00:31:53 ]
Resolver cache restored
DNSBL update [ 1379690 | PASSED  ]... completed

                                And of course now pfBlockerNG is broken and not blocking the domain anymore BUT conduit.com keeps getting blocked by pfblockerng fine:

                                > server 192.168.100.1
Default Server:  odin.vXXXXXX.lan
Address:  192.168.100.1

> conduit.com
Server:  odin.vXXXXXX.lan
Address:  192.168.100.1

Name:    conduit.com
Address:  0.0.0.0

> www.conduit.com
Server:  odin.vXXXXXX.lan
Address:  192.168.100.1

Non-authoritative answer:
Name:    www.va.conduit.com
Address:  198.49.162.233
Aliases:  www.conduit.com

> www.ku6.com
Server:  odin.vXXXXXX.lan
Address:  192.168.100.1

Non-authoritative answer:
Name:    3a1d905b3d90ef4e-cl2.qcloudcjgj.com
Address:  118.89.206.86
Aliases:  www.ku6.com
          fa2a686c758fa3d110164df993a0b8c4.qcloudzygj.com

> www.vmn.net
Server:  odin.vXXXXXX.lan
Address:  192.168.100.1

Non-authoritative answer:
Name:    www.vmn.net
Address:  54.146.142.3

>

                                So www.conduit.com is broken but conduit.com is being blocked fine. Even after I run the DNS benchmark tool multiple times conduit.com keeps working.

                                I am glad you are seeing the same behavior like me. Looks like something is crashing when stressed with multiple (and fast) DNS queries .

                                GertjanG 1 Reply Last reply Reply Quote 0
                                • GertjanG
                                  Gertjan @vjizzle
                                  last edited by

                                  Same test, back on work.
                                  Home is a 1 Mbytes 'small' VM, work use its own i5, with 8 Mbyyes - although I don't think this matters.

                                  First test :
                                  nslookup with full debug :

                                  set db2

                                  > www.conduit.com.
Serveur :   pfsense.my-work-domain.net
Address:  2001:470:xxxx:5c0:2::1

------------
SendRequest(), len 33
    HEADER:
        opcode = QUERY, id = 20, rcode = NOERROR
        header flags:  query, want recursion
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = A, class = IN

------------
------------
Got answer (49 bytes):
    HEADER:
        opcode = QUERY, id = 20, rcode = NOERROR
        header flags:  response, auth. answer, want recursion, recursion avail.
        questions = 1,  answers = 1,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = A, class = IN
    ANSWERS:
    ->  www.conduit.com
        type = A, class = IN, dlen = 4
        internet address = 0.0.0.0
        ttl = 60 (1 min)

------------
------------
SendRequest(), len 33
    HEADER:
        opcode = QUERY, id = 21, rcode = NOERROR
        header flags:  query, want recursion
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = AAAA, class = IN

------------
------------
Got answer (33 bytes):
    HEADER:
        opcode = QUERY, id = 21, rcode = SERVFAIL
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = AAAA, class = IN

------------
Nom :    www.conduit.com
Address:  0.0.0.0

                                  Fine, "www.conduit.com" yields 0.0.0.0

                                  Now I launch a DNS benchmark, and have it finished (30 seconds later)

                                  I "nslookup" again :

                                  > www.conduit.com.
Serveur :   pfsense.my-work-domain.net
Address:  2001:470:xxxx:5c0:2::1

------------
SendRequest(), len 33
    HEADER:
        opcode = QUERY, id = 22, rcode = NOERROR
        header flags:  query, want recursion
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = A, class = IN

------------
------------
Got answer (70 bytes):
    HEADER:
        opcode = QUERY, id = 22, rcode = NOERROR
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 2,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = A, class = IN
    ANSWERS:
    ->  www.conduit.com
        type = CNAME, class = IN, dlen = 9
        canonical name = www.va.conduit.com
        ttl = 86227 (23 hours 57 mins 7 secs)
    ->  www.va.conduit.com
        type = A, class = IN, dlen = 4
        internet address = 198.49.162.233
        ttl = 86227 (23 hours 57 mins 7 secs)

------------
Réponse ne faisant pas autorité :
------------
SendRequest(), len 33
    HEADER:
        opcode = QUERY, id = 23, rcode = NOERROR
        header flags:  query, want recursion
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = AAAA, class = IN

------------
------------
Got answer (33 bytes):
    HEADER:
        opcode = QUERY, id = 23, rcode = SERVFAIL
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        www.conduit.com, type = AAAA, class = IN

------------
Nom :    www.va.conduit.com
Address:  198.49.162.233
Aliases:  www.conduit.com

                                  Note the difference :
                                  www.conduit.com resolves to the CNAME "www.va.conduit.com" (domain not listed), and that one resolves to "198.49.162.233"

                                  No "help me" PM's please. Use the forum, the community will thank you.
                                  Edit : and where are the logs ??

                                  1 Reply Last reply Reply Quote 0
                                  • V
                                    vjizzle
                                    last edited by vjizzle

                                    Hey Gertjan. This is correct about eventually resolving to a CNAME. I can see it on my end also that after running the GRC benchmark testing tool the domain resolves to a CNAME (www.va.conduit.com) and so pfblockerNG is not blocking it anymore. I think the same thing is happening with "www.ku6.com" and "www.vmn.net".

                                    Can we conclude that the CNAME blocking on pfBlockerNG is not working as expected?
                                    Because the same test on AdguardHome server always gets the domains blocked. I think that is also the reason why I am getting the low percentage in the pfblockerNG dashboard widget (AdguardHome is at 10% blocked DNS requests at the moment).

                                    I also think the same thing is happening with other domains as well. Just found out that "diagnostics.meethue.com" gets blocked but after benchmarking with GRC it is not blocked anymore.
                                    I have never seen "diagnostics.meethue.com" in the pfblockerNG DNSBL Block Stats tab being in the list. After running my whole network for 24 hrs on AdGuardhome it is the domain being blocked the most.

                                    BBcan177B 1 Reply Last reply Reply Quote 0
                                    • BBcan177B
                                      BBcan177 Moderator @vjizzle
                                      last edited by

                                      @vjizzle

                                      Thanks for the report! Its appreciated!

                                      The issue is that GRC is using a mixture of lowercase and uppercase in their DNS Tests.

                                      I have fixed the code to force everything to lowercase before the domains are validated.

                                      A fix can be downloaded here until the next version is available:

                                      curl -o /var/unbound/pfb_unbound.py "https://gist.githubusercontent.com/BBcan177/1e400f170b7ef7e8cb76b548d6524b31/raw"

                                      Then Restart Unbound

                                      Thanks!

                                      "Experience is something you don't get until just after you need it."

                                      Website: http://pfBlockerNG.com
                                      Twitter: @BBcan177  #pfBlockerNG
                                      Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                      V 1 Reply Last reply Reply Quote 3
                                      • V
                                        vjizzle @BBcan177
                                        last edited by

                                        @bbcan177

                                        Thank you for being so fast in fixing this and the great work you are doing with pfBlockerNG!

                                        With the modifications all is working well :).

                                        1 Reply Last reply Reply Quote 3
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.