Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfBlockerNG not blocking domain after first DNS lookup attempt

    Scheduled Pinned Locked Moved pfBlockerNG
    34 Posts 3 Posters 3.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      vjizzle @jrey
      last edited by

      @jrey
      It is a virtual machine yes. Check the screenshot below showing the difference between using pihole vs pfblockerng.

      screenshot06.png

      1 Reply Last reply Reply Quote 0
      • J
        jrey @vjizzle
        last edited by

        @vjizzle

        Okay where are you making the entry to block msn.com ?

        You have a couple of items checked that I don't use/require however... I'll have to check and see if any of them make a difference

        johnpozJ 1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator @jrey
          last edited by

          @jrey if you set unbound on pfsense to block something, or via pfblocker... It will be blocked every single time..

          If your having a client get an answer, then it got it elsewhere - doh in your browser would be the most likely suspect.

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          V J 2 Replies Last reply Reply Quote 0
          • V
            vjizzle @johnpoz
            last edited by

            @johnpoz Yes I know about the DOH in browsers indeed, that is not what this problem is in my opinion.

            @jrey www.msn.com is blocked via CNAME a-0003.a-msedge.net. This record exists in the following blocklist: https://raw.githubusercontent.com/WindowsLies/BlockWindows/master/hostslist

            I just tried changing pfBlockerNG to Unboud mode and a force reload afterwards. Also cleared the dns cache of my client but still the same. pfSense/pfBlocker still resolves www.msn.com unfortunately.

            J 1 Reply Last reply Reply Quote 0
            • V
              vjizzle
              last edited by

              I just tested this out from my Macbook. It's still the same problem. www.msn.com is being resolved. It doesn't matter if pfBlockerNG is in Unbound mode or Python mode. After reloading pfBlockerNG, the first query for www.msn.com results in a 0.0.0.0 answer, but every query after the first one is resolved.

              XXXX@caldigit-dock ~ % nslookup
              > > server 192.168.100.1
              > Default server: 192.168.100.1
              > Address: 192.168.100.1#53
              > > www.msn.com
              > Server:		192.168.100.1
              > Address:	192.168.100.1#53
              > 
              > Non-authoritative answer:
              > Name:	www.msn.com
              > Address: 0.0.0.0
              > > www.msn.com
              > Server:		192.168.100.1
              > Address:	192.168.100.1#53
              > 
              > Non-authoritative answer:
              > www.msn.com	canonical name = www-msn-com.a-0003.a-msedge.net.
              > www-msn-com.a-0003.a-msedge.net	canonical name = a-0003.a-msedge.net.
              > Name:	a-0003.a-msedge.net
              > Address: 204.79.197.203
              > > 
              > > 
              > 
              
              1 Reply Last reply Reply Quote 0
              • J
                jrey @johnpoz
                last edited by

                @johnpoz said in pfBlockerNG not blocking domain after first DNS lookup attempt:

                doh in your browser would be the most likely suspect.

                unlikely doh, it is an nslookup - port 53

                it is a good question, I've preformed the same tests as the OP, and my DoH rule has never tripping, but at the same time it just blocks every single time.

                Let's ask this @vjizzle are you loading / have you loaded msn at any point during testing in a browser.

                as for the browser, on my windows machine
                Screen Shot 2023-11-30 at 10.09.23 AM.png

                and the nslookup all still returned 0.0.0.0 after. No DoH recorded.

                but something could be loading the cache,

                "Message cache elements are prefected..."
                "Serve cache records even with TTL of 0"

                Personally I've never had much use for either those settings, OP may have a requirement ?

                1 Reply Last reply Reply Quote 0
                • V
                  vjizzle
                  last edited by

                  I have not loaded www.msn.com (or the CNAME) in my browser or other device in my network. The DNS options for prefetch and server TTL of 0 is only there to speedup host lookups. The same options are enabled on my pihole which is using local installed unbound in resolver mode.

                  Somehow the pfSense resolver manages to bypass pfBlockerNG or CNAME blocking fails in pfBlockerNG, looks like to me at least.

                  johnpozJ 1 Reply Last reply Reply Quote 0
                  • johnpozJ
                    johnpoz LAYER 8 Global Moderator @vjizzle
                    last edited by johnpoz

                    @vjizzle said in pfBlockerNG not blocking domain after first DNS lookup attempt:

                    CNAME blocking fails in pfBlockerNG

                    you could have some problem with something looking up the cname but not the original that is blocked..

                    ;; QUESTION SECTION:
                    ;www.msn.com.                   IN      A
                    
                    ;; ANSWER SECTION:
                    www.msn.com.            11194   IN      CNAME   www-msn-com.a-0003.a-msedge.net.
                    www-msn-com.a-0003.a-msedge.net. 56 IN  CNAME   a-0003.a-msedge.net.
                    a-0003.a-msedge.net.    56      IN      A       204.79.197.203
                    

                    Blocking msn.com doesn't prevent lookup of a-msedge.net.

                    If I wanted to block msn.com what I would do is put this directly in unbound custom box, since I don't use blocking in pfblocker..

                    local-zone: "msn.com" always_nxdomain

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                    J 1 Reply Last reply Reply Quote 0
                    • J
                      jrey @vjizzle
                      last edited by

                      @vjizzle said in pfBlockerNG not blocking domain after first DNS lookup attempt:

                      www.msn.com is blocked via CNAME a-0003.a-msedge.net

                      look above when I ran the test the canonical name returned is not on the list you provided.

                      I'd like to see the specific setup you have for this list you are providing.
                      (ie the DNSBL setup containing that list)

                      Hopefully you have it added under an entry within DNSBL Groups?

                      also in the pfblockerng.log there should be a section showing it being processed.
                      provide that log section please.
                      anything in the error.log (pfblockerng)

                      then most likely what you are seeing is that www.msn.com is not on the list, and you are returning the CNAME on subsequent queries some of those are multi-named

                      not also that the CNAME I returned is www.msn.com.a-0003.a-msedge.net
                      and the CNAME you see is a-0003.a-msedge.net

                      traceroute a-0003.a-msedge.net
                      traceroute to a-0003.a-msedge.net (204.79.197.203), 64 hops max, 52 byte packets
                      
                      traceroute www.msn.com.a-0003.a-msedge.net
                      traceroute to a-0003.a-msedge.net (204.79.197.203), 64 hops max, 52 byte packets
                      
                      

                      on your DNSBL category, where you have the list being added, goto the bottom DNSBL Custom_list and add the entry msn.com
                      no quotes, no other sub just msn.com
                      Screen Shot 2023-11-30 at 10.37.00 AM.png

                      then force reload the DNSBL on the update tab
                      Screen Shot 2023-11-30 at 10.38.06 AM.png

                      check the same 2 logs referenced above, for the processing of the list and the new custom entry

                      try the test again.

                      1 Reply Last reply Reply Quote 0
                      • J
                        jrey @johnpoz
                        last edited by

                        @johnpoz

                        yup, I really want to see the settings of the specific DNSBL category. being used,

                        using the local-zone would also work, but in this case the OP already has a "list" in pfBocker, that blocks a bunch of other specific things, just not all the "specific things"

                        msn.com on the DNSBL category, would keep it all together for the OP.

                        certainly I don't use the list, and blocking the lookup by DNSBL works and does not return a CNAME.

                        However there might be a problem with the TLD because from the list
                        a-0003.a-msedge.net is listed.

                        The setting, CNAME Validation set being enable MIGHT be the source of the "leak". Still trying to recreate the OP's various settings to see if I can recreate the issue.

                        V 1 Reply Last reply Reply Quote 0
                        • V
                          vjizzle
                          last edited by

                          When I add msn.com to a custom DNSBL Group for blocking, pfBlockerNG works perfectly and keeps blocking msn.com and www.msn.com. That goes for every domain I add to my custom blocklist.

                          The problems starts to occur when there is CNAME resolving involved. As you can see www.msn.com resolves to a CNAME a-0003.a-msedge.net. This hostname is in the blocklist I am using:
                          screenshot01.png

                          I have TLD blocking and CNAME blocking options enabled in pfBlockerNG:
                          screenshot 02.png

                          That is why I would expect pfBlockerNG to keep blocking www.msn.com because the CNAME it resolves to is in the blocklist.

                          This is also the behaviour in pihole. When it resolves to a CNAME in a blocklist, the initial domain is blocked:
                          screenshot03.png

                          J 1 Reply Last reply Reply Quote 0
                          • V
                            vjizzle @jrey
                            last edited by

                            @jrey Please find the screenshots below on how I have setup the DNSBL group:

                            screenshot 03.png screenshot 04.png

                            1 Reply Last reply Reply Quote 0
                            • J
                              jrey @vjizzle
                              last edited by jrey

                              @vjizzle

                              That's why I want to see the log files ;-)
                              Edit and the DNSBL Groups settings for the specific group/list you have setup containing the list you referenced. oops your second post appeared lated.

                              You also have Regex Blocking enabled, do you have any entries in the list that appears below on the screen when that is enabled?

                              V 1 Reply Last reply Reply Quote 0
                              • V
                                vjizzle @jrey
                                last edited by

                                @jrey Ah yes. Are the screenshots I just posted enough? What logfiles do you want to see?

                                J 1 Reply Last reply Reply Quote 0
                                • J
                                  jrey @vjizzle
                                  last edited by

                                  @vjizzle
                                  pfblocker -> logs tab

                                  pfblockerng.log
                                  error.log (if there is anything in it)

                                  You only need to provide the sections related to processing your CustomFeed and TLD processing.

                                  1 Reply Last reply Reply Quote 0
                                  • V
                                    vjizzle
                                    last edited by vjizzle

                                    Below a snippet of the logfile:

                                    [ Force Reload Task - DNSBL ]
                                     UPDATE PROCESS START [ v3.2.0_7 ] [ 11/30/23 16:54:38 ]
                                    
                                    ===[  DNSBL Process  ]================================================
                                    
                                     Loading DNSBL Statistics... completed
                                     Loading DNSBL SafeSearch... enabled
                                     Loading DNSBL Whitelist... completed
                                    
                                    [ AdAway_Default_Blocklist ]	 Reload [ 11/30/23 16:54:39 ] . completed ..
                                      Whitelist: XXXXXXXXXnet|
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      6540     6540       0          2          0          6538                 
                                      ----------------------------------------------------------------------
                                    
                                    [ adguarddns_justdomains ]	 Reload . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      48994    48993      902        0          0          48091                
                                      ----------------------------------------------------------------------
                                    
                                    [ AmazonFire_Tracking ]		 Reload [ 11/30/23 16:54:40 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      16       16         4          0          0          12                   
                                      ----------------------------------------------------------------------
                                    
                                    [ AndroidTracking ]		 Reload . completed ..
                                      Whitelist: XXXXXXXom|
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      80       80         56         1          0          23                   
                                      ----------------------------------------------------------------------
                                    
                                    [ Disconnect_Me ]		 Reload [ 11/30/23 16:54:41 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      2701     2701       738        0          0          1963                 
                                      ----------------------------------------------------------------------
                                    
                                    [ easylist_justdomains ]	 Reload . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      22066    22045      21350      0          0          695                  
                                      ----------------------------------------------------------------------
                                    
                                    [ easyprivacy_justdomains ]	 Reload [ 11/30/23 16:54:42 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      20270    19116      7260       0          0          11856                
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_AdAway_Hosts ]		 Reload [ 11/30/23 16:54:43 ] . completed ..
                                      Whitelist: XXXXXXXXpulse.net|
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      6540     6540       6538       2          0          0                    
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_Adguard ]			 Reload . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      57509    57508      28109      0          0          29399                
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_Admiral ]			 Reload [ 11/30/23 16:54:44 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      1014     1001       919        0          0          82                   
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_Anudeep ]			 Reload [ 11/30/23 16:54:45 ] . completed ..
                                      Whitelist:XXXXXXXXse.net|
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      42536    42363      2955       1          0          39407                
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_EasyList ]			 Reload [ 11/30/23 16:54:46 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      31527    31527      31403      0          0          124                  
                                      ----------------------------------------------------------------------
                                    
                                    [ FB_jdlingyu ]			 Reload . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      1707     1645       574        0          0          1071                 
                                      ----------------------------------------------------------------------
                                    
                                    [ Firebog_easy_privacy ]	 Reload [ 11/30/23 16:54:47 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      21758    21758      10606      0          0          11152                
                                      ----------------------------------------------------------------------
                                    
                                    [ nextdns_cname_cloacking ]	 Reload . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      35       35         19         0          0          16                   
                                      ----------------------------------------------------------------------
                                    
                                    [ Public_proxies ]		 Reload [ 11/30/23 16:54:48 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      265      244        1          0          0          243                  
                                      ----------------------------------------------------------------------
                                    
                                    [ StevenBlack ]			 Reload . completed ..
                                      Whitelist: XXXXXXXXXXXn|
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      160200   160200     13687      3          0          146510               
                                      ----------------------------------------------------------------------
                                    
                                    [ TikTok ]			 Reload [ 11/30/23 16:54:51 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      3699     3699       78         0          0          3621                 
                                      ----------------------------------------------------------------------
                                    
                                    [ WindowsLies ]			 Reload [ 11/30/23 16:54:53 ] . completed ..
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      107      107        82         0          0          25                   
                                      ----------------------------------------------------------------------
                                    
                                    [ CustomFeeds_custom ]		 Reload [ 11/30/23 16:54:54 ].
                                      ----------------------------------------------------------------------
                                      Orig.    Unique     # Dups     # White    # TOP1M    Final                
                                      ----------------------------------------------------------------------
                                      11       11         7          0          0          4                    
                                      ----------------------------------------------------------------------
                                    
                                    Assembling DNSBL database...... completed [ 11/30/23 16:57:08 ]
                                    TLD:
                                     Blocking full TLD/Sub-Domain(s)... |cn|by|zip|mov| completed
                                    TLD analysis..................................... completed [ 11/30/23 16:57:25 ]
                                    TLD finalize...
                                     ----------------------------------------
                                     Original    Matches    Removed    Final     
                                     ----------------------------------------
                                     3649180     1412714    1873986    1775194   
                                     -----------------------------------------
                                    TLD finalize... completed [ 11/30/23 16:57:52 ]
                                    
                                    Saving DNSBL statistics... completed [ 11/30/23 16:58:01 ]
                                    Reloading Unbound Resolver (DNSBL python).
                                    Stopping Unbound Resolver.
                                    Unbound stopped in 2 sec.
                                    Additional mounts (DNSBL python):
                                      No changes required.
                                    Starting Unbound Resolver... completed [ 11/30/23 16:58:05 ]
                                    Resolver cache restored
                                    DNSBL update [ 1775194 | PASSED  ]... completed [ 11/30/23 16:58:07 ]
                                    

                                    screenshot05.png

                                    J 1 Reply Last reply Reply Quote 0
                                    • J
                                      jrey @vjizzle
                                      last edited by

                                      @vjizzle

                                      okay, i've recreated the issue.

                                      hold please ;-), I need to narrow it down

                                      V 1 Reply Last reply Reply Quote 1
                                      • V
                                        vjizzle @jrey
                                        last edited by

                                        @jrey
                                        I am offline for the night but I am curious what you find!

                                        J 2 Replies Last reply Reply Quote 0
                                        • J
                                          jrey @vjizzle
                                          last edited by

                                          @vjizzle

                                          Wow that's a large list

                                          TLD finalize...
                                          Original Matches Removed Final
                                          3649180 1412714 1873986 1775194

                                          Only reason I mention this is the one specific list you are providing hasn't been touched in something like 8 years. Many of the domain names in it don't even exist anymore. You might want to review your lists for being current and accurate

                                          That said with settings similar to yours, I was able to recreate the issue, with a much smaller list set.
                                          As I mentioned above I think it is a cache issue, and once I got the system into a state of working like yours, the only way I could get it functional again was:

                                          (also flush the DNS cache on the clients)

                                          on the netgate
                                          Turn off CNAME Validation
                                          Turn off Resolver Cache,
                                          restart the system
                                          let the system start fully, don't query anything.

                                          I added the top 3 entries on this list (there are many others, I checked my upstream DNS and what it had been resolving) keep in mind these are all the result of outbound queries, and won't happen unless there is a specific DNS request for them).

                                          Screen Shot 2023-12-01 at 7.10.32 AM.png
                                          Once running, you can re-enable both
                                          CNAME and Resolver Cache
                                          Reload the DNSBL as shown above
                                          then once that competes, wait a couple of minutes visit Status -> Services and restart unbound one more time.

                                          Test.
                                          I need to go back and setup the failure again, but I'd like to see your results when you do this.

                                          are you running DHCP Server on the device?
                                          also Status -> System Logs (System Tab/ DNS Resolver)
                                          filter like this, Screen Shot 2023-12-01 at 7.39.14 AM.png

                                          Thanks

                                          1 Reply Last reply Reply Quote 0
                                          • J
                                            jrey @vjizzle
                                            last edited by

                                            @vjizzle

                                            please note that once I completed the above steps,
                                            all of these lookups are within the same session, with the steps taken on pfSenses noted within.

                                             nslookup
                                            > server 10.168.1.1
                                            Default server: 10.168.1.1
                                            Address: 10.168.1.1#53
                                            > 
                                            > www.msn.com
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            Name:   www.msn.com
                                            Address: 0.0.0.0
                                            ** server can't find www.msn.com: SERVFAIL
                                             
                                            * at this point I turned off the list, removed the 3 entries and force reload the DNSBL * 
                                            
                                            > www.msn.com
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            www.msn.com     canonical name = www-msn-com.a-0003.a-msedge.net.
                                            www-msn-com.a-0003.a-msedge.net canonical name = a-0003.a-msedge.net.
                                            Name:   a-0003.a-msedge.net
                                            Address: 204.79.197.203
                                            
                                            * at this point I turned the list on again, and added the three entries, force reload the DNSBL * 
                                            
                                            > www.msn.com
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            Name:   www.msn.com
                                            Address: 0.0.0.0
                                            ** server can't find www.msn.com: SERVFAIL
                                            > a.msedge.net
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            Name:   a.msedge.net
                                            Address: 0.0.0.0
                                            ** server can't find a.msedge.net: SERVFAIL
                                            > a-msedge.net
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            Name:   a-msedge.net
                                            Address: 0.0.0.0
                                            ** server can't find a-msedge.net: SERVFAIL
                                            > dc-msedge.net
                                            Server:         10.168.1.1
                                            Address:        10.168.1.1#53
                                            
                                            Non-authoritative answer:
                                            Name:   dc-msedge.net
                                            Address: 0.0.0.0
                                            ** server can't find dc-msedge.net: SERVFAIL
                                            > 
                                            
                                            

                                            all results above from the client are within the same nslookup session. (didn't exit and/or do anything special on the client) the system is just returning the expected results again.

                                            V 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.