Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfBlockerNG v2.0 w/DNSBL

    Scheduled Pinned Locked Moved pfBlockerNG
    1.1k Posts 192 Posters 1.7m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      erwintwr
      last edited by

      Hi guys

      i am busy running a test server where i plan to simulate our current Squid+squidguard setup, but add DNSBL to it.

      one think i note is that some ad's are still getting through ( comparing with AdBlock on chrome)

      Is it possible to load an AdBlock list ( and from where?) , so that user experience is similar to having AdBlock extension activate?

      Thx

      1 Reply Last reply Reply Quote 0
      • P
        pfBasic Banned
        last edited by

        It looks like I got one thing working on my setup but killed DNSBL in the process.

        I recently setup policy based routing where my pass firewall rules determine the Gateway to be used. I did this so that I could have one interface route all traffic through my VPN client and another interface not utilize the VPN at all. That works great, but ever since I set that up I have 0 hits on all of my DNSBL feeds and ads are coming through.

        Any ideas on how I can use pfBlockerNG DNSBL while still utilizing policy based routing for my purposes?

        1 Reply Last reply Reply Quote 0
        • X
          XmickS
          last edited by

          My DNSBL has stopped working. When I do a Fprce Update, I get this in the log

          ------------------------------------------
          Assembling database... completed
          Validating database... completed
          Reloading Unbound... Failed to Reload... Restoring previous database.... Not completed.
          
          *** DNSBL update [ 0 ] [ 8654 ] ... OUT OF SYNC ! ***
          ------------------------------------------
          

          When I execute "Shell Output - unbound-control -c /var/unbound/unbound.conf status"
          I get this:

          error: SSL handshake failed
          34386131464:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/builder/pfsense-232/tmp/FreeBSD-src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1191:
          

          I've just uninstalled PfBlockerNG and after that a reboot, but it still persists. Any ideas?

          1 Reply Last reply Reply Quote 0
          • L
            laviniuc
            last edited by

            the cron/reload/update job stalls for me. no error, just stops at dnsbl. works if i disable alexa whitelist or if i stop the dnsbl component. anyone else having this problem?

            1 Reply Last reply Reply Quote 0
            • BBcan177B
              BBcan177 Moderator
              last edited by

              @laviniuc:

              the cron/reload/update job stalls for me. no error, just stops at dnsbl. works if i disable alexa whitelist or if i stop the dnsbl component. anyone else having this problem?

              It might be related to Alexa retiring its TOP1M feed… They have since restored the feed, but uncertain of its ultimate fate...

              https://twitter.com/adamcaudill/status/800813805009768448
              https://twitter.com/Alexa_Support/status/801167423726489600

              I would try the following and see if it fixes this issue:

              mv /var/db/pfblockerng/top-1m.csv /var/db/pfblockerng/top-1m.csv.old
              

              Then run a "Force Reload" and see if it re-downloads the Alexa Archive and rebuilds the Alexa whitelist…

              As reference, the Alexa TOP1M feed url:

              https://s3.amazonaws.com/alexa-static/top-1m.csv.zip
              

              "Experience is something you don't get until just after you need it."

              Website: http://pfBlockerNG.com
              Twitter: @BBcan177  #pfBlockerNG
              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

              1 Reply Last reply Reply Quote 0
              • L
                laviniuc
                last edited by

                @BBcan177:

                @laviniuc:

                the cron/reload/update job stalls for me. no error, just stops at dnsbl. works if i disable alexa whitelist or if i stop the dnsbl component. anyone else having this problem?

                It might be related to Alexa retiring its TOP1M feed… They have since restored the feed, but uncertain of its ultimate fate...

                https://twitter.com/adamcaudill/status/800813805009768448
                https://twitter.com/Alexa_Support/status/801167423726489600

                I would try the following and see if it fixes this issue:

                mv /var/db/pfblockerng/top-1m.csv /var/db/pfblockerng/top-1m.csv.old
                

                Then run a "Force Reload" and see if it re-downloads the Alexa Archive and rebuilds the Alexa whitelist…

                As reference, the Alexa TOP1M feed url:

                https://s3.amazonaws.com/alexa-static/top-1m.csv.zip
                

                thanks, much obliged, it works perfectly now. i'll keep on the lookout for if they drop the feed again… maybe you could check if anything weird happens on the import code if the url can't be found? before i stopped the import / disabled dnsbl it was stalling the cpu/dns resolver (guessing weird things happened when the first import took as long as the refresh timeout and then the second import started wihtout the first one completing?)

                1 Reply Last reply Reply Quote 0
                • T
                  TomT
                  last edited by

                  Hi
                  I've just installed pfBlockerNG 2.1.1_4 on a test pfSense install.

                  I've enabled it and then gone to update.
                  I've selected the option to reload but nothing is downloaded.

                  Checking /var/db/aliastables there is nothing showing.
                  Can anyone point me in the right direction ?

                   UPDATE PROCESS START [ 12/01/16 11:26:27 ]
                  
                  Clearing all DNSBL Feeds... 
                  ** DNSBL Disabled **
                  
                  ===[  Continent Process  ]============================================
                  
                  ===[  IPv4 Process  ]=================================================
                  
                  ===[  IPv6 Process  ]=================================================
                  
                  ===[  Aliastables / Rules  ]==========================================
                  
                  No changes to Firewall rules, skipping Filter Reload
                  No Changes to Aliases, Skipping pfctl Update
                  
                   UPDATE PROCESS ENDED
                  

                  Thanks

                  1 Reply Last reply Reply Quote 0
                  • RonpfSR
                    RonpfS
                    last edited by

                    Did you define any tables ?

                    2.4.5-RELEASE-p1 (amd64)
                    Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                    Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                    1 Reply Last reply Reply Quote 0
                    • D
                      DigitalJer
                      last edited by

                      Is it possible to use an alternative list, such as:

                      https://blog.majestic.com/development/alexa-top-1-million-sites-retired-heres-majestic-million/

                      –------------------------------------------------
                      2.4.3-RELEASE (amd64)
                      built on Mon Mar 26 18:02:04 CDT 2018
                      FreeBSD 11.1-RELEASE-p7
                      VM in ESXi 5.5
                      1 x 1000baseTX (WAN)
                      1 x 1000baseTX (LAN)

                      1 Reply Last reply Reply Quote 0
                      • B
                        BoxMacNCheese
                        last edited by

                        @tonymorella:

                        @Atlan:

                        I am receiving certificate invalid errors after enabling the DNSBL when navigating certain sites - so far the culprit seems to mainly be googleads.g.doubleclick.net when trying to serve content over an HTTPS enabled website.

                        The attempt still shows in the Alerts tab of pfBlockerNG, the main site still loads, but it seems to be wanting to use the firewall's self signed certificate as the certificate for the ad server host.

                        Did I miss a setting somewhere to prevent this from happening?

                        Ya ran into the same problem while beta testing, completely forgot about this issue until I read your post :) For a quick fix I created a CA and web certificate via pfsense GUI then copies this cert to /var/unbound/dnsbl_cert.pem which dnsbl light http will use.  Then push out the CA to all the desktops, could be done via GPO or login scripts. With https://letsencrypt.org/ picking up traction, everything is going to be running over TLS or SSL soon going to have to implement this type of fix.

                        **EDIT: Sorry guys did a bad job on this post and what I was doing, it does not fix the underlying issue with the TLS stream being bad because the cert is not valid, reggie14 did a much better job testing this than I. In my setup is simply helped limit the errors in Chrome. Again just a quick fix that might help. Again sorry for the confusion.

                        FYI the update from Chrome 45 to 46 made a difference in what come up under security warnings:
                        http://arstechnica.com/information-technology/2015/10/chrome-finally-kills-off-the-http-https-mixed-content-warning/**

                        Thanks
                        Tony M

                        I am new to pfsense and new to this forum. I do not tend to go digging up the past but was there any conclusion to the https page errors received while using DNSBL? I initially scrapped my setup of squidguard due to its failure to even pass through to HTTPS sites. Now the DNSBL is working adequately but if I am to block certain domains (such as www.yahoo.com), I receive an insecure connection error on my browser instead of the 1x1 pixel error page. Is there a way for me to get rid of this error? Or is there a way to potentially redirect this https domain being blocked to an internal webpage?

                        Also if this is in the wrong section or if this has already been answered I apologize in advance.

                        1 Reply Last reply Reply Quote 0
                        • BBcan177B
                          BBcan177 Moderator
                          last edited by

                          @Knyte:

                          Is it possible to use an alternative list, such as:

                          https://blog.majestic.com/development/alexa-top-1-million-sites-retired-heres-majestic-million/

                          Alexa has re-instated the Alexa Top1M Feed, not sure if/when they will discontinue it again….
                              https://twitter.com/Alexa_Support/status/801167423726489600

                          However, Cisco has recently released an alternative TOP1M version:
                              https://blog.opendns.com/2016/12/14/cisco-umbrella-1-million/

                          I will be adding this to the next release as an alternative option to the package...

                          "Experience is something you don't get until just after you need it."

                          Website: http://pfBlockerNG.com
                          Twitter: @BBcan177  #pfBlockerNG
                          Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                          1 Reply Last reply Reply Quote 0
                          • BBcan177B
                            BBcan177 Moderator
                            last edited by

                            @BoxMacNCheese:

                            …https page errors received while using DNSBL?

                            Usually this occurs with Safari Browsers only… I don't know of any workarounds, except for whitelisting those domains that are causing this behaviour...

                            "Experience is something you don't get until just after you need it."

                            Website: http://pfBlockerNG.com
                            Twitter: @BBcan177  #pfBlockerNG
                            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                            1 Reply Last reply Reply Quote 0
                            • A
                              akishore
                              last edited by

                              Hi BBcan177,

                              Thanks for the awesome package. I had a quick question for you. I added quite a few feeds that you had mentioned in the beginning of this thread and everything was working great. Anyway, I found some more feeds, like really big ones and tried to add those.

                              It loaded, but the database grew to about 1.4 million entries. Firstly, it started giving me a message about a 2 million hard limit and also, my entire device basically crashed. I have the SG-2440, so it's not one of the higher end models, but I didn't think it would crash the device.

                              Anyway, Unbound could not be reloaded and it reverted back to the smaller database (which was about 500,000 URLs). Can my device just not handle that big of a table or is there some other way to fix this?

                              FYI, the two lists that I added that killed the system were these:

                              http://osint.bambenekconsulting.com/feeds/dga-feed.gz
                              http://osint.bambenekconsulting.com/feeds/c2-dommasterlist.txt

                              Thanks,

                              AK

                              1 Reply Last reply Reply Quote 0
                              • RonpfSR
                                RonpfS
                                last edited by

                                Try to raise System/ Advanced / Firewall & NAT : Firewall Maximum Table Entries, but this may only be useful for IP tables.

                                Maybe you are just running out of memory, those DNSBL tables are huge.
                                -rw-r–r--  1 root  wheel  45701811 Dec 18 14:22 BBC_DGA.txt

                                2.4.5-RELEASE-p1 (amd64)
                                Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                1 Reply Last reply Reply Quote 0
                                • B
                                  BoxMacNCheese
                                  last edited by

                                  Okay so as I understand I will not be able to avoid the HTTPS errors in my browser due to the nature of DNS. For example, if I were to block www.aol.com in a DNSBL feed, I would be redirected to the 1x1 pixel block page at my DNSBL VIP of 192.168.1.10. If I went to https://www.aol.com I receive a browser error page of "ssl_error_bad_cert_domain" because the DNSBL certificate obviously doesn't match up with the DNS entry of www.aol.com and the internal DNSBL VIP.

                                  1. Would I be able to redirect all of the traffic that is blocked and goes to the DNSBL VIP address to go to an internal (or external) custom webpage. Or perhaps have it be redirected to Google? Even if it was HTTPS would it not receive an error and pass through?

                                  2. Where is the filepath of DNSBL package, or more directly, where is the location of the 1x1 pixel block page (or image) since I would like to put my own error notice.

                                  Thanks.

                                  1 Reply Last reply Reply Quote 0
                                  • RonpfSR
                                    RonpfS
                                    last edited by

                                    @BoxMacNCheese:

                                    2. Where is the filepath of DNSBL package, or more directly, where is the location of the 1x1 pixel block page (or image) since I would like to put my own error notice.

                                    https://forum.pfsense.org/index.php?topic=120253.0

                                    2.4.5-RELEASE-p1 (amd64)
                                    Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                    Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      BoxMacNCheese
                                      last edited by

                                      @BoxMacNCheese:

                                      1. Would I be able to redirect all of the traffic that is blocked and goes to the DNSBL VIP address to go to an internal (or external) custom webpage. Or perhaps have it be redirected to Google? Even if it was HTTPS would it not receive an error and pass through?

                                      Thanks.

                                      1 Reply Last reply Reply Quote 0
                                      • QinnQ
                                        Qinn
                                        last edited by

                                        @BBcan177:

                                        Here are some basic instructions to get started with DNSBL.

                                        1. Open the pfBNG "DNSBL" Tab:

                                        (Use the defaults unless you have a need to use otherwise)
                                        Enter the DNSBL VIP as 10.10.10.1
                                        Enter the DNSBL Listening Port as 8081
                                        Enter the DNSBL SSL Listening port as 8443
                                        Select the DNSBL Listening Interface as Lan

                                        For the DNSBL Firewall Rule select all of the LAN subnets that access the DNS Resolver.
                                        Ensure that all Devices that use the DNS Resolver, have the Resolver as its only DNS setting for DNSBL to function properly.

                                        DNSBL IP Firewall Rule Settings:
                                        Select Deny outbound or as per your requirements
                                        select Enable logging

                                        Alexa  (is optional, you can skip this until later if you wish)
                                        Select Top 1K
                                        Select the TLD Inclusions as ca,co,com,io,me,net,org or as required.

                                        In the Custom List you may enter any domain you wish to Whitelist.

                                        Save your settings

                                        1. Open the "DNSBL Feeds" Tab:

                                        Create a new DNSBL Alias

                                        Enter DNS Group Name as ADs
                                        Enter Description as DNSBL ADverts

                                        DNSBL:

                                        Enter the Header/Label and Source URL as follows:
                                        (Use copy/paste as plain text for the URL)

                                        Format Auto and State ON

                                        yoyo
                                        http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext

                                        hpHosts_ads
                                        http://hosts-file.net/ad_servers.txt

                                        Adaway
                                        https://adaway.org/hosts.txt

                                        Cameleon
                                        http://sysctl.org/cameleon/hosts

                                        Select List Action as Unbound
                                        Select Update Frequency as Once a day

                                        Alexa:
                                        Do not enable the Alexa Whitelist for this ADverts based alias, as Alexa also posts the top ADvert servers. So using Alexa whitelist, will interfere with ADvert Blocking.

                                        Add any other domains that you wish to block in the Custom List.

                                        Save your settings

                                        1. Open the "DNSBL" Tab:

                                        Click DNSBL Enable checkbox.
                                        Save your settings

                                        1. Open the "Update" Tab:

                                        Select Force Update

                                        You should now see the DNSBL Feeds being downloaded and parsed. Once that is complete, goto the Dashboard, and confirm that the widget is populated correctly.

                                        1. Goto the pfBlockerNG "Alerts" Tab:

                                        Any domain that is blocked will be reported here. For HTTPS alerts, the SRC IP and URL are not captured due to Browser security measures.

                                        As a test, goto www.aol.com  and www.yahoo.com  and see some alerts populate.

                                        There are several other DNSBL Feeds that can be used with pfBNG DNSBL. I will post that at a later date, once users get their basic configurations working. There is also an ADBlock Easylist tab, which is pretty self-explanatory.

                                        I tried to look for an answer, but I cannot find it (maybe it's obvious), but can someone assure me?
                                        BBcan177 advises not to filter the DNSBL feeds mentioned above with Alexa. So  does that mean that for all DNSBL feeds, other than advertising, you should enable Alexa?

                                        Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                                        Firmware: Latest-stable-pfSense CE (amd64)
                                        Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                                        1 Reply Last reply Reply Quote 0
                                        • C
                                          chrcoluk
                                          last edited by

                                          alexa is a preference thing.

                                          Enabling it means you will not accidentally block popular websites.  On the flipside should one of those sites ever get compromised, you also wouldnt be filtering it.

                                          My own reason for posting here is I wonder if anyone has an issue with unbound been unresponsive during reload's?  If I add the cryptolocker domains of which there is almost 1 million of them (surely some are dead and not been cleaned up), unbound then needs 20+ seconds to reload at which time it is unresponsive.

                                          pfSense CE 2.7.2

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            chrcoluk
                                            last edited by

                                            @akishore:

                                            Hi BBcan177,

                                            Thanks for the awesome package. I had a quick question for you. I added quite a few feeds that you had mentioned in the beginning of this thread and everything was working great. Anyway, I found some more feeds, like really big ones and tried to add those.

                                            It loaded, but the database grew to about 1.4 million entries. Firstly, it started giving me a message about a 2 million hard limit and also, my entire device basically crashed. I have the SG-2440, so it's not one of the higher end models, but I didn't think it would crash the device.

                                            Anyway, Unbound could not be reloaded and it reverted back to the smaller database (which was about 500,000 URLs). Can my device just not handle that big of a table or is there some other way to fix this?

                                            FYI, the two lists that I added that killed the system were these:

                                            http://osint.bambenekconsulting.com/feeds/dga-feed.gz
                                            http://osint.bambenekconsulting.com/feeds/c2-dommasterlist.txt

                                            Thanks,

                                            AK

                                            My device is higher end than yours, it didnt crash but unbound started been weird, it kept restarting and on each restart it was unresponsive.

                                            I then did a bit of research as I found it very hard to believe that dga list is all live domains and it turns out I am right.

                                            Basically the dga list is someone has inputted known seeds used by botnets and made a list of all the possible domains to be used from those seeds, these are not verified live domains so given the resource complications I removed the dga, the c2 list is verified active domains,. but of course is also much smaller so shouldnt give you problems to use.

                                            pfSense CE 2.7.2

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.