Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfBlockerNG v2.0 w/DNSBL

    Scheduled Pinned Locked Moved pfBlockerNG
    1.1k Posts 192 Posters 1.9m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      renegade
      last edited by

      I have added all lists on my apu.
      Now my unbound is very slow. It takes more than 800ms to resolve a hostname.
      The flicker.conf list is about 60mb.
      I have increased some values like Message Cache Size and Number of Hosts to Cache. 50mb and 100000. But it is still so sloooow.
      Any ideas?
      Thank you

      1 Reply Last reply Reply Quote 0
      • BBcan177B
        BBcan177 Moderator
        last edited by

        @renegade:

        I have added all lists on my apu.
        Now my unbound is very slow. It takes more than 800ms to resolve a hostname.
        The flicker.conf list is about 60mb.
        I have increased some values like Message Cache Size and Number of Hosts to Cache. 50mb and 100000. But it is still so sloooow.

        Ensure that your LAN devices can ping the DNSBL VIP, and also that they can browse to the DNSBL VIP and get the 1x1 pix.  If any of those two tests fail, you will see slowness as the browser is timing out waiting for the DNS resolution to the blocked Domains.

        "Experience is something you don't get until just after you need it."

        Website: http://pfBlockerNG.com
        Twitter: @BBcan177  #pfBlockerNG
        Reddit: https://www.reddit.com/r/pfBlockerNG/new/

        1 Reply Last reply Reply Quote 0
        • R
          renegade
          last edited by

          Thats works fine.
          I tested the dns resolution time in the Diagnostic menu.
          With top I can see that unbound takes a lot of cpu for some time.

          1 Reply Last reply Reply Quote 0
          • BBcan177B
            BBcan177 Moderator
            last edited by

            Is the Resolver in "Forwarder" or "Resolver" mode?

            What DNS settings are defined in the pfSense General Tab? As these settings are used by the pfSense box itself…

            When testing the ping/browse to the DNSBL VIP, do that from the LAN clients and not from pfSense itself.

            If you're in a multi-segmented LAN, ensure to use the 'DNSBL Permit Rule' option in the DNSBL Tab. Also clear your Browser/OS cache.

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            1 Reply Last reply Reply Quote 0
            • R
              renegade
              last edited by

              Resolver Mode.
              No multi segmented LAN.
              VIP is tested from remote.
              I tested a dns resolution like www.IBM.com on pfsense box.
              The slow network is reproducible on many devices.
              Maybe there is a mistake in my config which unbound makes that slow.

              image.png
              image.png_thumb

              1 Reply Last reply Reply Quote 0
              • W
                wolfsden3
                last edited by

                @BBcan177:

                Is the Resolver in "Forwarder" or "Resolver" mode?

                What DNS settings are defined in the pfSense General Tab? As these settings are used by the pfSense box itself…

                When testing the ping/browse to the DNSBL VIP, do that from the LAN clients and not from pfSense itself.

                If you're in a multi-segmented LAN, ensure to use the 'DNSBL Permit Rule' option in the DNSBL Tab. Also clear your Browser/OS cache.

                Unfortunately not 100% sure of what fixed it, but the DNSBL alerts are current. On to the next…

                1 Reply Last reply Reply Quote 0
                • BBcan177B
                  BBcan177 Moderator
                  last edited by

                  @hulleyrob:

                  Ive just noticed an annoying bug when pressing save on the Firewall/pfBlockerNG/Edit/IPv4 page if there are any errors all your data gets wiped out :-(

                  Can you please leave the data so if can be corrected?

                  FYI, this bug was fixed by the following pfSense commit for pkg_edit.php:
                  https://github.com/pfsense/pfsense/commit/a654d899cd5d288501fea1ec52dba2e3f0e479ba

                  Its in the pfSense Dev version and will be in the next pfSense 2.3.2 release.

                  "Experience is something you don't get until just after you need it."

                  Website: http://pfBlockerNG.com
                  Twitter: @BBcan177  #pfBlockerNG
                  Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                  1 Reply Last reply Reply Quote 0
                  • luckman212L
                    luckman212 LAYER 8
                    last edited by

                    @BBcan177:

                    pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

                    Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

                    1 Reply Last reply Reply Quote 0
                    • R
                      renegade
                      last edited by

                      I think i found out, why my network is so slow and unresponseful sometimes.
                      The Unbound seems to restart pretty often :(

                      Jul 17 13:57:29	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:57:29	unbound	64290:0	info: 1.000000 2.000000 4
Jul 17 13:57:29	unbound	64290:0	info: 0.524288 1.000000 3
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 2
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 14
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 10
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 13
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.0598646 median[50%]=0.127795 [75%]=0.236398
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.298118 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: requestlist max 14 avg 7.29787 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: 73 queries, 26 answers from cache, 47 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 1
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 2
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 2
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 1
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.057344 median[50%]=0.114688 [75%]=0.212992
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.156474 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: 18 queries, 11 answers from cache, 7 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:55:09	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:55:08	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:33	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: requestlist max 31 avg 16.4314 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: 53 queries, 2 answers from cache, 51 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: 2 queries, 0 answers from cache, 2 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:15	filterdns		clearing entry 192.30.253.113 from table debian_updates on host github.com
Jul 17 13:54:15	filterdns		adding entry 192.30.253.112 to table debian_updates on host github.com
Jul 17 13:53:58	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:53:58	unbound	64290:0	info: 0.262144 0.524288 14
Jul 17 13:53:58	unbound	64290:0	info: 0.131072 0.262144 26
Jul 17 13:53:58	unbound	64290:0	info: 0.065536 0.131072 18
Jul 17 13:53:58	unbound	64290:0	info: 0.032768 0.065536 9
Jul 17 13:53:58	unbound	64290:0	info: 0.016384 0.032768 4
Jul 17 13:53:58	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:53:58	unbound	64290:0	info: [25%]=0.0828302 median[50%]=0.153758 [75%]=0.243239
Jul 17 13:53:58	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:53:58	unbound	64290:0	info: average recursion processing time 0.165017 sec
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: requestlist max 12 avg 4.98592 exceeded 0 jostled 0
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: 82 queries, 11 answers from cache, 71 recursions, 0 prefetch
                      1 Reply Last reply Reply Quote 0
                      • BBcan177B
                        BBcan177 Moderator
                        last edited by

                        @luckman212:

                        @BBcan177:

                        pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

                        Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

                        Hi Luke,

                        When 2.3 was in testing, I attempted to switch to NGINX; however, ran into issues on DNSBL logging for HTTPS alerts. Lighttpd has an error-conditional-log that contains some details that the pkg uses to provide some Alert details for HTTPS alerts. With NGINX, when the browser drops the connection to the DNSBL webserver due the incorrect Certificate provided, no other logging is completed by NGINX. I managed to get the Devs to add LUA support to NGINX, but there were issues with OpenSSL at the time that prevented further testing. I may revisit this as time permits, but so far, there hasn't been any issues with both NGINX and Lighttpd running at the same time.

                        "Experience is something you don't get until just after you need it."

                        Website: http://pfBlockerNG.com
                        Twitter: @BBcan177  #pfBlockerNG
                        Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                        1 Reply Last reply Reply Quote 0
                        • BBcan177B
                          BBcan177 Moderator
                          last edited by

                          @renegade:

                          I think i found out, why my network is so slow and unresponseful sometimes.
                          The Unbound seems to restart pretty often

                          Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

                          "Experience is something you don't get until just after you need it."

                          Website: http://pfBlockerNG.com
                          Twitter: @BBcan177  #pfBlockerNG
                          Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                          1 Reply Last reply Reply Quote 0
                          • BBcan177B
                            BBcan177 Moderator
                            last edited by

                            pfBlockerNG v2.1.1 w/TLD - Pull request has been posted:
                                https://forum.pfsense.org/index.php?topic=115357.0

                            "Experience is something you don't get until just after you need it."

                            Website: http://pfBlockerNG.com
                            Twitter: @BBcan177  #pfBlockerNG
                            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                            1 Reply Last reply Reply Quote 0
                            • R
                              renegade
                              last edited by

                              @BBcan177:

                              @renegade:

                              I think i found out, why my network is so slow and unresponseful sometimes.
                              The Unbound seems to restart pretty often

                              Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

                              I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
                              Yes, i have both boxes checked. (I need this).
                              I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

                              1 Reply Last reply Reply Quote 0
                              • P
                                pfcode
                                last edited by

                                @renegade:

                                @BBcan177:

                                @renegade:

                                I think i found out, why my network is so slow and unresponseful sometimes.
                                The Unbound seems to restart pretty often

                                Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

                                I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
                                Yes, i have both boxes checked. (I need this).
                                I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

                                There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

                                Release: pfSense 2.4.3(amd64)
                                M/B: Supermicro A1SRi-2558F
                                HDD: Intel X25-M 160G
                                RAM: 2x8Gb Kingston ECC ValueRAM
                                AP: Netgear R7000 (XWRT), Unifi AC Pro

                                1 Reply Last reply Reply Quote 0
                                • BBcan177B
                                  BBcan177 Moderator
                                  last edited by

                                  @renegade:

                                  @BBcan177:

                                  @renegade:

                                  I think i found out, why my network is so slow and unresponseful sometimes.
                                  The Unbound seems to restart pretty often

                                  Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

                                  I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
                                  Yes, i have both boxes checked. (I need this).
                                  I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

                                  Check the Resolver.log for additional details… You might have to increase the Log Verbosity to get it to show more details to help diagnose the issue.

                                  "Experience is something you don't get until just after you need it."

                                  Website: http://pfBlockerNG.com
                                  Twitter: @BBcan177  #pfBlockerNG
                                  Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                  1 Reply Last reply Reply Quote 0
                                  • BBcan177B
                                    BBcan177 Moderator
                                    last edited by

                                    @pfcode:

                                    There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

                                    Please post this issue in Redmine. Its the best way to help get this resolved.

                                    https://redmine.pfsense.org/projects/pfsense

                                    "Experience is something you don't get until just after you need it."

                                    Website: http://pfBlockerNG.com
                                    Twitter: @BBcan177  #pfBlockerNG
                                    Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                    1 Reply Last reply Reply Quote 0
                                    • R
                                      renegade
                                      last edited by

                                      So i would have to disable DNSBL every morning after my 24h disconnect, restart unbound and enable DNSBL again…?
                                      I do not really understand this bug, so i think that i am not the eligible person to file an issue.

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        trumee
                                        last edited by

                                        I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:

                                        
dotomi.com
dpbolvw.net
evyy.net
doubleclick.net
avantlink.com
tkqlhce.com
pjatr.com
shareasale.com
pntrack.com
ojrq.net
clickserve.cc-dt.com
jdoqocy.com
pntrac.com
affiliatefuture.com
anrdoezrs.net
gan.doubleclick.net
emjcd.com
commission-junction.com
cj.com
apmebf.com
gopjn.com
bfast.com
redirect.at
redirectingat.com
7eer.net
linksynergy.com
kqzyfj.com
cc-dt.com
altfarm.mediaplex.com
onenetworkdirect.net
go2jump.org
pntra.com
pjtra.com
qksrv.net
ad.doubleclick.net
cj.dotomi.com

                                        The following link failed:

                                        
http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com
                                        1 Reply Last reply Reply Quote 0
                                        • BBcan177B
                                          BBcan177 Moderator
                                          last edited by

                                          @trumee:

                                          I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:

                                          The following link failed:

                                          
http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com

                                          You also need to Whitelist:
                                              www.tkqlhce.com

                                          add that to the Whitelist, and run a "Force Reload - DNSBL"

                                          "Experience is something you don't get until just after you need it."

                                          Website: http://pfBlockerNG.com
                                          Twitter: @BBcan177  #pfBlockerNG
                                          Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            trumee
                                            last edited by

                                            Is it possible to have a regex like *tkqlhce.com to block all the subdomains?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.