PfBlockerNG v2.0 w/DNSBL




  • Moderator

    A PR has been merged for pfBlockerNG. v2.1.1_6
      https://github.com/pfsense/FreeBSD-ports/pull/253

    Fixes:

    • EasyList has recently changed the format of the feed. DNSBL parser has been updated.
    • Fix XMLRPC Sync setting that handles the Sync of the General settings to remote hosts.
    • Fix DNSBL Certificate creation for pfSense 2.4


  • @BBcan177:

    A PR has been merged for pfBlockerNG. v2.1.1_6
      https://github.com/pfsense/FreeBSD-ports/pull/253

    Fixes:

    • EasyList has recently changed the format of the feed. DNSBL parser has been updated.
    • Fix XMLRPC Sync setting that handles the Sync of the General settings to remote hosts.
    • Fix DNSBL Certificate creation for pfSense 2.4

    Hello, thanks for all your work on this. I just wanted to report that I'm seeing the following in system -> pkg manager -> installed, even after reinstalling the package.

    http://imgur.com/a/I3mep


  • Banned

    @mk6032:

    I just wanted to report that I'm seeing the following in system -> pkg manager -> installed, even after reinstalling the package.

    http://imgur.com/a/I3mep

    Yes. And the point of the report is exactly what?



  • @doktornotor:

    @mk6032:

    I just wanted to report that I'm seeing the following in system -> pkg manager -> installed, even after reinstalling the package.

    http://imgur.com/a/I3mep

    Yes. And the point of the report is exactly what?

    Please review the linked image. There are two points :

    1. "Package is configured but not (fully) installed" (even after reinstallation)
    2. "Newer version available" (without the update option)

    Regards,
    Matt



  • @mk6032:

    Please review the linked image. There are two points :

    1. "Package is configured but not (fully) installed" (even after reinstallation)
    2. "Newer version available" (without the update option)

    Regards,
    Matt

    That is the Legend for the Package Manager  ::)

    So now look a the pfBlockerNG line and interpreting the legend the status of the package is "Current"



  • @RonpfS:

    That is the Legend for the Package Manager  ::)

    So now look a the pfBlockerNG line and interpreting the legend the status of the package is "Current"

    Ah, my bad! The lack of a separator there as opposed to the ones above it is a bit confusing ( http://imgur.com/a/lwSFx ).

    Regards,
    Matt


  • Banned

    @mk6032:

    @RonpfS:

    That is the Legend for the Package Manager  ::)

    So now look a the pfBlockerNG line and interpreting the legend the status of the package is "Current"

    Ah, my bad! The lack of a separator there as opposed to the ones above it is a bit confusing ( http://imgur.com/a/lwSFx ).

    Regards,
    Matt

    LOL, ok… https://redmine.pfsense.org/issues/7203



  • @doktornotor:

    LOL, ok… https://redmine.pfsense.org/issues/7203

    Maybe just add "Legend : "


  • Banned

    @RonpfS:

    @doktornotor:

    LOL, ok… https://redmine.pfsense.org/issues/7203

    Maybe just add "Legend : "

    Yeah I tried and that looked even worse than now. Leaving this to someone else.


  • Banned

    Hello,

    Regarding Adaway,

    https://adaway.org/hosts.txt

    Although the list is downloaded:

    [ Adaway ] Downloading update .. 200 OK. completed ..

    I got this error:

    [ pfB_Adaway Adaway ] List Error ]

    Please advise


  • Moderator

    I assume that Adaway is also the name of the Alias?

    Do you have anything in the custom list at the bottom of this alias? I think that's the part that's not parsing properly. Anything in a custom list must be formatted properly.


  • Banned

    @BBcan177:

    I assume that Adaway is also the name of the Alias?

    Do you have anything in the custom list at the bottom of this alias? I think that's the part that's not parsing properly. Anything in a custom list must be formatted properly.

    The name of the alias is Adaway, and I don't have anything in the custom list.

    I've changed the name of the alias, and the label like this:

    Adaway_list ] Downloading update .. 200 OK. completed ..
    [ pfB_Adaway1 Adaway_list ] List Error ]


  • Moderator

    It looks like you added this list to the IPv4 tab when it's a domain based list and should only be used in the DNSBL tab.


  • Banned

    @BBcan177:

    It looks like you added this list to the IPv4 tab when it's a domain based list and should only be used in the DNSBL tab.

    I removed it from IPV4, sorry to bother you with this.

    Thank you



  • @BBcan177:

    Here are more DNSBL Feeds that can be used in pfBlockerNG.
    (Copy and paste URLS as plain text)

    1. Create a new alias for these.
      These are not necessarily ADvert domains. So I named mine "Malicious"

    malc0de
    https://malc0de.com/bl/BOOT

    I get this error for malc0de. all other lists are fine

    [ DNSBL_Malicious - malc0de ] Download FAIL [ 02/21/17 01:21:47 ]
      Firewall and/or IDS are not blocking download.

    Set up exactly the way you have described.

    I'm on 2.3.3 RC


  • Moderator

    @yodaphone:

    I get this error for malc0de. all other lists are fine

    [ DNSBL_Malicious - malc0de ] Download FAIL [ 02/21/17 01:21:47 ]
      Firewall and/or IDS are not blocking download.

    Not having any issues with that feed…  Try to

    ping malc0de.com
    

    and see if it responds… Check the Alerts Tab to see if any alerts get reported... Could be Snort/Suricata, or another Firewall rule....



  • Hey BBCAN,
    Is there a way to get Reek's anti-adblocking list to work under Pfblocker? This would be extremely convenient.

    My apologies if this has been asked a trillion times before….


  • Moderator

    @HeatmiserNYC:

    Is there a way to get Reek's anti-adblocking list to work under Pfblocker? This would be extremely convenient

    Unfortunately not. DNSBL can't manipulate the browser except for sinkholing the DNS request.



  • Does anyone have a good list for blocking porn sites please?b

    I have google and searched and cant find any.


  • Banned

    @Brutos:

    Does anyone have a good list for blocking porn sites please?b

    I have google and searched and cant find any.

    https://forum.pfsense.org/index.php?topic=125863.0



  • @BBcan177:

    @D0X:

    Thanks for your support so far, but for now I've given up on pfblockerng. Not that the package is flawed, but Safari on OSX and IOS is driving me nuts.

    NP…. Should you decide to pick it backup at some point, try to see why the Ping/Browse to the DNSBL VIP wasn't working. That will cause issues on its own.... also since it was one Domain that was predominantly causing issues,  you could also Whitelist it...

    Quoting this old post, but the problem is still occurring. Tried PfblockerNG again this morning, but all Apple devices still bombards me with certificate errors.


  • Moderator

    @D0X:

    Quoting this old post, but the problem is still occurring. Tried PfblockerNG again this morning, but all Apple devices still bombards me with certificate errors.

    Is this with Safari, or all other browsers?

    Can you review this link:
        https://forum.pfsense.org/index.php?topic=124945.msg691099#msg691099



  • Does anyone have a good list for blocking porn sites please?b

    There are more options to get better results and for porn sites an OpenDNS account might be a good solution as well!
    Squid & SquidGuard or DansGuardian plus pfBlockerNG & DNSBL plus an OpenDNS account



  • @BBcan177:

    @D0X:

    Quoting this old post, but the problem is still occurring. Tried PfblockerNG again this morning, but all Apple devices still bombards me with certificate errors.

    Is this with Safari, or all other browsers?

    Can you review this link:
        https://forum.pfsense.org/index.php?topic=124945.msg691099#msg691099

    Safari indeed, on all devices, moving to another browser is not an option here. Doing the mod you suggested, no logging whatsoever will work?

    Running Pi-Hole here for a few days now, as an alternative. But this means adding another box to my network. Plus the config is a little weird (pfSense DCHP server forwarding Pi-Hole dns address to clients, clients using Pi-Hole and Pi-Hole using the pfSense DNS Resolver again as its upstream DNS, don't know, feels weird).


  • Moderator

    Both solutions use Lighttpd and both push a 1x1 pix. Strange that it doesn't show any cert errors… Will have to do some digging...

    In the link, there is another option to drop just the HTTPS alerts with a firewall rule.



  • pihole it use only http, not https ( because of that on https you have error with site incorrect configured…).



  • I have been banging my head against the desk for almost a month now and have not been able to find a way to get either of my 2 pfSense VMs (vmware esxi host) to remain stable for ~12 hours with DNSBL enabled while using feeds that were on the first few pages of this forum post.  Issues that I tend to have, sometimes as little as a few hours of uptime.. sometimes around 12-20 hours.  I have been using pfblockerng for quite some time using the traditional ipv4 blocklists with no problems.  Only after I have enabled DNSBL, and added several feeds have I had these symptoms:

    • The Web GUI becomes unresponsive
    • Unbound becomes unresponsive, leading to DNS being unable to resolve for clients
    • The console is unresponsive
    • DHCPD stops functioning
    • Less frequently than the above symptoms, routing stops all together.  Most of the time as long as Unbound is responsive, then everything is fine besides being able to make changes or view anything in the WebGUI or console.

    All of these issues are only fixed by hard rebooting the instances.  Because of this, I have had to reinstall and restore via the xml config quite a few times, as I have had issues upgrading due to the instability.

    Things I have tried (but none seem to help):

    • Increasing the memory of my VMs from 4gb -> 8gb
    • Increasing Max Threads of the WebGUI to 12
    • Using Service Watchdog on impacted services

    The only thing that has improved my stability is disabling DNSBL, and I do not want to do this.  I would love to retain this functionality, but at this point it seems like no matter what I do I end up with the above symptoms.

    Any advice would be much appreciated.  I am on the latest 2.4 daily snapshots, and update as frequently as I can.


  • Moderator

    Do you see any errors in the Resolver.log?

    Its best not to use the Service watchdog with DNSBL, since it can try to restart Unbound at in appropriate times…

    Are you using the literal IP address to access to webgui?



  • @BBcan177:

    Do you see any errors in the Resolver.log?

    Its best not to use the Service watchdog with DNSBL, since it can try to restart Unbound at in appropriate times…

    Are you using the literal IP address to access to webgui?

    I have been connecting to the WebGUI only via ip address, not hostname.  I have turned off the service watchdog for DNSBL and Unbound.

    Here is the entire log for 1 of my VMs,  I am in the process of restoring the other one from XML at the moment, but will give you that one when I have a chance.

    https://pastebin.com/059BCEeV

    Edit:
    Here are some screenshots of everything that occured during syncing the DNSBL feeds of my other VM whilst restoring the XML.  I had to hard reboot the instance a few times for it to complete:

    https://imgur.com/a/788UU

    Here is the resolver.log:
    https://pastebin.com/R9jGU7Xs

    I have disabled DNSBL on 1 of my VMs, as I need at least 1 of them to be stable today (working from home).  Will monitor the other one for instability and provide an updated log when I can.  Are there any other logs that you may be interested in?



  • Did you disable the two "DHCP registrations" checkboxes?
    @BBcan177:

    Some recommendations:

    • The DNS Resolver can also be used in 'Forwardering mode'; however its best to not use this 'Forwarding mode' and keep it in 'resolver mode' as this will query the Root DNS servers for the DNS queries instead of relying on an ISPs DNS etc…

    • If you use the 'DNS Resolver Forwarder mode', only configure 'DNSSEC' if the configured DNS servers support DNSSEC. The enabling of 'DNSSEC' to harden your DNS security is highly recommended.

    • Disable the two "DHCP registrations" checkboxes, unless you really require those options.



  • @RonpfS:

    Did you disable the two "DHCP registrations" checkboxes?
    @BBcan177:

    Some recommendations:

    • The DNS Resolver can also be used in 'Forwardering mode'; however its best to not use this 'Forwarding mode' and keep it in 'resolver mode' as this will query the Root DNS servers for the DNS queries instead of relying on an ISPs DNS etc…

    • If you use the 'DNS Resolver Forwarder mode', only configure 'DNSSEC' if the configured DNS servers support DNSSEC. The enabling of 'DNSSEC' to harden your DNS security is highly recommended.

    • Disable the two "DHCP registrations" checkboxes, unless you really require those options.

    1 of my VMs has them enabled, the other does not (before reading your message).  Just wanted to point out that they are both experiencing the same, despite the initial difference in settings.  I have disabled those settings on the VM that initially had that set.



  • Look at the DHCP logs.

    DHCP registrations restart Unbound, that may takes minutes to complete.
    If another registration happens before unbound completed the restart, unbound may crash.

    Use DNS Resolver Host Overrides.



  • @RonpfS:

    Look at the DHCP logs.

    DHCP registrations restart Unbound, that may takes minutes to complete.
    If another registration happens before unbound completed the restart, unbound may crash.

    Use DNS Resolver Host Overrides.

    Good to know.  Thank you for the tip.



  • I haven't had any stability issues since disabling DNSBL across both of my VMs.  But on one of them I am getting a strange error that I cannot seem to get to go away on boot.  This doesn't seem to impact the functionality of pfBlockerNG, but it is bugging me that I can't get it to go away.  I have even tried restoring an older XML on a fresh install, but it seems that the XML is not older than when the issue began.

    Starting package pfBlockerNG… Something wrong happened while reading request.

    How can I troubleshoot this and alleviate this issue from my system?  Thank you in advance!


  • Moderator

    @TyphooN:

    I haven't had any stability issues since disabling DNSBL across both of my VMs.  But on one of them I am getting a strange error that I cannot seem to get to go away on boot.  This doesn't seem to impact the functionality of pfBlockerNG, but it is bugging me that I can't get it to go away.  I have even tried restoring an older XML on a fresh install, but it seems that the XML is not older than when the issue began.

    Starting package pfBlockerNG… Something wrong happened while reading request.

    How can I troubleshoot this and alleviate this issue from my system?  Thank you in advance!

    I read some of your posts, but there isn't much details to help you… Did you use all the default settings for DNSBL? What interfaces did you select? What VIP address? Do these overlap with other subnets? What feeds are you using in DNSBL? Did you review the pfblockerng.log/System Log/Resolver Log for clues?



  • I fixed the stability problem and the issue was not DNSBL.  It just appears that enabling DNSBL was a catalyst in setting off the real issue (I/O).  I was getting some SCSI errors after updating to the latest version of ESXI.  Once I changed the device that my VM's disks were attached to to the SATA device rather than SCSI, all issues were eliminated.  My VMs have been stable with DNSBL enabled for over 24 hours now, which has never happened before.  I am sorry for going down the wrong rabbit hole initially, but I am excited that I can now use DNSBL without any issues!  Thank you BBcan177 for all of your hard work on pfBlockerNG!


  • Moderator

    @TyphooN:

    I fixed the stability problem and the issue was not DNSBL.

    Glad you got it sorted out  :)



  • Hi,

    I have one IPV4 alias list that works before and now are generating the "/var/db/aliastables/pfB_Allow_List.txt"

    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

    <title>400 - Bad Request</title>

    400 - Bad Request

    Why that happening?

    Thanks


  • Moderator

    @heliop100:

    <title>400 - Bad Request</title>

    What URL are you using? Load the URL in the browser and see what it returns. You're getting a 400 error on connection. Check the Alerts tab if something is blocking access to that URL.

    "The 400 Bad Request error is an HTTP status code that means that the request you sent to the website server, often something simple like a request to load a web page, was somehow incorrect or corrupted and the server couldn't understand it."


Log in to reply