pfBlockerNG-devel v2.2.5_20 PR #610
-
Link: https://www.patreon.com/posts/23743067
MaxMind has deprecated GeoIP in favor of their new MMDB database format. This pull request is for the pfBlockerNG-devel version.
Changelog:
Update Maxmind library from GeoIP -> libmaxminddb Port
Relocate EasyList to the Feeds tab to be added like all other Feeds.
Modify EasyList parser
Allow other Easylist/ADBlock/uBlock/ADGuard feed syntax to the DNSBL parser
Add more EasyList Language specific feeds
Add "Server.max-request-size = 1" to Lighttpd config
Alerts Tab/Logs: Collect NAT IP addresses by Target:Port
Improve SQLite3 DB validations
Fix issue causing DNSBL/Unbound Counters to report over 100%
The next version of pfBlockerNG-devel is planned for February 2019 which will include DNSBL IPv6 Blocking, and the new Python Unbound Integration that will be sure to blow your socks off ! ! -
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
The next version of pfBlockerNG-devel is planned for February 2019 which will include DNSBL IPv6 Blocking, and the new Python Unbound Integration that will be sure to blow your socks off ! !
How is pfBlockerNG going to activate the Python support in Unbound?
I have already added Python support on my install by using a System Patch (https://github.com/twitched/pfsense/commit/1ff1605e8d2e2c9f87aac489fd7af7a407b3440c.patch) and an early shell command to nullfs mount the python libraries into the unbound chroot (/sbin/mount -t nullfs /usr/local/lib/python2.7 /var/unbound/usr/local/lib/python2.7).
Are you going to do it in a similar way? I just want to make sure there isn't going to be conflict when this gets released.
-
I pushed a PR to add Python options to the pfSense Resolver GUI.
https://github.com/pfsense/pfsense/pull/4029
But only one script can run at a time.
-
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
I pushed a PR to add Python options to the pfSense Resolver GUI.
https://github.com/pfsense/pfsense/pull/4029
So for most users this will also require a pfSense update before it can be used, as this isn't in 2.4.4p2 as of yet.
But only one script can run at a time.
No problem, I'll just merge my script with yours using the System Patches package.
-
@grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
@grimson
I pushed a PR to add Python options to the pfSense Resolver GUI.
https://github.com/pfsense/pfsense/pull/4029So for most users this will also require a pfSense update before it can be used, as this isn't in 2.4.4p2 as of yet.
But only one script can run at a time.
No problem, I'll just merge my script with yours using the System Patches package.
Yes it will require an update or 2.4.5 I believe.
Depending on what your script does, it might have to be integrated differently in the pfB python script. -
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
Depending on what your script does, it might have to be integrated differently in the pfB python script.
It's mostly an extension of this one: https://gist.github.com/FiloSottile/e2cffde2bae1ea0c14eada229543aebd/ to prevent IPv6 resolution for services that don't like the He.net tunnel prefixes.
If you want to add that functionality into pfBlockerNG I wouldn't mind. But I'm not sure how much sense this would make, as I doubt there are that many user interested in it. Also I have no problem doing it on my own.
-
Another question, is it intentional that version 2.2.5_20 no longer uses a lot of the domains listed under
---------------------------Third-party advertisers---------------------------!
! *** easylist:easylist/easylist_adservers.txt ***in the Easylist feeds. Basicly all domains ending with "$third-party" are no longer filtered.
Edit:
Correction, all domain entries containing a "$" are no longer filtered, that also includes things like "$popup,third-party".Edit2:
As an interim solution I created a little patch that removes some of the filter options (https://adblockplus.org/filter-cheatsheet#filter-options) before the lists are parsed:--- pfblockerng.inc 2019-01-12 22:07:42.021169200 +0100 +++ pfblockerng.inc 2019-01-12 21:59:19.000000000 +0100 @@ -5655,6 +5655,7 @@ // Variables for Easylists $easylist = $validate_header = FALSE; + $e_pre_replace = array( '$popup,third-party', '$popup', '$script,third-party', '$script', '$image,third-party', '$image', '$third-party' ); $e_replace = array( '||', '.^', '^' ); $run_once = $csv_parser = FALSE; @@ -5694,6 +5695,9 @@ $line = trim($line, " \t\n\r\0\x0B\xC2\xA0"); if ($easylist) { + //Remove Easylist filter options + $line = str_replace($e_pre_replace, '', $line); + if (substr($line, 0, 2) !== '||' || substr($line, -1) !== '^' || strpos($line, '$') !== FALSE ||
-
@grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:
How is pfBlockerNG going to activate the Python support in Unbound?
I have already added Python support on my install by using a System Patch (https://github.com/twitched/pfsense/commit/1ff1605e8d2e2c9f87aac489fd7af7a407b3440c.patch) and an early shell command to nullfs mount the python libraries into the unbound chroot (/sbin/mount -t nullfs /usr/local/lib/python2.7 /var/unbound/usr/local/lib/python2.7).
Are you going to do it in a similar way? I just want to make sure there isn't going to be conflict when this gets released.This might be of interest for you instead of the python script:
https://nlnetlabs.nl/documentation/unbound/unbound.conf/
DNS64 Module Options The dns64 module must be configured in the module-config: "dns64 val- idator iterator" directive and be compiled into the daemon to be enabled. These settings go in the server: section. dns64-prefix: <IPv6 prefix> This sets the DNS64 prefix to use to synthesize AAAA records with. It must be /96 or shorter. The default prefix is 64:ff9b::/96. dns64-synthall: <yes or no> Debug option, default no. If enabled, synthesize all AAAA records despite the presence of actual AAAA records. dns64-ignore-aaaa: <name> List domain for which the AAAA records are ignored and the A record is used by dns64 processing instead. Can be entered mul- tiple times, list a new domain for which it applies, one per line. Applies also to names underneath the name given.
-
@grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:
Another question, is it intentional that version 2.2.5_20 no longer uses a lot of the domains listed under
---------------------------Third-party advertisers---------------------------!
! *** easylist:easylist/easylist_adservers.txt ***in the Easylist feeds. Basicly all domains ending with "$third-party" are no longer filtered.
The previous DNSBL parser was a bit aggressive in the domains that it would parse from the EasyList feeds. I intentionally reduced the parser to lines that start with "||" and end with "^"... The other variations can lead to FPs... other DNSBL Feeds will most likely add any missing domains.
If I am missing something, please let me know...
-
This post is deleted! -
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
This might be of interest for you instead of the python script:
https://nlnetlabs.nl/documentation/unbound/unbound.conf/
DNS64 Module Options
Not really. This could probably be used to generate false AAAA records, but that is not the intention. The intention is to return no AAAA records at all, so that the (for example Netflix) client isn't even trying to use IPv6 at all. False records would be the same as blocking IPv6, it will cause disruptions and possibly error messages.
But you don't have to worry about that, I have no issues with adding that functionality by myself.
@bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:
The previous DNSBL parser was a bit aggressive in the domains that it would parse from the EasyList feeds. I intentionally reduced the parser to lines that start with "||" and end with "^"... The other variations can lead to FPs... other DNSBL Feeds will most likely add any missing domains.
If I am missing something, please let me know...
Well the EasyList feeds are IMHO among the best, but with the current parser they are mostly useless as they classify most ad networks/sources as third-party given that you rarely visit these domains directly but only get the ads from them. Prime example is googleadservices*com, with the current parser (TLD is still enabled) it is no longer blocked using EasyList.
Adding other (probably less maintained) lists to fill those holes isn't a solution I'm too happy with, but again that is IMHO and I have no problem patching the parser for my needs.
-
Unbound 1.9.3 will allow multiple Python modules: https://github.com/NLnetLabs/unbound/pull/6