pfBlockerNG-devel v2.2.5_20 PR #610


  • Moderator

    Link: https://www.patreon.com/posts/23743067

    MaxMind has deprecated GeoIP in favor of their new MMDB database format. This pull request is for the pfBlockerNG-devel version.

    Changelog:
    Update Maxmind library from GeoIP -> libmaxminddb Port
    Relocate EasyList to the Feeds tab to be added like all other Feeds.
    Modify EasyList parser
    Allow other Easylist/ADBlock/uBlock/ADGuard feed syntax to the DNSBL parser
    Add more EasyList Language specific feeds
    Add "Server.max-request-size = 1" to Lighttpd config
    Alerts Tab/Logs: Collect NAT IP addresses by Target:Port
    Improve SQLite3 DB validations
    Fix issue causing DNSBL/Unbound Counters to report over 100%
    The next version of pfBlockerNG-devel is planned for February 2019 which will include DNSBL IPv6 Blocking, and the new Python Unbound Integration that will be sure to blow your socks off ! !



  • @bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:

    The next version of pfBlockerNG-devel is planned for February 2019 which will include DNSBL IPv6 Blocking, and the new Python Unbound Integration that will be sure to blow your socks off ! !

    How is pfBlockerNG going to activate the Python support in Unbound?

    I have already added Python support on my install by using a System Patch (https://github.com/twitched/pfsense/commit/1ff1605e8d2e2c9f87aac489fd7af7a407b3440c.patch) and an early shell command to nullfs mount the python libraries into the unbound chroot (/sbin/mount -t nullfs /usr/local/lib/python2.7 /var/unbound/usr/local/lib/python2.7).

    Are you going to do it in a similar way? I just want to make sure there isn't going to be conflict when this gets released.


  • Moderator

    @grimson

    I pushed a PR to add Python options to the pfSense Resolver GUI.

    https://github.com/pfsense/pfsense/pull/4029

    But only one script can run at a time.



  • @bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:

    @grimson

    I pushed a PR to add Python options to the pfSense Resolver GUI.

    https://github.com/pfsense/pfsense/pull/4029

    So for most users this will also require a pfSense update before it can be used, as this isn't in 2.4.4p2 as of yet.

    But only one script can run at a time.

    No problem, I'll just merge my script with yours using the System Patches package.


  • Moderator

    @grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:

    @bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:

    @grimson
    I pushed a PR to add Python options to the pfSense Resolver GUI.
    https://github.com/pfsense/pfsense/pull/4029

    So for most users this will also require a pfSense update before it can be used, as this isn't in 2.4.4p2 as of yet.

    But only one script can run at a time.

    No problem, I'll just merge my script with yours using the System Patches package.

    Yes it will require an update or 2.4.5 I believe.
    Depending on what your script does, it might have to be integrated differently in the pfB python script.



  • @bbcan177 said in pfBlockerNG-devel v2.2.5_20 PR #610:

    Depending on what your script does, it might have to be integrated differently in the pfB python script.

    It's mostly an extension of this one: https://gist.github.com/FiloSottile/e2cffde2bae1ea0c14eada229543aebd/ to prevent IPv6 resolution for services that don't like the He.net tunnel prefixes.

    If you want to add that functionality into pfBlockerNG I wouldn't mind. But I'm not sure how much sense this would make, as I doubt there are that many user interested in it. Also I have no problem doing it on my own.



  • Another question, is it intentional that version 2.2.5_20 no longer uses a lot of the domains listed under

    ---------------------------Third-party advertisers---------------------------!
    ! *** easylist:easylist/easylist_adservers.txt ***

    in the Easylist feeds. Basicly all domains ending with "$third-party" are no longer filtered.

    Edit:
    Correction, all domain entries containing a "$" are no longer filtered, that also includes things like "$popup,third-party".

    Edit2:
    As an interim solution I created a little patch that removes some of the filter options (https://adblockplus.org/filter-cheatsheet#filter-options) before the lists are parsed:

    --- pfblockerng.inc	2019-01-12 22:07:42.021169200 +0100
    +++ pfblockerng.inc	2019-01-12 21:59:19.000000000 +0100
    @@ -5655,6 +5655,7 @@
     
     								// Variables for Easylists
     								$easylist = $validate_header = FALSE;
    +								$e_pre_replace = array( '$popup,third-party', '$popup', '$script,third-party', '$script', '$image,third-party', '$image', '$third-party' );
     								$e_replace = array( '||', '.^', '^' );
     
     								$run_once = $csv_parser = FALSE;
    @@ -5694,6 +5695,9 @@
     											$line = trim($line, " \t\n\r\0\x0B\xC2\xA0");
     
     											if ($easylist) {
    +												//Remove Easylist filter options
    +												$line = str_replace($e_pre_replace, '', $line);
    +
     												if (substr($line, 0, 2) !== '||' ||
     												    substr($line, -1) !== '^' ||
     												    strpos($line, '$') !== FALSE ||
    
    

  • Moderator

    @grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:

    How is pfBlockerNG going to activate the Python support in Unbound?
    I have already added Python support on my install by using a System Patch (https://github.com/twitched/pfsense/commit/1ff1605e8d2e2c9f87aac489fd7af7a407b3440c.patch) and an early shell command to nullfs mount the python libraries into the unbound chroot (/sbin/mount -t nullfs /usr/local/lib/python2.7 /var/unbound/usr/local/lib/python2.7).
    Are you going to do it in a similar way? I just want to make sure there isn't going to be conflict when this gets released.

    This might be of interest for you instead of the python script:

    https://nlnetlabs.nl/documentation/unbound/unbound.conf/

    DNS64 Module Options
           The  dns64  module must be configured in the module-config: "dns64 val-
           idator iterator" directive and  be  compiled  into  the  daemon  to  be
           enabled.  These settings go in the server: section.
    
           dns64-prefix: <IPv6 prefix>
                  This  sets  the  DNS64  prefix to use to synthesize AAAA records
                  with.  It must  be  /96  or  shorter.   The  default  prefix  is
                  64:ff9b::/96.
    
           dns64-synthall: <yes or no>
                  Debug  option,  default  no.   If  enabled,  synthesize all AAAA
                  records despite the presence of actual AAAA records.
    
           dns64-ignore-aaaa: <name>
                  List domain for which the AAAA records are  ignored  and  the  A
                  record is used by dns64 processing instead.  Can be entered mul-
                  tiple times, list a new domain for which  it  applies,  one  per
                  line.  Applies also to names underneath the name given.
    

  • Moderator

    @grimson said in pfBlockerNG-devel v2.2.5_20 PR #610:

    Another question, is it intentional that version 2.2.5_20 no longer uses a lot of the domains listed under

    ---------------------------Third-party advertisers---------------------------!
    ! *** easylist:easylist/easylist_adservers.txt ***

    in the Easylist feeds. Basicly all domains ending with "$third-party" are no longer filtered.

    The previous DNSBL parser was a bit aggressive in the domains that it would parse from the EasyList feeds. I intentionally reduced the parser to lines that start with "||" and end with "^"... The other variations can lead to FPs... other DNSBL Feeds will most likely add any missing domains.

    If I am missing something, please let me know...