Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    https://oisd.nl

    Scheduled Pinned Locked Moved pfBlockerNG
    58 Posts 8 Posters 10.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      totowentsouth @andrebrait
      last edited by totowentsouth

      @andrebrait I updated a pfSense box to 7c3a4eaef2c714c9d97466ec2430e7e867cfd414. I think the extraction of IP addresses in DNSBL is no longer extracting and storing those IPs... This particular pfSense install was using pfblockerng-next -- i.e. before pfblockerng-adblock. FWIW, I uninstalled pfblockerng and removed orphaned files. Then I installed pfblockerng-devel and applied a patch to install 7c3a4. I have yet to try pfblockerng-adblock. In particular, DNSBLIP_v4.txt is absent and original/DNSBL_v4.orig has only one entry 127.1.7.7.
      Here is an example of a list that includes domains and IPv4:
      https://malware-filter.gitlab.io/malware-filter/phishing-filter.txt
      I will do more testing and verification in the next day or so.

      Edit & Update: https://malware-filter.gitlab.io/malware-filter/phishing-filter-agh.txt is their adblock style. After switching to this list, the IPs are extracted. All is well now.

      T 1 Reply Last reply Reply Quote 0
      • T
        totowentsouth @totowentsouth
        last edited by

        This post is deleted!
        T 1 Reply Last reply Reply Quote 0
        • T
          totowentsouth @totowentsouth
          last edited by

          @andrebrait I began a solution for automated test coverage of pfBlockerNG's DNSBL and IP list consolidation. The setup is a little involved and undocumented. I'll flush some documentation for it over the next few days. It is on github at babilon/pfblockerng-tests. I'm now able to trivially run a suite of tests against changes to pfBlockerNG.

          T 1 Reply Last reply Reply Quote 0
          • T
            totowentsouth @totowentsouth
            last edited by

            @andrebrait Functionally, everything appears well. I noticed these duplicate calls to shell functions:

            diff --git a/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc b/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
            index df3dc385c5f2..03e9990d64cd 100644
            --- a/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
            +++ b/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
            @@ -9119,8 +9119,6 @@ function sync_package_pfblockerng($cron='') {
             			// Consolidate all exclusions 
             			exec("{$pfb['script']} dnsbl_py_assemble_exclusions_file unused unused unused {$elog}");
             
            -			exec("{$pfb['script']} dnsbl_py_assemble_redundants_file unused unused unused {$elog}");
            -
             			// Process Whitelists
             			foreach ($postprocess_dnsbl as $header_esc) {
             
            @@ -9139,8 +9137,6 @@ function sync_package_pfblockerng($cron='') {
             				exec("{$pfb['script']} dnsbl_py_remove_redundant {$header_esc} unused unused {$elog}");
             			}
             
            -			exec("{$pfb['script']} dnsbl_py_cleanup_exclusions_file unused unused unused {$elog}");
            -
             			exec("{$pfb['script']} dnsbl_py_cleanup_redundants_file unused unused unused {$elog}");
             		}
             
            -- 
            
            
            A 1 Reply Last reply Reply Quote 0
            • A
              andrebrait @totowentsouth
              last edited by andrebrait

              @totowentsouth the function names are slightly different. One set assembles/removes the master exclusions file and the other assembles/removed the master "might make other entries redundant" file.

              Because EasyLists can also contain exclusions, in order to minimize the processed lists as much as possible, I've added a post-processing step to process all files and remove block entries that would be nullified by exclusions, as well as a step to remove redundant entries (e.g. mail.google.com becomes redundant if a wildcard rule for google.com exists).

              The old logic already did that a bit, but in a different manner.

              Or am I missing what you're referring to?

              T 1 Reply Last reply Reply Quote 0
              • T
                totowentsouth @andrebrait
                last edited by

                @andrebrait 🤦 my bad on the duplication claim. I shoulda tried <shift># and I'd have seen the difference.
                All is well. I retract my previous claims of issues. Sorry for any inconviences.
                I've applied the latest to all my pfSense boxes BTW.

                E 1 Reply Last reply Reply Quote 0
                • E
                  emikaadeo @totowentsouth
                  last edited by

                  Hi @andrebrait
                  just wanted to ask if there's any progress in implementaion of Adblock syntax in pfBlockerNG ?

                  A 1 Reply Last reply Reply Quote 0
                  • A
                    andrebrait @emikaadeo
                    last edited by

                    @emikaadeo yes. I went through some issues in my personal life that affected how much time I could put into it, but I should be able to start working on it again, gradually.

                    The implementation does exist and it does work, but it has not been merged yet

                    E 1 Reply Last reply Reply Quote 1
                    • E
                      emikaadeo @andrebrait
                      last edited by

                      @andrebrait
                      Thank you for answer.
                      I hope you're OK

                      1 Reply Last reply Reply Quote 0
                      • CreationGuyC
                        CreationGuy
                        last edited by

                        What is the best OISD list to use as of right now on the latest non-deval build?

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.