Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    https://oisd.nl

    Scheduled Pinned Locked Moved pfBlockerNG
    58 Posts 8 Posters 10.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      totowentsouth @andrebrait
      last edited by

      @andrebrait Yes, the EasyPrivacy URL https://easylist.to/easylist/easyprivacy.txt in the pfBlockerNG feeds tab is the same. I create groups and provide the URLs in lieu of using the feeds tab.

      A 1 Reply Last reply Reply Quote 0
      • A
        andrebrait @totowentsouth
        last edited by

        @totowentsouth I split the check to determine whether it's an EasyList and the parsing. Now there's a first pass through the file for checking for the EasyList headers and entries before moving on to the actual parsing (which I also refined).

        I checked and the offending entries are not ending up in the file anymore. Let me know if you can reproduce the fix.

        T 1 Reply Last reply Reply Quote 0
        • T
          totowentsouth @andrebrait
          last edited by

          @andrebrait I updated my patch to include 4da5a631ae8d82a109fa7880429eff63c4cfa46f and all is well when using the EasyPrivacy list. Thanks!

          A 1 Reply Last reply Reply Quote 1
          • A
            andrebrait @totowentsouth
            last edited by

            @totowentsouth I gave it some polishing, cleaned up the commit history and produced the pfblockerng-adblock-clean branch (now on 7c3a4eaef2c714c9d97466ec2430e7e867cfd414) .
            Could you give it a last go so I have someone else test it?

            T 1 Reply Last reply Reply Quote 0
            • T
              totowentsouth @andrebrait
              last edited by totowentsouth

              @andrebrait I updated a pfSense box to 7c3a4eaef2c714c9d97466ec2430e7e867cfd414. I think the extraction of IP addresses in DNSBL is no longer extracting and storing those IPs... This particular pfSense install was using pfblockerng-next -- i.e. before pfblockerng-adblock. FWIW, I uninstalled pfblockerng and removed orphaned files. Then I installed pfblockerng-devel and applied a patch to install 7c3a4. I have yet to try pfblockerng-adblock. In particular, DNSBLIP_v4.txt is absent and original/DNSBL_v4.orig has only one entry 127.1.7.7.
              Here is an example of a list that includes domains and IPv4:
              https://malware-filter.gitlab.io/malware-filter/phishing-filter.txt
              I will do more testing and verification in the next day or so.

              Edit & Update: https://malware-filter.gitlab.io/malware-filter/phishing-filter-agh.txt is their adblock style. After switching to this list, the IPs are extracted. All is well now.

              T 1 Reply Last reply Reply Quote 0
              • T
                totowentsouth @totowentsouth
                last edited by

                This post is deleted!
                T 1 Reply Last reply Reply Quote 0
                • T
                  totowentsouth @totowentsouth
                  last edited by

                  @andrebrait I began a solution for automated test coverage of pfBlockerNG's DNSBL and IP list consolidation. The setup is a little involved and undocumented. I'll flush some documentation for it over the next few days. It is on github at babilon/pfblockerng-tests. I'm now able to trivially run a suite of tests against changes to pfBlockerNG.

                  T 1 Reply Last reply Reply Quote 0
                  • T
                    totowentsouth @totowentsouth
                    last edited by

                    @andrebrait Functionally, everything appears well. I noticed these duplicate calls to shell functions:

                    diff --git a/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc b/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
                    index df3dc385c5f2..03e9990d64cd 100644
                    --- a/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
                    +++ b/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc
                    @@ -9119,8 +9119,6 @@ function sync_package_pfblockerng($cron='') {
                     			// Consolidate all exclusions 
                     			exec("{$pfb['script']} dnsbl_py_assemble_exclusions_file unused unused unused {$elog}");
                     
                    -			exec("{$pfb['script']} dnsbl_py_assemble_redundants_file unused unused unused {$elog}");
                    -
                     			// Process Whitelists
                     			foreach ($postprocess_dnsbl as $header_esc) {
                     
                    @@ -9139,8 +9137,6 @@ function sync_package_pfblockerng($cron='') {
                     				exec("{$pfb['script']} dnsbl_py_remove_redundant {$header_esc} unused unused {$elog}");
                     			}
                     
                    -			exec("{$pfb['script']} dnsbl_py_cleanup_exclusions_file unused unused unused {$elog}");
                    -
                     			exec("{$pfb['script']} dnsbl_py_cleanup_redundants_file unused unused unused {$elog}");
                     		}
                     
                    -- 
                    
                    
                    A 1 Reply Last reply Reply Quote 0
                    • A
                      andrebrait @totowentsouth
                      last edited by andrebrait

                      @totowentsouth the function names are slightly different. One set assembles/removes the master exclusions file and the other assembles/removed the master "might make other entries redundant" file.

                      Because EasyLists can also contain exclusions, in order to minimize the processed lists as much as possible, I've added a post-processing step to process all files and remove block entries that would be nullified by exclusions, as well as a step to remove redundant entries (e.g. mail.google.com becomes redundant if a wildcard rule for google.com exists).

                      The old logic already did that a bit, but in a different manner.

                      Or am I missing what you're referring to?

                      T 1 Reply Last reply Reply Quote 0
                      • T
                        totowentsouth @andrebrait
                        last edited by

                        @andrebrait 🤦 my bad on the duplication claim. I shoulda tried <shift># and I'd have seen the difference.
                        All is well. I retract my previous claims of issues. Sorry for any inconviences.
                        I've applied the latest to all my pfSense boxes BTW.

                        E 1 Reply Last reply Reply Quote 0
                        • E
                          emikaadeo @totowentsouth
                          last edited by

                          Hi @andrebrait
                          just wanted to ask if there's any progress in implementaion of Adblock syntax in pfBlockerNG ?

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            andrebrait @emikaadeo
                            last edited by

                            @emikaadeo yes. I went through some issues in my personal life that affected how much time I could put into it, but I should be able to start working on it again, gradually.

                            The implementation does exist and it does work, but it has not been merged yet

                            E 1 Reply Last reply Reply Quote 1
                            • E
                              emikaadeo @andrebrait
                              last edited by

                              @andrebrait
                              Thank you for answer.
                              I hope you're OK

                              1 Reply Last reply Reply Quote 0
                              • CreationGuyC
                                CreationGuy
                                last edited by

                                What is the best OISD list to use as of right now on the latest non-deval build?

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.