Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    REGEX blocking

    Scheduled Pinned Locked Moved pfBlockerNG
    48 Posts 8 Posters 8.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • U
      Uglybrian
      last edited by

      Hi JC, I am not entirely sure you are able to do it the way you would like. I have tried in the past to set up regex on the ip side of pfBlocker ( in the format drop down you can select regex). For the source I put the url for the list. But,the aliases it created had 12 ip address, that never hit for me.
      So I went to the DNSBL side and found a Python Regex List. This is where i did a copy and paste of the list. (see pic) After i did the save and update I started to get the hits. I just check the list periodically to see if there any updates.regex list from 2022-10-20 12-03-45.png

      1 Reply Last reply Reply Quote 0
      • GertjanG
        Gertjan @jc1976
        last edited by

        @jc1976 said in REGEX blocking:

        Is there a way to just 'input' the url to that list which you provided, where it auto updates with a cron event, just like any of the other lists in pfblockerng?

        I guess the regex feature is in it's feature stage : it was added because 'why not'.
        regex is very powerful (many falls positives ahead), though. It usage should be limited - not many lines.

        No "help me" PM's please. Use the forum, the community will thank you.
        Edit : and where are the logs ??

        1 Reply Last reply Reply Quote 0
        • Cool_CoronaC
          Cool_Corona
          last edited by

          So in short, this is very easy to block specific domains this way?

          GertjanG 1 Reply Last reply Reply Quote 0
          • GertjanG
            Gertjan @Cool_Corona
            last edited by

            @cool_corona

            yes.

            No "help me" PM's please. Use the forum, the community will thank you.
            Edit : and where are the logs ??

            Cool_CoronaC 1 Reply Last reply Reply Quote 0
            • Cool_CoronaC
              Cool_Corona @Gertjan
              last edited by

              @gertjan Why is there no package for this to translate the regex language??

              Like type in facebook.com -> Regex translated and voila..... Then you dont have to worry about ASN and ip address....

              GertjanG 1 Reply Last reply Reply Quote 0
              • GertjanG
                Gertjan @Cool_Corona
                last edited by

                @cool_corona said in REGEX blocking:

                to translate the regex language??

                Translate what to what ?
                If you're new to regex, never used command line utilities like 'grep', 'awk' and 'sed', then I understand ... that a whole new world just opened to you ;)

                I said above, IMHO, the pfblockerng-devel - python - regex functionality was added because it needed one function (line) and it is soooooooooooo powerfull ***:

                def pfb_regex_match(q_name):
                    global regexDB
                
                    if q_name:
                        for k,r in regexDB.items():
                            if r.search(q_name):
                                return k
                    return False
                

                where the q_name is the 'hostname' to be macthed.
                regexDB.items(): is the list with regular expressions. You can find that list here : /var/unbound/pfb_unbound.ini (bottom part of the file, the [REGEX] section).

                **** there will be a lot of 'shot in de foot' situations, as real power needs to be managed.

                No "help me" PM's please. Use the forum, the community will thank you.
                Edit : and where are the logs ??

                Cool_CoronaC 1 Reply Last reply Reply Quote 0
                • Cool_CoronaC
                  Cool_Corona @Gertjan
                  last edited by Cool_Corona

                  @gertjan Yes and thats why a GUI would be great for the noobs like me 😆

                  Should I use

                  def pfb_regex_match(**facebook.com**):
                      global regexDB
                  
                      if q_name:
                          for k,r in regexDB.items():
                              if r.search(q_name):
                                  return k
                      return False
                  
                  

                  And then facebook is blocked?

                  GertjanG 1 Reply Last reply Reply Quote 0
                  • GertjanG
                    Gertjan @Cool_Corona
                    last edited by

                    @cool_corona said in REGEX blocking:

                    And then facebook is blocked?

                    Noop.
                    you should add "facebook.com" to the regex list.

                    An facebook example is present in the list @Uglybrian has shown.

                    No "help me" PM's please. Use the forum, the community will thank you.
                    Edit : and where are the logs ??

                    Cool_CoronaC 1 Reply Last reply Reply Quote 0
                    • Cool_CoronaC
                      Cool_Corona @Gertjan
                      last edited by

                      @gertjan Thanks :)

                      I cant find Regex under DNSBL tab despite using unbound python mode....

                      GertjanG 1 Reply Last reply Reply Quote 0
                      • GertjanG
                        Gertjan @Cool_Corona
                        last edited by

                        @cool_corona

                        41a5cc5f-667f-4fe5-85b5-61661149df1f-image.png

                        and then :

                        bfdd633c-6255-4d37-949d-e20ea91e2c9e-image.png

                        and then the list get available :

                        b82ff1ed-2601-426e-99ba-da94a786504f-image.png

                        No "help me" PM's please. Use the forum, the community will thank you.
                        Edit : and where are the logs ??

                        Cool_CoronaC 2 Replies Last reply Reply Quote 0
                        • Cool_CoronaC
                          Cool_Corona @Gertjan
                          last edited by

                          @gertjan Got it 👍 Thanks man!

                          1 Reply Last reply Reply Quote 0
                          • Cool_CoronaC
                            Cool_Corona @Gertjan
                            last edited by Cool_Corona

                            @gertjan Sometimes it works...sometime it doesnt.

                            Kind of annoying....

                            
                            ^ad([sxv]?[0-9]*|system)[_.-]([^.[:space:]]+\.){1,}|[_.-]ad([sxv]?[0-9]*|system)[_.-]
                            ^(.+[_.-])?adse?rv(er?|ice)?s?[0-9]*[_.-]
                            ^(.+[_.-])?telemetry[_.-]
                            ^adim(age|g)s?[0-9]*[_.-]
                            ^adtrack(er|ing)?[0-9]*[_.-]
                            ^advert(s|is(ing|ements?))?[0-9]*[_.-]
                            ^aff(iliat(es?|ion))?[_.-]
                            ^analytics?[_.-]
                            ^banners?[_.-]
                            ^beacons?[0-9]*[_.-]
                            ^count(ers?)?[0-9]*[_.-]
                            ^mads\.
                            ^pixels?[-.]
                            ^stat(s|istics)?[0-9]*[_.-]
                            # ^(.+[_.-])?(facebook|fb(cdn|sbx)?|tfbnw)\.[^.]+$
                            

                            Facebook block only works sometimes and it makes DNS horribly slow. (its intentional that I commented it out in the current config).

                            GertjanG 1 Reply Last reply Reply Quote 0
                            • GertjanG
                              Gertjan @Cool_Corona
                              last edited by

                              @cool_corona

                              I'm not sure you can comment it out with a #. I advise you to remove it.
                              This is regex, not some script ;)

                              Btw :
                              See line 1451 in /var/unbound/pfb_unbound.py
                              as long as there are items on the regex list, the pfb_regex_match(q_name) function is called.
                              The main advantage of regex is : is 'blindingly' fast (way faster as what PHP could have done, python is far superior). When I activated that rule, my browser instantaneously showed

                              9b3ccba4-ad33-4d29-9ef0-6f077a8277bd-image.png

                              when I entered www.facebook.com in the URL bar.

                              The DNSBL Python logs :

                              8127dec6-8456-47d0-a98a-6b911135a931-image.png

                              Regex blocking is as fast as it can gets, way faster as the parsing needed for all the main DNSBL list, compiled from all your DNBL feeds.

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              GertjanG 1 Reply Last reply Reply Quote 0
                              • GertjanG
                                Gertjan @Gertjan
                                last edited by

                                WTF ....

                                I added a 'facebook' regex above so I could collect some DNSBL log lines - see image above.

                                I also saw :

                                2611c22e-aae9-4881-a83b-96b3da600c98-image.png

                                and, as 'facebook' is in the hostname, all looks fine.

                                I removed the facebook regex, and reloaded pfBlockerng.

                                Still, Whatsapp didn't work (on my phone).
                                And wtf, when I deactivated wifi on my phone, still Whatsapp "doesn't work".

                                Facebook did it again : they have managed to shut themselves, whatsapp this time, out of the Internet. Probably on a planetary level 👍

                                No "help me" PM's please. Use the forum, the community will thank you.
                                Edit : and where are the logs ??

                                Cool_CoronaC A 3 Replies Last reply Reply Quote 0
                                • Cool_CoronaC
                                  Cool_Corona @Gertjan
                                  last edited by

                                  @gertjan Thats exactly my issue.... that other apps are affected because they are all mining on each others data og dns records.

                                  Kind of annoying...

                                  1 Reply Last reply Reply Quote 0
                                  • Cool_CoronaC
                                    Cool_Corona @Gertjan
                                    last edited by

                                    @gertjan https://www.independent.co.uk/tech/whatsapp-down-not-working-status-b2209835.html?utm_content=Echobox&utm_medium=Social&utm_campaign=Main&utm_source=Twitter#Echobox=1666682232

                                    GertjanG 1 Reply Last reply Reply Quote 0
                                    • GertjanG
                                      Gertjan @Cool_Corona
                                      last edited by

                                      @cool_corona Twitter ?
                                      Also on CNN, MSNBC, Foxnews, BBC, all major (local) french tv news, and soon many Youtube "facebook did it again" videos.

                                      No "help me" PM's please. Use the forum, the community will thank you.
                                      Edit : and where are the logs ??

                                      J 1 Reply Last reply Reply Quote 0
                                      • J
                                        jc1976 @Gertjan
                                        last edited by

                                        @gertjan so i guess a really slick feature to pfblockerng-devel would be the ability to add regex lists from pihole, like how suricata can use snort lists.

                                        1 Reply Last reply Reply Quote 0
                                        • P
                                          pfsjap
                                          last edited by

                                          At the bottom of Python Regex List section reads "Changes to this option will require a Force Update to take effect" but actually Reload is required.

                                          1 Reply Last reply Reply Quote 1
                                          • F
                                            Forumis
                                            last edited by Forumis

                                            This post is deleted!
                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.