REGEX blocking

[jc1976]

@uglybrian

yes, that's what i'm going for. I dunno if i was very clear at all in my description when i reread it.

to clarify (and you probably understood where i was already going but i'll continue anyway).

That pihole URL has a github list where people can test it and update it as time goes on, just like the lists in pfblockerng.

in pihole, the user merely pastes that url into some place (or where ever... i've never used pihole so i don't know) and pihole automatically goes to that link and imports the contents of the file, and it does so however many times, at whatever hour, is specified.

Is there a way to just 'input' the url to that list which you provided, where it auto updates with a cron event, just like any of the other lists in pfblockerng?

the regex module in pfblocker is such that 'you put a check in the box to activate it, then you go below to the white space and paste a list'.. however you're not pasting a 'link to a list' where pfblocker will go to that url and import the list like it would with the others.

[Uglybrian]

Hi JC, I am not entirely sure you are able to do it the way you would like. I have tried in the past to set up regex on the ip side of pfBlocker ( in the format drop down you can select regex). For the source I put the url for the list. But,the aliases it created had 12 ip address, that never hit for me.
So I went to the DNSBL side and found a Python Regex List. This is where i did a copy and paste of the list. (see pic) After i did the save and update I started to get the hits. I just check the list periodically to see if there any updates.

[Gertjan]

@jc1976 said in REGEX blocking:

Is there a way to just 'input' the url to that list which you provided, where it auto updates with a cron event, just like any of the other lists in pfblockerng?

I guess the regex feature is in it's feature stage : it was added because 'why not'.
regex is very powerful (many falls positives ahead), though. It usage should be limited - not many lines.

[Cool_Corona]

So in short, this is very easy to block specific domains this way?

[Gertjan]

@cool_corona

yes.

[Cool_Corona]

@gertjan Why is there no package for this to translate the regex language??

Like type in facebook.com -> Regex translated and voila..... Then you dont have to worry about ASN and ip address....

[Gertjan]

@cool_corona said in REGEX blocking:

to translate the regex language??

Translate what to what ?
If you're new to regex, never used command line utilities like 'grep', 'awk' and 'sed', then I understand ... that a whole new world just opened to you ;)

I said above, IMHO, the pfblockerng-devel - python - regex functionality was added because it needed one function (line) and it is soooooooooooo powerfull ***:

def pfb_regex_match(q_name):
    global regexDB

    if q_name:
        for k,r in regexDB.items():
            if r.search(q_name):
                return k
    return False

where the q_name is the 'hostname' to be macthed.
regexDB.items(): is the list with regular expressions. You can find that list here : /var/unbound/pfb_unbound.ini (bottom part of the file, the [REGEX] section).

**** there will be a lot of 'shot in de foot' situations, as real power needs to be managed.

[Cool_Corona]

@gertjan Yes and thats why a GUI would be great for the noobs like me

Should I use

def pfb_regex_match(**facebook.com**):
    global regexDB

    if q_name:
        for k,r in regexDB.items():
            if r.search(q_name):
                return k
    return False

And then facebook is blocked?

[Gertjan]

@cool_corona said in REGEX blocking:

And then facebook is blocked?

Noop.
you should add "facebook.com" to the regex list.

An facebook example is present in the list @Uglybrian has shown.

[Cool_Corona]

@gertjan Thanks :)

I cant find Regex under DNSBL tab despite using unbound python mode....

[Gertjan]

@cool_corona

and then :

and then the list get available :

[Cool_Corona]

@gertjan Got it Thanks man!

[Cool_Corona]

@gertjan Sometimes it works...sometime it doesnt.

Kind of annoying....

^ad([sxv]?[0-9]*|system)[_.-]([^.[:space:]]+\.){1,}|[_.-]ad([sxv]?[0-9]*|system)[_.-]
^(.+[_.-])?adse?rv(er?|ice)?s?[0-9]*[_.-]
^(.+[_.-])?telemetry[_.-]
^adim(age|g)s?[0-9]*[_.-]
^adtrack(er|ing)?[0-9]*[_.-]
^advert(s|is(ing|ements?))?[0-9]*[_.-]
^aff(iliat(es?|ion))?[_.-]
^analytics?[_.-]
^banners?[_.-]
^beacons?[0-9]*[_.-]
^count(ers?)?[0-9]*[_.-]
^mads\.
^pixels?[-.]
^stat(s|istics)?[0-9]*[_.-]
# ^(.+[_.-])?(facebook|fb(cdn|sbx)?|tfbnw)\.[^.]+$

Facebook block only works sometimes and it makes DNS horribly slow. (its intentional that I commented it out in the current config).

[Gertjan]

@cool_corona

I'm not sure you can comment it out with a #. I advise you to remove it.
This is regex, not some script ;)

Btw :
See line 1451 in /var/unbound/pfb_unbound.py
as long as there are items on the regex list, the pfb_regex_match(q_name) function is called.
The main advantage of regex is : is 'blindingly' fast (way faster as what PHP could have done, python is far superior). When I activated that rule, my browser instantaneously showed

when I entered www.facebook.com in the URL bar.

The DNSBL Python logs :

Regex blocking is as fast as it can gets, way faster as the parsing needed for all the main DNSBL list, compiled from all your DNBL feeds.

[Gertjan]

WTF ....

I added a 'facebook' regex above so I could collect some DNSBL log lines - see image above.

I also saw :

and, as 'facebook' is in the hostname, all looks fine.

I removed the facebook regex, and reloaded pfBlockerng.

Still, Whatsapp didn't work (on my phone).
And wtf, when I deactivated wifi on my phone, still Whatsapp "doesn't work".

Facebook did it again : they have managed to shut themselves, whatsapp this time, out of the Internet. Probably on a planetary level

[Cool_Corona]

@gertjan Thats exactly my issue.... that other apps are affected because they are all mining on each others data og dns records.

Kind of annoying...

[Cool_Corona]

@gertjan https://www.independent.co.uk/tech/whatsapp-down-not-working-status-b2209835.html?utm_content=Echobox&utm_medium=Social&utm_campaign=Main&utm_source=Twitter#Echobox=1666682232

[Gertjan]

@cool_corona Twitter ?
Also on CNN, MSNBC, Foxnews, BBC, all major (local) french tv news, and soon many Youtube "facebook did it again" videos.

[jc1976]

@gertjan so i guess a really slick feature to pfblockerng-devel would be the ability to add regex lists from pihole, like how suricata can use snort lists.

[pfsjap]

At the bottom of Python Regex List section reads "Changes to this option will require a Force Update to take effect" but actually Reload is required.