pfBlocker locked me out of my pfSense web-managment !!!

Gertjan

@johnpoz said in pfBlocker locked me out of my pfSense web-managment !!!:

Maybe there are?

Yep. It has been done : IP feeds that suddenly included RFC1918 space => surprise !!
So now the bullet is out of the box, in the chamber. And the admin decided that it is 'gun cleaning time', completely non-respecting the cleaning rules (like : disarm gun first).

The thing is : these :

are created and maintained by people. And people can make mistakes.
So, if a RFC1918 could make it on the list, the impossible can becomes possible.
I'm not sure if pfBlockerng checks every list before it starts to use it, but what I do know : if there are to many, pfBlockerng will activate the list "as is" without parsing/checking any futher. See for yourself : select a maximum of IP feeds => select them all.
Now, go to Firewall > pfBlockerNG > Update and do a force reload. Look at the logs produced, and you'll see what happens. If there are to many, there will be a (not very clear, I agree) message that IPs are copied 'as is'. ( Btw : Even if your pfSense has 128 Gbytes of RAM, it will probably 'explode'/''fail' anyway.)
Now the gun hammer is set.
The admin removed the anti lockout rule as : "less rules is better". Now the security is off.

Happy gun cleaning time ^^

Actually, before using a DNSBL or IP feeds, you should download and open the file in your favorite text editor and read (check) it .... as you can't be sure of the quality of the file. We all presume they are fine ...

JeGr

@Gertjan said in pfBlocker locked me out of my pfSense web-managment !!!:

So, if a RFC1918 could make it on the list, the impossible can becomes possible.
I'm not sure if pfBlockerng checks every list before it starts to use it, but what I do know : if there are to many, pfBlockerng will activate the list "as is" without parsing/checking any futher. See for yourself : select a maximum of IP feeds => select them all.

Nope, that's only happening without suppression and that is enabled by default. Suppresion should take care of that, RFC1918 space is normally removed from lists for exactly that reason. Works e.g. for firehol1 - you can check with that as it includes RFC1918 space and those get removed after downloading in post processing.

Suppression: Default enabled. This will prevent Selected IPs (and RFC1918/Loopback addresses) from being blocked. Only for IPv4 lists (/32 and /24).

Also couldn't reproduce the "too many lists" ad hoc, but may be limited by the state table entries in Sys/Advanced options?

johnpoz

So this guy didn't just unlock his gun box and find some bullets in the nightstand draw, he actually drove to an ATM 30 minutes away, then drove to gun shop, 45 minutes in the opposite direction and bought bullets. Showed 2 different forms of ID and then waited for his background check to clear.

And then went home and loaded the gun and turned off the safety and pointed it as his foot..

Gertjan

@JeGr said in pfBlocker locked me out of my pfSense web-managment !!!:

Nope, that's only happening without suppression and that is enabled by default. Suppresion should take care of that, RFC1918 space is normally removed from lists for exactly that reason. Works e.g. for firehol1 - you can check with that as it includes RFC1918 space and those get removed after downloading in post processing.

Suppression: Default enabled. This will prevent Selected IPs (and RFC1918/Loopback addresses) from being blocked. Only for IPv4 lists (/32 and /24).

Your last " (/32 and /24)" disproofs you 'Nope' ^^
IPv6 is a thing these days.

But I had a look. It's this part that I was talking about :

This fragment shows the process of the assembly of the main DNSBL file. The dots marked in green == one dot for every 10k or 100k DNSBL lines.
If you pick to many DNSBL, the main DNSBL file get to big (PHP memory is finite) and the filter/sort/whitelist proces aborts and the DNSBL files are copied in without to much checking.
Anyway : this is true for DNSBL ...

I knew that RFC1918 is now actively searched for, and removed.
I'll try to dig up that post from the past where someone found RFC1918 in a IP feed - it happened, I'm sure of it.

@JeGr said in pfBlocker locked me out of my pfSense web-managment !!!:

Also couldn't reproduce the "too many lists" ad hoc

Wait ... you can select and use them all ?? Doesn't that boil down to the entire 0.0.0.0/0 (the entire Internet ^^)
You've tried IPv6 feeds?
Also for DNSBL ??

Gertjan

@johnpoz
Yeah, looks strange when you align up all the events like that. That can't be true.
But not everything probably happened at the same moment.
Not all events are needed. The "find a gun" up until "check to clear" isn't needed in Switzerland (Swiss), everybody has already everything at home. And not the big-dig US stuff, but army stuff, as part of their military-readyness.
And a lot is also often based on the "saw that on the Internet, and I trust that guy".

And we'll all agree, I guess, that pfBlockerng really makes it look like a overly complicated thing can be used and suddenly it looks rather simple.
There are moments that I tend to think that pfBlockerng doesn't even belong on a firewall, but more on a dedicated device like a pi-hole - you're using such a device, right ?
On the other hand, pfBlockerng by itself does not much.
It (can) prepares firewall lists loaded with aliases.
It (can) prepares lists with DNSBL that will be used when the resolver does its resolver thing.
It makes some nice stats.
That's it.

johnpoz

@Gertjan Yeah I run pihole - the eye candy is prettier to be honest ;) Just run it on a pi had laying around.. I have a few of them - I am easy mark for cheap hardware that can run linux on it ;) And do fun shit with, I have another as ntp server, simple gps hat you plug in.

I do use pfblocker quite a bit, but only for management of native aliases that I use in my own rules. Its a great tool for sure.. Many years ago I had asked if maybe a lite version where it just did alias management, no auto rules this or all those feeds, etc..

pfblocker has an amazing amount features - but sure it might be a bit more of a learning curve than say pihole for your typical user. You can run it on a pi, some vm or even just a docker container.. And guess pfblocker is easier to shoot your foot with than just something filtering dns only.. that you can look to see what exactly was blocked and what wasn't..

Also a fan of unfiltered dns (if I want it) just simple point my client to pfsense IP directly vs pihole IP, or just do a direct dig to pfsense IP, etc.

JeGr

@Gertjan said in pfBlocker locked me out of my pfSense web-managment !!!:

Your last " (/32 and /24)" disproofs you 'Nope' ^^
IPv6 is a thing these days.

But I had a look. It's this part that I was talking about :

Don't know what exactly you are talking about but that's mixing a lot of stuff that don't match together - I think? Or I'm not really understanding where you're going with that line of thought :)

The "/32 and /24" disproofs nothing. The suppression toggle is default on. So you'd have to actively disable the option. With that on, RFC1918 networks are always filtered out of IP lists. Tested it and couldn't disproof it without actively disabling the suppresion option. That you can add further IPs with /32 or /24 (like adding IPs from CDNs and stuff to be removed from lists proactively) is no disproof of the fact, that it says exactly that in its description: it filters out private addresses so you don't lock yourself out.
Yes IPv6 is a thing these days but IPv6 has no dedicated private IPs like RFC1918. FE80 and stuff isn't on blocklists. FC00/FD00 is mostly random and shouldn't be on lists, too, as it isn't routable. So how has that anything to do with the hypothesis, that pfBNG could lock you out with adding LAN IPs? Even if fd00:: prefixes would be on a list, your box normally would've an fe80:: one, too. And potentially GUAs. So multiple IP6 that would be available.
DNSBL again I see nothing related with IP lists. I don't know what you were trying to say with the TLD analysis stuff etc. - but again I don't see how that would lock you out from your LAN? Perhaps I don't see where you're going with that thought

@Gertjan said in pfBlocker locked me out of my pfSense web-managment !!!:

I knew that RFC1918 is now actively searched for, and removed.
I'll try to dig up that post from the past where someone found RFC1918 in a IP feed - it happened, I'm sure of it.

Yeah, by said suppression option! And one can simply reproduce it very easily. Just use firehol1 as a list. Disable suppression. Save and let it update. Check with the Tables and look into it, you'll find 10/8, 172.16/12 and 192.168/16 in it. Now enable suppresion again, force reload of every list so it'll recalculate -> poof RFC1918 IPs are gone.

@Gertjan said in pfBlocker locked me out of my pfSense web-managment !!!:

Wait ... you can select and use them all ?? Doesn't that boil down to the entire 0.0.0.0/0 (the entire Internet ^^)

Why should it be the whole internet?
I have test VMs running, that have 4GB of RAM for exactly the reason to test out, what IP lists or DNSBL lists one can enable and get away with it. Adding almost every DNSBL list was possible but hard on the RAM. IP lists are easy compared with that. pf can manage IP tables much better then unbound does DNSBL domains although when using python mode it gets comparably better. When adding Shalla or UF1, that went out of the window and crashed hard (out of memory) but the normal "small'ish" lists that are provided as a starting point? Could add most of them without problems on a naked test machine.

Cheers

JeGr

@Gertjan said in pfBlocker locked me out of my pfSense web-managment !!!:

And we'll all agree, I guess, that pfBlockerng really makes it look like a overly complicated thing can be used and suddenly it looks rather simple.

Yeah, also the UX isn't perfectly clear on a few things and the handling of HA setups - I made an extra topic for that. It's not nice.

There are moments that I tend to think that pfBlockerng doesn't even belong on a firewall, but more on a dedicated device like a pi-hole - you're using such a device, right ?

I'd disagree on the IP part. THAT one - IMHO - belongs where it is. And it's far more powerful then the integrated Alias Handling / IP list handling of naked pf.
For the DNS part I agree. I'm seeing the usecase for small scenarios where you'd like to have everything in one place. But anything a tad more complex I'd go with a pihole or adguard setup for DNS to get more flexibility.

On the other hand, pfBlockerng by itself does not much.

That's not really true. Sure, you don't see much of it but it actually DOES more then that behind the curtains:

It (can) prepares firewall lists loaded with aliases.

That's already too far. It fetches IP and DNS lists first. And that by itself is far more sophisticated and extending the normal capabilities of the Alias system, that it's undervalued. So I'm stating again: it does MUCH more then "just fetch this list".

Just as an example: default IP tables for pf can just deal with CIDR formatted lists, one CIDR per line.
pfBlocker though? You can grab IP lists from that but also extract them out of websites (when needed), feed it JSON lists, mixed lists with IPs/DNS names (for those situations where service providers only give you a stupid mixed list you can't feed into systems) or even fetch IPs/IP Blocks from ASNs or other files. Just because it isn't that visible normally, it IS quite powerful. There were more then one situations where we were glad to have JSON parsing or ASN lists available so we could filter out specific blocks.
Also GeoIP.

It (can) prepares lists with DNSBL that will be used when the resolver does its resolver thing.

Jup, but it can do so in many more options, like block whole TLDs, Alias and CNAME blocking etc. - it's not just "fetch list, prepare list, block it via unbound". The processing and options you can run are quite a thing even when some of them are implemented a bit rough.

It makes some nice stats.

Jep. But it also links the reports/stats to the corresponding list. That in itself is also very nice, as it gives you a pointer WHICH list has that IP you're looking for and you don't have to go to the console or other ways to look through all the lists you downloaded and check if an IP/DNS name is in there to allow it or suppress it.
That's again one of the things we just "shrug off" (yeah sure there it goes) but especially if you merge, suppress, compress/dedup lists etc. it is great to have the visibility to see where it originally came from.

But I get it, that's one of those things behind the scenes and not that flashy and visible so it doesn't pop up front. But when you compare it to the fact that you can't simply "search" IP alias tables for IPs that simple, pointing you the right way for example is quite nice :)

That's it.

I'd say "that's it for the UX that users see". Technically I see far more then those 3 points.

Just to point out an example: you can pre- and post-process lists so you can tweak them post-download before or after they are converted for internal use. pfBNG has an example of this with the AWS pre processing - e.g. loading a JSON style list, using the pre-processing to using a specific JSON query string so only IPs from a specific region are included in your alias e.g. selecting US WEST from the whole AWS list. You can do that to other things, too. Or use WHOIS/ASN Lookups to populate IP lists. Just because those advanced options are typically folded and not shown doesn't mean they aren't there :)

I'm more in agreement when it comes to DNSBL - that feature set I can easily see being cloned by Pihole or Adguard. But the IP stuff is way more useful than many realize.

Cheers :)

michmoor

@johnpoz said in pfBlocker locked me out of my pfSense web-managment !!!:

I do use pfblocker quite a bit, but only for management of native aliases that I use in my own rules. Its a great tool for sure.. Many years ago I had asked if maybe a lite version where it just did alias management, no auto rules this or all those feeds, etc..

curious man, what are you blocking or need an alias for? Looking for ideas.

michmoor

@JeGr said in pfBlocker locked me out of my pfSense web-managment !!!:

I'm more in agreement when it comes to DNSBL - that feature set I can easily see being cloned by Pihole or Adguard. But the IP stuff is way more useful than many realize.

although i generally agree, having pfblocker be granular enough to dnbl based on networks is way more useful than the way its implemented today. Deploy pfsense in a SOHO or school that doesn't want to purchase a separate DNS server but wants filtering, you need granularity which pfblocker doesn't support. So that would be a use case for the extensibility.