pfBlockerNG not blocking ADs

wc2l · Dec 27, 2024, 5:11 PM

I'm still new to this and want to make sure I understand how this works and what I'm doing wrong. I'm running pfSense 24.11. I've got pfBlockerNG 3.2.0_16 installed. At one point, I did have 3.2.1_20 (or similar) installed. Netgate support had me go back to current installed.

It appears a few things have changed.
ADs are coming through.
I also have this error:
[ pfB_PRI3_v4 - MaxMind_BD_Proxy_v4 ] Download FAIL
DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.
The Following List has been REMOVED [ MaxMind_BD_Proxy_v4 ]

How do I stop the ADs and what packages should change to or add? I just want to make things a bit better!!

Gertjan · Jan 7, 2025, 10:42 PM

@wc2l said in pfBlockerNG not blocking ADs:

It appears a few things have changed.

That's very vague. Can you give details ?

@wc2l said in pfBlockerNG not blocking ADs:

ADs are coming through

What ? Just don't go back to that site and issue solved.
.. ok, sorry.
Couldn't this be explained by the simple fact that you visited a web site, and it has a new contract with another market content manager (the one that filled your pages with publicity) and this new content managers uses IPs that are not known to anybody, so can't be listed by any pfBlockerng ?

Do you know what pfBlockerng does ?
It gets files from 'places', and these files are filled with lines like this :

Who put these (can you see them ?) host names in this file ? The people that 'work for' Stevenblack DNSBL AD list. And this could be you, me, and who ever wants to signal a host name to somebody so they get listed on some so called DNSBL feed like "Steven ADs" list.

These host name (URL) lists are made available to the pfSense resolver.
Let's get an example : You see www.marrketstrategy.com ?
Let's test :

PS C:\Users\Gauche> nslookup www.marrketstrategy.com
Serveur :   pfSense.bhf.tld.net
Address:  2a01:cb19:907:dead:beef:77ff:fe29:392c

Réponse ne faisant pas autorité :
Nom :    www.marrketstrategy.com
Address:  0.0.0.0

The answer is 0.0.0.0 so the browser now knows : don't even bother asking IP 0.0.0.0 as 0.0.0.0 is a 'know known' address.
You see ? No rocket science, this is how it add servers are blocked : an URL is found on a web page, so your browser goes out looking for it The browser can't work host names, so it will have it resolved (== DNS) first.
And resolving happens on pfSense, which, as we saw, produces a nice 0.0.0.0 as the ad server domain name was listed on a DNSBL you put into pfBlockerng (who put it into the resolver).

Now for IP lists, which looks like (part of the BBCaN77 IP list) this :

isn't really different.
These are IP address, so they all together (with other IP lists) stashed into one big firewall alias, and then used as a floating rule, so my local network devices can't connect to these IPs anymore.

Example : see the list, and find the first IP 91.121.162.48.
When I use that IP in browser (so I will use port 443 on that IP), my browser will error out.
I could also try Telnet, SSH, POP, IMAP, etc etc etc ports. They will all time out, as I cant' reach this 91.121.162.48 anymore.
The only action I can see is this :

As you can see, the hit counter go up, as I was trying to access (== outbound direction) this IP. So me trying to access that IP, my traffic never even reached the pfSense WAN interface.

wc2l · Jan 8, 2025, 6:52 AM

@Gertjan
Sorry for the slow response. I have been under the weather. Today is the first night back on the network with a clean mind. I'm hoping to spend few hours tonight.

I'm still learning about what pfBlockerNG is doing. Some things make sense and of course some is lack of knowledge!! Slowly making progress.

So what used to happen, email messages that had links to try, test, etc. would fail and say not allowed. Then if I went to certain webpages, it would say that I need to shut off my adblockers to get to the site. Since I have reloaded, neither happens anymore.

One of my failed downloads is this:
https://www.maxmind.com/en/high-risk-ip-sample-list
If I go to https://talosintelligence.com/documents/ip-blacklist
It prompts me to accept. When I try another browser, I start over.
Is there a way to get the pfBlockerNG to get accept and get the list? My guess that is what is going on.

The ADs coming into the web pages are
.googleadservices.com. or .https://doubleclick.net. as examples.

A lot of what I'm seeing for examples are old versions with very bad or changed settings. Some may have moved, but I'm trying.

Gertjan · Jan 8, 2025, 6:52 AM

@wc2l said in pfBlockerNG not blocking ADs:

One of my failed downloads is this:
https://www.maxmind.com/en/high-risk-ip-sample-list
If I go to https://talosintelligence.com/documents/ip-blacklist
It prompts me to accept. When I try another browser, I start over.
Is there a way to get the pfBlockerNG to get accept and get the list? My guess that is what is going on.

These two URLs are not the ones pfBlockerng uses.
To download lists from maxmind.com, you need a subscription first, and have to obtain login credentials.
Then you set pfBlockerng up like this :

I can't see the URLS used by pfBlockerng to access pages like this :

but I know pfBlockerng uses my access codes to get them.

@wc2l said in pfBlockerNG not blocking ADs:

The ADs coming into the web pages are
.googleadservices.com. or .https://doubleclick.net. as examples.

"googleadservices.com" is world's best known add server blocked by nearly every DNSBL list (StevenBlack_ADs - DNSBL_ADs_Basic in my case)

wc2l · Jan 8, 2025, 1:58 PM

@Gertjan I'm getting more confused!!
I have a maxmind account.

So it shoiuld be working. New keys don't work. Test again this morning
My have to go back to see if I have other notes. I think I used this post to create my last key:
https://forum.netgate.com/topic/149343/pfblockerng-maxmind-registration-required-to-continue-to-use-the-geoip-functionality

I have this setup:

My

Gertjan · Jan 8, 2025, 2:23 PM

@wc2l said in pfBlockerNG not blocking ADs:

https://forum.netgate.com/topic/149343/pfblockerng-maxmind-registration-required-to-continue-to-use-the-geoip-functionality

Already 5 years go ...
But yeah, that's the one I used to upgrade me GeoIP account.

How do you use this ?

My case :

and I use these list with 'french' IPs so I can use it like this :

My goal is to allow only french IPv4 to access my OpenVPN port.

Why would you "Deny Inbound" ?
Why not using "the build in free of maintenance, good for everybody" default block all rule (on your WAN interface).

if the context of this "Deny Inbound" is used for your WAN (it is, right ?) that, normally', everybody and everything is already blocked.
Except : your exceptions, like NAT rules.
Why do you make exceptions for "Top Spammers" ? Do you host a mail server behind pfSense or something like that ?

wc2l · Jan 8, 2025, 2:45 PM

@Gertjan
Here is my floating rules:

I don't remember changing or why changed the GeoIP Stuff
Now I have switched it back.

Now to chase down the DNSBL & Maxmind issues.
When I run the command: nslookup www.marrketstrategy.com
The results are:
Server: one.one.one.one
Address: 1.1.1.1
Non-authoritative answer:
Name: marrketstrategy.com
Address: 209.38.99.34
Aliases: www.marrketstrategy.com

[ MaxMind_BD_Proxy_v4 ] Downloading update .. 403 Forbidden
[ pfB_PRI3_v4 - MaxMind_BD_Proxy_v4 ] Download FAIL
DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.
The Following List has been REMOVED [ MaxMind_BD_Proxy_v4 ]

Uglybrian · Jan 8, 2025, 3:55 PM

It looks like Doh is being utilized. Maybe in your DNS settings or in your web browser.
My result using resolver

Look here for some insight into firefox Doh: https://forum.netgate.com/topic/133679/heads-up-be-aware-of-trusted-recursive-resolver-trr-in-firefox

wc2l · Jan 8, 2025, 4:02 PM

@Uglybrian
I wasn't using Firefox.
That was just a PS terminal.. I will reread stuff.

So what are you using for DNS servers? ISP?
Do you point your clients to your Netgate device?
Maybe that is what has changed in mine??

Gertjan · Jan 8, 2025, 4:17 PM

@wc2l said in pfBlockerNG not blocking ADs:

When I run the command: nslookup www.marrketstrategy.com
The results are:
Server: one.one.one.one
Address: 1.1.1.1
Non-authoritative answer:
Name: marrketstrategy.com
Address: 209.38.99.34
Aliases: www.marrketstrategy.com

Lol.
You control your pfSense, you can install things like pfBlockerng and do all kind of nifty DNS trick ...
And then some one decides that your LAN devices should use 1.1.1.1 as their DNS (1.1.1.1 is not pfSense) thus totally bypassing your DNS, so pfBlocker would never be able to see, and act upon your DNS request (from this LAN device).
Go call 1.1.1.1 and ask if they can install pfBlockerng on their system for you ?
(ok, was joking )

You can also remove this one :

as this floating rule operates on WAN, traffic coming from the alias IP list "pfB_PRI_v4".
All these IPs will be blocked anyway. You should lose any CPU cycles on them.

Uglybrian · Jan 8, 2025, 4:26 PM

I am using the authoritative name servers directly via the PF Sense resolver. Instead of asking Google or my ISP and having them answer either by their cache or them asking the name servers for me. I just use PFS to go to the name servers directly.PFS will answer from its cash or ask the name servers. I have no other typed in DNS servers And all clients point to PFS for their DNS. In addition, I also have firewall rules blocking all known Doh servers along one rule blocking the Dot port.
And all web browsers have an option for Doh usage that is usually switched on by default. You have to go into the browser settings and manually turn it off. This is not just on Firefox, but every web browser.

wc2l · Jan 8, 2025, 5:11 PM

@Uglybrian
I made sure there was nothing being added to my DHCP server, I need to figure out what is happening. Maybe I will open a ticket and see if that can help. Something is amiss...

pfBlockerNG update is still giving me errors not matter what I do. I will go back and look again. Still getting the ADs in one of the pages that used to be blocked :-(

Uglybrian · Jan 8, 2025, 5:42 PM

Does your resolver settings look like this?

From what I see you have something on your network using Cloudflare DNS, thus bypassing PFBlocker as Gertjan has pointed out.

I also use this firewall rule found in the PFS configuration recipes
https://docs.netgate.com/pfsense/en/latest/recipes/dns-block-external.html

Another layer I have is a floating firewall rule blocking DOH servers in the outbound direction.

wc2l · Jan 8, 2025, 5:53 PM

@Uglybrian
Took me a bit to figure where "DNS Resolution Behavior" was.. Mine was to "fall back to remote". I may have to clear cache and few other things. I need to look at the other settings yet.

Gertjan · Jan 9, 2025, 8:47 AM

@wc2l said in pfBlockerNG not blocking ADs:

I made sure there was nothing being added to my DHCP server, I need to figure out what is happening. Maybe I will open a ticket and see if that can help. Something is amiss...

On a device on your LAN :

When I run the command: nslookup www.marrketstrategy.com
The results are:
Server: one.one.one.one
Address: 1.1.1.1

so that device uses 1.1.1.1 as it DNS source.
1.1.1.1 can come from two sources :

The device user has set up the device with static DNS parameters, he entered 1.1.1.1.
It was given by the pfSense (?) DHCP server. So the admin has set that "1.1.1.1" as the DNS to be given to the LAN DHCP clients.

For both reasons, there is no need to open a ticket

You said that 2) isn't possible, so you have to deal with device's owner.
Or use a firewall rule ^^
"On LAN, block all DNS requests that are not directed to pfSense" and bye bye the 1.1.1.1 issue.

The owner of the device will ask for you very soon, and a discussion will take place.
He wants to circumvent your pfSense DNS.
You want his DNS requests to go to to pfSense, so you can filter it with pfBlockerng.

As the device's user is using your network, you will win. The discussion will end with : "if not happy, go elsewhere" as you can decides what happens on your network.

wc2l · Jan 9, 2025, 3:35 PM

@Gertjan
Looks like the issue is resolved. changing from "fall back to remote" to "ignore remote". I also made sure that there was NO other references for external DNS servers.

I've also been working on making sure that all of the downloads are working. I have found that some have changed policies or paths.

THANK YOU!