PFBlockerNG Python-Mode - Source-IP in Reports

andrebrait

@mOrbo the relevant lines from your dnsbl.log and dns_reply.log files would be helpful. Same for a screenshot of the Reports tab.

Both when in Unbound and Python modes.

Also, until that gets fixed, you can use a checkbox on the DNSBL page that disables logging from inside Python and uses the DNSBL webserver for that instead. It's normally used to allow for AD environments but it should work for your case.

mOrbo

@andrebrait I did some tests now.

First in python mode. The DNS Server IP is 10.11.1.15, the Client IP is 10.11.3.6.

As you can see, the report shows the IP of our internal DNS Server.

Same thing in the dnsbl.log:

DNSBL-python,Oct 12 08:41:33,youporn.com,10.11.1.15,Python,TLD_A,DNSBL_UT1,youporn.com,UT1_adult,+

The dns_reply.log does not show any entries regarding this blocked request:

DNS-reply,Oct 12 08:41:24,reply,PTR,PTR,15699,90.182.178.59.in-addr.arpa,10.11.1.15,triband-del-59.178.182.90.bol.net.in,unk
DNS-reply,Oct 12 08:41:24,reply,A,CNAME,20,binaries.templates.cdn.office.net,10.11.1.15,89.27.247.9,DE
DNS-reply,Oct 12 08:41:26,reply,A,CNAME,33,nexus.officeapps.live.com,10.11.1.15,52.111.236.26,IE
DNS-reply,Oct 12 08:41:29,reply,A,A,126,0.de.pool.ntp.org,10.11.1.15,194.25.134.196,DE
DNS-reply,Oct 12 08:41:30,reply,PTR,PTR,1149,111.30.235.114.in-addr.arpa,10.11.1.15,NXDOMAIN,unk
DNS-reply,Oct 12 08:41:30,reply,A,A,60,prod.nexusrules.live.com.akadns.net,10.11.1.15,52.111.236.23,IE
DNS-reply,Oct 12 08:41:33,reply,A,A,60,4.sophosxl.net,10.11.1.15,52.215.114.87,IE
DNS-reply,Oct 12 08:41:34,reply,A,CNAME,Unk,odc-commonafdrk-geo.onedrive.akadns.net,10.11.1.15,13.107.42.12,US
DNS-reply,Oct 12 08:41:37,reply,A,CNAME,2,umwatson.events.data.microsoft.com,10.11.1.15,20.189.173.21,US
DNS-reply,Oct 12 08:41:37,reply,A,CNAME,43,wu-bg-shim.trafficmanager.net,10.11.1.15,93.184.221.240,GB
DNS-reply,Oct 12 08:41:39,reply,PTR,PTR,1798,75.118.240.91.in-addr.arpa,10.11.1.15,NXDOMAIN,unk
DNS-reply,Oct 12 08:41:45,reply,PTR,PTR,247,102.134.237.37.in-addr.arpa,10.11.1.15,NXDOMAIN,unk

After that I switched to unbound mode. Here you can see the Client-IP as source:

Also in the dnsbl.log file:

DNSBL-Full,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,+
DNSBL-1x1,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,-

As in python mode, the dns_reply.log file is empty for this timestamp.

andrebrait

@mOrbo thanks for the logs.

I'll see if I can find out whether I can get that data and whatnot. I need to build a similar setup (it isn't hard, but I need to do it) and try to see the data that Unbound provides the Python code with to see if there's anything there that's perhaps not in their docs.

mOrbo

@andrebrait Thanks for your help!

Gertjan

@mOrbo said in PFBlockerNG Python-Mode - Source-IP in Reports:

Also in the dnsbl.log file:

DNSBL-Full,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,+
DNSBL-1x1,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,-

Question : what does this text "GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 " doing in a DNS request ?

Also : a DNS 'server' will not send upstream the original IP that initially for a DNS request.
Like unbound talking to "8.8.8.8" : "Hey, 192.168.10.45 wanted to know what the A record of "google.com" is".

Your ' internal DNS-Server' won't do neither. So, 'pfBlockerng' (unbound) DNS logs can't produce that info if the request came from an intermediate DNS server. It can only know about 'who', the IP the request came from, was asking not who was originally asking.

What probably happens is that DNSBL does its job, finds a DNSBL entry so it returns the pfBlockerng web server IP (the one that shows the "You are blocked " page).
Now, as the initial client was a web browser, it got redirected to this web server IP, this is the (default) 10.10.10.1
An that web server will 'see' the IP of the client.

Btw : the web page from this pfBlockerng web server will not show up on the client's device, as it can't show the page : no one can break https : meaning : if the browser wanted to visit https://whatever.tld and https://pfsense-pfblockerng-web-server/ was answering it will 'fail'
I do presume that no one use http anymore, as these sites simply do not exist anymore.

IMHO : the build in "pfBlockerng web server" is actually quiet useless.
But your question made me discover a reason to use it.

I switched to "Null block" a long time ago:

as my pfSense is the DNS of all my LAN's, I do see the originating (requester) IPs.

mOrbo

@Gertjan Probably your theory is right and the client IP comes from the pfblockerng webserver. So this IP could also be available in python mode.

What's not right is, that a https redirect will fail. It simply throws a certificate issue warning. So the client actually connects to the pfblockerng websever, but the browser displays a cert warning. Most users will close the site then, but if you ignore the warning you will see the block page.

So if the Client-IP comes from this webserver, it does not matter if its http or https.

Gertjan

@mOrbo said in PFBlockerNG Python-Mode - Source-IP in Reports:

you ignore the warning you will see the block page.

Very correct.
The thing is, you know that, and you probably know that black part of the URL is showing what is your pfSense device.
Like this URL :

The netgate.com part is in black. Because that the one that is important. That's the one matching the certificate from Netgate. So, ok, you know what you do. Internet isn't a dangerous place for you.

Now for the other 99,x % of us : if they all start to click through this "wrong cert" warning, pishing sites and other fake ones will have a bright future front of them.
So I tell them always : don't think - close it right away. Do not click any where.

IMHO : we as pfSense admins shouldn't contribute to the fact that our network users see this kind of info. It will introduce bad habits. And these 'stupid' users will pay the price later on.

Their IP will get logged of course, thus feeding the pfBlockerng stats.

seraph77

Hi,

I just ran into the exact same issue after switching to python mode. Since the update to 24.07.1 I have no more logs (unified /Alerts) when in Unbound mode. Before it worked like a charm. It still blocks but does not report. So I believe there's a bug somehow. When switching to Python mode, the logs are created again , but only showing the IP of the internal DNS, which is not that helpful for analysis. Is there a way to fix either of it (Python mode reporting source client ip or Unbound logging again as before)?

Gertjan

@seraph77

You don't see something like this :

?

@seraph77 said in PFBlockerNG Python-Mode - Source-IP in Reports:

but only showing the IP of the internal DNS

? Where ?

seraph77

@Gertjan I see those entry's in Python mode only (Unified and Alerts Log). And there also only, the request coming from the internal dns server (before the update I saw the IP of the Clients under Unified and Alerts mode Unbound only). With unbound only mode, the logs are not generated now (not in Unified and not in Alerts). Since I did read your previous Posts I decided to switch to Domain Overrides. So just all internal DNS requests would be redirected to my internal dns. So now I see all the Client IP's again in Python mode (Unified and Alerts log). I guess it's just the way it is at the moment. Probably DNS is anyway faster with this config. What's still not clear to me is, if i would put Null Block (logging) would it be faster than going throug DNSBL webserver/VIP?

BBcan177

@seraph77

For Python mode, when you use an internal dns server, you can either null block or check the option "DNSBL Event Logging", which will provide a workaround for this issue.

seraph77

@BBcan177 Thanks for the help. I wil try that.

Gertjan

@seraph77 said in PFBlockerNG Python-Mode - Source-IP in Reports:

What's still not clear to me is, if i would put Null Block (logging) would it be faster than going throug DNSBL webserver/VIP?

No speed difference here.
If a listed == blocked domain was requested, instead of resolving it, pfBlocker intercepts the request, and 'teminates' it for unbound right away with the answer '0.0.0.0', (the "Null Block (logging)) option which will indicate the requesting source, i.e. your web browser, that their is no answer (no A or AAAA) so the content won't get loaded.
When you select "DNSBL webserver/VIP" instead of "Null Block (logging), then an IP will be return, by default "10.10.10.1" which is an IP served by a pfBlockerng web interface.
This means, if you where using a web server on the client, this 'page' would be loaded, and instead of the requested blocked domain name, this would be shown :

but there is a but : this is pretty worthless.
These days, most http requests are not http any more, but https. This means the web browser will request a certificate from the pfBlockerng web server.

You'll see :

because the browser was requesting initially "a-pfblockerng-blocked-host.tld" and it got back an IP for this host : "10.10.10.1". When connecting to it using https, the web browser will ask for a certicate and here it comes : this certiccate will not / can not contain the SAN "a-pfblockerng-blocked-host.tld" but something else. click on "Advanced" and you'll see :

So, two issues : the cert is self signed, which is not a good sign when you (try to) visit a public web site, and - not even shown but you can have a look at the cert : it doesn't contain "a-pfblockerng-blocked-host.tld" neither.

Mine showed :

which isn't "a-pfblockerng-blocked-host.tld" for sure.

This is what https is all about.
This also means that "DNSBL webserver/VIP" is pretty worthless in giving the visiting end user a reason why the web visit failed. Most people on earth don't understand what that browser screen really tells.
The will abort and or run away, which is a good thing.
When https is used, the pretty pfBlockerng web server page (see above) won't show up. The browser won't allow that.

Long story short : imho, use "Null Block (logging)" and call it a day.

mOrbo

The main problem is not the block site or if there is a https error. This does not matter. The problem is, that the client IP is missing in the logs as source IP. I for myself switched back to unbound-mode two years ago.

Back then there was no option for a workaround. @andrebrait wanted to look deeper into this, but he never posted a solution here. So I switched back to unbound-mode.

Don't know if the option from @BBcan177 helps to solve the problem now. But please let me know if it helps. Would be nice.

seraph77

@Gertjan Thanks for the detailed answer. I will leave it to null block (logging) then.

seraph77

@mOrbo Well I just checked and I will stay on my new config (Client gets Pfsense as DNS entry). It seems that I gained some speed with this setup and I don't want to spend more time with this. For my internal DNS request from the clients, I did just put my internal domain as Domain Override and point it to the ip of the internal DNS Server. That worked flawlessly.

mOrbo

@seraph77

Yes this could be a solution. But I’m afraid of all the consequences this can have. For example problems with automatic DNS updates from clients over VPN Tunnels, revers DNS Lookups or multiple internal domains including DNS-Split-Setups for several hybrid solutions. From my experience the internal windows dns service works best for a windows domain. Especially if this domain has a lot services relying on dns. Switching to the pfsense DNS for all clients would include a lot of testing. Unbound mode was the easier solution for me, as my pfsense has enough ram and cpu for handling unbound mode with over 5 million entries.

seraph77

@mOrbo O.k. i see. Under such circumstances i would also stay on the internal DNS. Well just give it a try with

@BBcan177 said in PFBlockerNG Python-Mode - Source-IP in Reports:

For Python mode, when you use an internal dns server, you can either null block or check the option "DNSBL Event Logging", which will provide a workaround for this issue.

So as far i remember, it did not work with Python mode and DNSBL Null block (logging). But i surely did not test it with checking "DNSBL Event Logging" and DNSBL Webserver / VIP.