PFBlockerNG Python-Mode - Source-IP in Reports

mOrbo · Oct 7, 2023, 8:57 PM

Hi,

we just changed from unbound to python mode. It seems to run smooth and with a lot less resources. I like that!

But we have a problem with the logs/reports. In unbound-mode, it is possible to see the real source of a DNS request, not the IP from the internal DNS Server in the middle.

Our Clients are requesting DNS from our internal DNS Server, to be able to resolve internal things. This internal DNS-Server then uses our pfsense for external DNS request. So all external DNS requests go through the pfsense, but are "relayed/forwarded" by our internal DNS Server.

In unbound-mode I can see the real source IP from the client which is requesting a DNS, in python-mode I see only the source IP of our internal DNS Server. This makes it hard to find false positive blocks on a client.

In unbound-mode I can search for the IP of the client and see all blocked request. This helps to find blocked entries which causes problems on this client.
If I search for a client IP in python mode, there are no entries. But if I search for the blocked domain, I can see, that the source of the request was our internal DNS. This makes it hard to find a problematic domain on a specific client.

Are there any settings in phython-mode to show the real source IP and not the forward Server-IP like in unbound-mode?

andrebrait · Oct 7, 2023, 8:57 PM

@mOrbo so when going from your Internal DNS server, it sends Unbound the originating IP address from its clients and Unbound isn't passing that IP to the Python module as the client IP, it seems.

I'm gonna try to replicate your setup with a VM and see where that gets me.

mOrbo · Oct 10, 2023, 12:22 PM

@andrebrait

Thanks for your help. If you need any logs/infos or tests from unbound or pfsense let me know :-)

andrebrait · Oct 12, 2023, 7:45 AM

@mOrbo the relevant lines from your dnsbl.log and dns_reply.log files would be helpful. Same for a screenshot of the Reports tab.

Both when in Unbound and Python modes.

Also, until that gets fixed, you can use a checkbox on the DNSBL page that disables logging from inside Python and uses the DNSBL webserver for that instead. It's normally used to allow for AD environments but it should work for your case.

mOrbo · Oct 12, 2023, 7:54 AM

@andrebrait I did some tests now.

First in python mode. The DNS Server IP is 10.11.1.15, the Client IP is 10.11.3.6.

As you can see, the report shows the IP of our internal DNS Server.

Same thing in the dnsbl.log:

DNSBL-python,Oct 12 08:41:33,youporn.com,10.11.1.15,Python,TLD_A,DNSBL_UT1,youporn.com,UT1_adult,+

The dns_reply.log does not show any entries regarding this blocked request:

DNS-reply,Oct 12 08:41:24,reply,PTR,PTR,15699,90.182.178.59.in-addr.arpa,10.11.1.15,triband-del-59.178.182.90.bol.net.in,unk
DNS-reply,Oct 12 08:41:24,reply,A,CNAME,20,binaries.templates.cdn.office.net,10.11.1.15,89.27.247.9,DE
DNS-reply,Oct 12 08:41:26,reply,A,CNAME,33,nexus.officeapps.live.com,10.11.1.15,52.111.236.26,IE
DNS-reply,Oct 12 08:41:29,reply,A,A,126,0.de.pool.ntp.org,10.11.1.15,194.25.134.196,DE
DNS-reply,Oct 12 08:41:30,reply,PTR,PTR,1149,111.30.235.114.in-addr.arpa,10.11.1.15,NXDOMAIN,unk
DNS-reply,Oct 12 08:41:30,reply,A,A,60,prod.nexusrules.live.com.akadns.net,10.11.1.15,52.111.236.23,IE
DNS-reply,Oct 12 08:41:33,reply,A,A,60,4.sophosxl.net,10.11.1.15,52.215.114.87,IE
DNS-reply,Oct 12 08:41:34,reply,A,CNAME,Unk,odc-commonafdrk-geo.onedrive.akadns.net,10.11.1.15,13.107.42.12,US
DNS-reply,Oct 12 08:41:37,reply,A,CNAME,2,umwatson.events.data.microsoft.com,10.11.1.15,20.189.173.21,US
DNS-reply,Oct 12 08:41:37,reply,A,CNAME,43,wu-bg-shim.trafficmanager.net,10.11.1.15,93.184.221.240,GB
DNS-reply,Oct 12 08:41:39,reply,PTR,PTR,1798,75.118.240.91.in-addr.arpa,10.11.1.15,NXDOMAIN,unk
DNS-reply,Oct 12 08:41:45,reply,PTR,PTR,247,102.134.237.37.in-addr.arpa,10.11.1.15,NXDOMAIN,unk

After that I switched to unbound mode. Here you can see the Client-IP as source:

Also in the dnsbl.log file:

DNSBL-Full,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,+
DNSBL-1x1,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,-

As in python mode, the dns_reply.log file is empty for this timestamp.

andrebrait · Oct 13, 2023, 6:34 AM

@mOrbo thanks for the logs.

I'll see if I can find out whether I can get that data and whatnot. I need to build a similar setup (it isn't hard, but I need to do it) and try to see the data that Unbound provides the Python code with to see if there's anything there that's perhaps not in their docs.

mOrbo · Oct 13, 2023, 6:34 AM

@andrebrait Thanks for your help!

Gertjan · Oct 13, 2023, 8:03 AM

@mOrbo said in PFBlockerNG Python-Mode - Source-IP in Reports:

Also in the dnsbl.log file:

DNSBL-Full,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,+
DNSBL-1x1,Oct 12 09:11:44,youporn.com,10.11.3.6,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60,DNSBL,DNSBL_UT1,youporn.com,UT1_adult,-

Question : what does this text "GET / HTTP/1.1|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/117.0.0.0 Safari/537.36 " doing in a DNS request ?

Also : a DNS 'server' will not send upstream the original IP that initially for a DNS request.
Like unbound talking to "8.8.8.8" : "Hey, 192.168.10.45 wanted to know what the A record of "google.com" is".

Your ' internal DNS-Server' won't do neither. So, 'pfBlockerng' (unbound) DNS logs can't produce that info if the request came from an intermediate DNS server. It can only know about 'who', the IP the request came from, was asking not who was originally asking.

What probably happens is that DNSBL does its job, finds a DNSBL entry so it returns the pfBlockerng web server IP (the one that shows the "You are blocked " page).
Now, as the initial client was a web browser, it got redirected to this web server IP, this is the (default) 10.10.10.1
An that web server will 'see' the IP of the client.

Btw : the web page from this pfBlockerng web server will not show up on the client's device, as it can't show the page : no one can break https : meaning : if the browser wanted to visit https://whatever.tld and https://pfsense-pfblockerng-web-server/ was answering it will 'fail'
I do presume that no one use http anymore, as these sites simply do not exist anymore.

IMHO : the build in "pfBlockerng web server" is actually quiet useless.
But your question made me discover a reason to use it.

I switched to "Null block" a long time ago:

as my pfSense is the DNS of all my LAN's, I do see the originating (requester) IPs.

mOrbo · Oct 13, 2023, 1:09 PM

@Gertjan Probably your theory is right and the client IP comes from the pfblockerng webserver. So this IP could also be available in python mode.

What's not right is, that a https redirect will fail. It simply throws a certificate issue warning. So the client actually connects to the pfblockerng websever, but the browser displays a cert warning. Most users will close the site then, but if you ignore the warning you will see the block page.

So if the Client-IP comes from this webserver, it does not matter if its http or https.

Gertjan · Oct 13, 2023, 1:09 PM

@mOrbo said in PFBlockerNG Python-Mode - Source-IP in Reports:

you ignore the warning you will see the block page.

Very correct.
The thing is, you know that, and you probably know that black part of the URL is showing what is your pfSense device.
Like this URL :

The netgate.com part is in black. Because that the one that is important. That's the one matching the certificate from Netgate. So, ok, you know what you do. Internet isn't a dangerous place for you.

Now for the other 99,x % of us : if they all start to click through this "wrong cert" warning, pishing sites and other fake ones will have a bright future front of them.
So I tell them always : don't think - close it right away. Do not click any where.

IMHO : we as pfSense admins shouldn't contribute to the fact that our network users see this kind of info. It will introduce bad habits. And these 'stupid' users will pay the price later on.

Their IP will get logged of course, thus feeding the pfBlockerng stats.