pfblockerNG Question(s)
-
I am somewhat new to pfsense and quite new to pfblocker. I have been rolling pfsense with pfblocker into prod of my home lab network over the past few weeks. I have a couple questions to better understand how it works.
I have a few DNSBL feeds downloaded and running, eg EasyList and EasyPrivacy. Both are set to deny both. I also have several non-DNSBL lists enabled and blocking also. For the non-DNSBL lists, each one I enable, I can see the firewall rules for, which are targeted to aliases that contain the IP's from the list. Pretty simple there. The one I understand less is the DNSBL blocking. I have mine set in unbound python mode, as that is what pfsense seems to suggest. As I said, I have 2 feeds enabled. When I go into DNSBL Groups, I only see the EasyList feeds so I assume it is grouping them together and thats why I don't see both entries. Now what I am unsure about are the firewall rules for DNSBL. When I look at the firewall rules, I have 1 rule on WAN and 1 rule on LAN for DNSBL. They are both targeted at the DNSBL Alias, which I can see IP's in that list. So my question is about how the lists, like EasyPrivacy and EasyList, do their blocking when the DNSBL alias referenced by my firewall rules only has like ~70 IP's in the alias? When I look at the raw contents of the EasyList/EasyPrivacy, I see lots of FQDN's, URI's, etc. I do not see those in any alias so how does pfblockerNG block those?
I suspect it may be doing something different than using a firewall rule but I really don't know. I just want to understand it better so I know what to look for and watch out for as I move forward with tightening my config over time.
-
Any thoughts?
-
@djtech2k DNSBL uses DNS so there should not be firewall rules. Blocked domains can for instance resolve to a block page if you browse using http (https of course will have a certain error).
-
Yeah I realize it’s a DNS black hole essentially. I’m just trying to get a better understanding of how it works. For example, let’s say foo.com was on a DNSBL blacklist. If I open a browser on my laptop and try to go to foo.com, what happens? My DNS in pfsense is setup with DNS Resolver with a conditional forwarder for 1 specific FQDN to go to a different DNS server on my network. Other than that, DNS Resolver is set to forward requests to the upstream DNS server from DHCP.
In that situation, how does my dns query for foo.com get sent to the DNSBL web server in pfblocker? And what should I expect to see in my browser requesting foo.com?
I know the private IP assigned to the DNSBL web server so I tried it in the browser. It just shows a web page with a table with 1 row and a message that says the page is blocked by admins. Is that normal?
-
@djtech2k said in pfblockerNG Question(s):
If I open a browser on my laptop and try to go to foo.com, what happens?
To make a pretty important story short, you actually want to know what DNS (really) is.
First, check with the settings of your browser (all browser you use) what DNS it is using.
For example, whne visiting about:preferences#privacy on a Firefox browsers, on the bottiom of that page, you might be in for a big surprise.
If it isn't :
then the browser will completely bypass pfSense (== the resolver (unbound) and thus pfBlockerng).
When "Off" is selected, the browser will use the DNS that your device obtained by a DHCP lease (or whatever you've set it if you use a static IP setup)
Next test, still on your device : ask your device what (who) the system DNS it :
C:\Users\Gauche>nslookup www.google.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:907:xxyy:92ec:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : www.google.com Addresses: 2a00:1450:4007:80c::2004 172.217.20.164Every line is important here.
Serveur : pfSense.bhf.tld = is the host name of my pfSense.
Address: 2a01:cb19:907:xxyy:92ec:77ff:fe29:392c : sorry for this, normally, it should be 192.168.1.1, the IPv4 address of my pfSense. But IPv4 start t be phased out so ... here it is, my pfSense LAN IPv6 ...And then the two answers :
An A (IPv4) and an AAAA (IPv6) record : these are the answers.Check also this :
ipconfig /all ...... Serveurs DNS. . . . . . . . . . . . . : 2a01:cb19:907:xxyy:92ec:77ff:fe29:392c 192.168.1.1 .....which means that the DJCP server (pfSense) gave these two DNS server IP's to my device.
Keep in mind : my 192.168.1.1 is the same interface as 2a01:cb19:907:xxyy:92ec:77ff:fe29:392c.So, now I k,ow that my device will use pfSense as it's DNS source.
On pfSense : if you use the default Resolver settings, as they ware present when you installed pfSense, your good.
There are several ways to check if DNS requests from your LAN device are handled by the resolver :
Goto Services > DNS Resolver > Advanced Settings page and select :
WARNING : do not forget to set this back to "Level 1" when you're done - The DNS (Resolvers) log will get huge very fast !
and now visit : Status > System Logs > System > DNS Resolver and you'll see all DNS requests handled by the resolver listed.
Or visit : Firewall > pfBlockerNG > Alerts and go to the Unified page.
You'll see all DNS requests that are handled by the Resolver (and thus handled by pfBlockerng).Also, be ware, none of the people that created DNS (they are still there !) can be found on toctic.
You can see them here : https://www.youtube.com/results?search_query=DNSThis one How DNS Works - Computerphile, is, IMHO, gold. But there are many more.
Look at a couple more, let it sink in, and look some more over time. -
Thanks for the detail.
My DNS Resolver is set to "Enable Forwarding Mode". I thought that would be best for my situation, but again I am relatively new to pfsense. I have a DNS server on my network that only has records for a specific domain, so I have a conditional forwarder (Domain Override) setup in my DNS Resolver for that domain. Otherwise, I set the "Enable Forwarding Mode" option because I assume using my ISP DNS servers is the best option. So I assumed that my internal domain will get DNS forwarded to my DNS server and all other DNS queries will get sent to the DNS servers provided by my ISP DHCP.
Am I misunderstanding this or is that config not the best way to go? Specifically, should I not have the Forwarding mode enabled? Does this config cause pfblocker to not work as expected or cause other issues?
Here are my DNS settings in pfsense:
As for my main question here, it was really about how the DNS requests get sent to the DNSBL webserver. In my example, if any device requests foo.com and foo.com is on a blacklist, how does pfblocker send that request to the pfblocker dnsbl webserver instead of the normal DNS resolver path? Since its not a FW rule, I cannot see it there. Unless I am misunderstanding, something must happen behind the scenes that sends the DNS request for foo.com to the dnsbl webserver.
-
@djtech2k said in pfblockerNG Question(s):
If I open a browser on my laptop and try to go to foo.com, what happens?
A blocked hostname resolves to the configured IP address:
Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
Upvote đź‘Ť helpful posts! -
@djtech2k said in pfblockerNG Question(s):
I thought that would be best for my situation
And that's fine of course.
Just keep in mind that you also thought that you know better then a company called Netgate that made pfSense.
After all, if "forwarding" to "what ever (commercial) DNS server" is more secure as resolving, then it would be the default setting.Guess what, it wasn't set like that.
Netgate even activated DNSSEC, for security reasons.But get me right, there might be conditions that forwarding to a (commercial) DNS server is better.
All the big players (the local government, your ISP, your favorite VPN, etc etc) already always know "where" you go. So asking the road ( .. no, the IP ^^) to these places is pretty much the same thing.
@djtech2k said in pfblockerNG Question(s):
Otherwise, I set the "Enable Forwarding Mode" option because I assume using my ISP DNS servers is the best option
That option exists, IMHO, for ancient reasons.
Another adds another misunderstanding these days I guess.
"In the past", when we had 9600 or 33k6 modem bit rate counted, every byte counted. You paid per bit, remember ?
Most ISPs made available their won local DNS servers so resolving would speed up a bit. Our own ISP interlink connections were also very slow.
These days : most ISPs still offer DNS functionalities as these servers bring in some extra cash ^^
An ISP that forces you to use their DNS : leave them.With the new, multiple Mbytes /sec ISP connection these days, we can do at home with pfSense what everybody already did back in the past on the big systems : we use DNS as it was meant to be used : we resolve.
pfSense, out of the box, resolvesIf you wanted to know the phone number of Paul, why would you ask Joo ?
Why not asking Paul directly ?Ok, you get it, I'm a resolving fan.
I can't, of course, say that forwarding is bad and resolving is good. Both have their advantages, and disadvantages.
Resolving + DNSSEC (if available) is good enough for me.
Also, its the most simple setup : out of the box the system - pfSense - is ok.
And Netgate told already many times that I'm not smarter as they are ^^ So, for me, resolving it will be.Extra info : I also have my own domain name servers for my domain names (for my websites, mail reverse and so on). I wanted to do the DNSSEC (and a lot of other stuff) 'myself', as implementing DNS myself was what I did to learn about DNS.
-
DNSSEC
Speaking of which, disable DNSSEC if you are forwarding. It can cause false positives.
https://forum.netgate.com/topic/190742/should-i-enable-dnssec-in-pfsense-when-using-quad9-and-full-dns-hijacking/2
-
Ok so something behind the scenes must pickup that DNS query and resolve it to the VIP instead of sending it on to a DNS server to get resolved as the real IP. Is that right? What about the URI's in the dnsbl blacklist? For example, there are entries like "/ad/" or something like that. Is there like a script that does a regex or something before the DNS query gets sent to the resolver?
I did a quick test and did a nslookup on a domain name that I know is on the list and getting blocked via he pfblocker reports. The IP return is 0.0.0.0 but I expected it to be the VIP IP of the dnsbl webserver. Is that expected? Here are my settings:
I am unclear on when a request would get sent to the dnsbl webserver or the "blocked webpage".
-
@djtech2k The unbound DNS server loads the DNSBL lists and essentially uses them as overrides.
Not sure about your other questions offhand.
-
This "10.10.10.1" is an option that will get ditched soon.
edit 2024-01-10 : That is me saying that, as this functionality only works well with http interception.
The thing is : there are no public https (se no https) web sites any more ... right ?
https interception : won't work, as per design.
end edit.
I use this :
A browser on your LAN wants to visit facebook.com
But you, as the pfSense admin, have blocked "facebook.com" (as a DNSBL)Do you think that a page like this :
will pop up to warn the user in its browser that 'facebook' has been blocked ?
Now way ! Because you, me and nobody can't break TLS. Remember, https is used. The browser want to visit facebook.com but it gets an answer back from some guy called "10.10.10.1". That is interception, and that is bad. The browser will bark.
And not showing any pages - no way, it will throw the complex and scary messages on the screen .... (we all know them)So, the admin that start to understand what TLS (= https) is, doesn't care about what the pfBlockerng web server "10.10.10.1" has to offer.
"10.10.10.1" was nice in the good old http days.
There are no http server anymore, they have been shut down (actually : they are still there doing just one thing : redirecting you to the https counterpart, as http traffic can be redirected.That's why I "Null log".
Btw : Null logging means : return 0.0.0.0 (means : doesn't exist, don't insist - do nothing)
means : return "10.10.10.1" - which is, imho, as I exposed above, pretty useless.
-
Ok so just curious why my nslookup to a known blacklisted domain name would come back with 0.0.0.0 as the IP? I thought with my options as posted above, it would come back to the 10.x address of the webserver?
On a side note about DNS, I posted my config in DNS Resolved and the General DNS server settings. I understand the concepts of all the settings but I am unclear on which ones take precedence. So on the General DNS Server settings, I have the checkbox (Allow DNS server list to be overridden by DNCP..." and the DNS Resolution Behavior is set to Local then remote DNS. I then have the DNS Resolver setting "DNS Query Forwarding" enabled. I understand what all those things mean but I am unclear on how they work together.
I enabled the forwarding because my thought was that if I did not, pfsense would be running its own DNS server, which I felt like it would be unneeded functionality or attack surface, so I thought sending those requests to my ISP would be fine. I do not care where my DNS queries get resolved as long as its as secure as it can be AND I can continue to use the conditional forwarder I mentioned above. I just don't know the net effect of these 3 different DNS settings. I am not opposed to changing them as long as it meets the need in the best way I can.
-
@djtech2k Do you have the null block set, as noted above?
On pfSense only allowed (LAN etc.) clients can access it for DNS, and they need to do so if forwarding also. Unless you allow port 53 on WAN, all traffic arriving on WAN is blocked by default.
You can have LAN clients not use pfSense/Unbound at all, but then DNSBL won't work. (side note: if their browsers use DoH then they will bypass DNSBL)
If DNS forwarding is enabled then Unbound will forward the queries to the configured DNS servers instead of asking the root servers.
If you have allowed WAN DHCP to override the DNS server list then it will use the DNS servers your ISP says to use:
https://docs.netgate.com/pfsense/en/latest/config/general.html#dns-resolution-behavior
(there's a ? icon on each page that links to the doc page)Normally we set to forward because we forward to Quad9. Others have different opinions of what is preferable.
-
No, I have it set to go to the webserver but when I tried the nslookup to a known blacklisted domain, it came back with 0.0.0.0 as the IP, which surprised me based on my config.
As for DNS, the pfsense value seems to be DNS Override enabled, Local >then> Remote in the DNS Behavior settings in the General Setup section, which is exactly how mine is set. So the only deviation I have from the default is the forwarding enabled in the DNS Resolver config. So I guess the real difference is that my DNS queries will be sent to the DNS servers from my ISP DHCP lease and if I uncheck the forwarder box, pfsense will try to resolve it from cache and then look for root hints for resolution. Does that sound accurate?
All of my DNS discussion is on the LAN side. Its all blocked on the WAN by default.
I think my gap in knowledge on this is mainly around the "Unbound" part and how DNSBL works, along with a little but about how the DNS Resolver (and its options) work in pfsense. As long as my override/conditional forwarder will work, it sounds like I can use the forwarder option or remove it. Like I said, I just initially thought it was a component/work that I did not need pfsense to do so i was thinking forwarding would be better. I could be wrong all along anyway.
-
@djtech2k said in pfblockerNG Question(s):
it came back with 0.0.0.0 as the IP, which surprised me based on my config.
Hummm.
DNSBL files - with the Webserver IP 10.100.1.1, or 0.0.0.0 = Null block, depending on your choice, are (re) build when you do a Force reload All.edit : as you said yourself :
-
I did a force reload and repeated the nslookup for a blacklisted site. It still comes back as 0.0.0.0. Its blocked, which is what I want, but I am expecting it to go to the 10.x webserver but it doesn't seem to be.
-
@djtech2k said in pfblockerNG Question(s):
but I am expecting it to go to the 10.x webserver
I agree. pfBockerng is out smarting you ;)
Not so bad, as You've read this, right ?I did a test :
I switched to :
Global setting is :
and I executed a Force Reload All.
I picked a test site from the "Easylist" (https://easylist-downloads.adblockplus.org/easylist_noelemhide.txt) : adxprtz.com and checked it :
C:\Users\Gauche>nslookup adxprtz.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:dead:beef:92ec:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : adxprtz.com Address: 10.10.10.1so that looks fine.
I pasted it in the browser
Cool.
The site doesn't do a http to https redirect, so it could be safely redirected.
As many domains, it's actually a dead site (the domain name is probably paid for, but abandoned, there is no real add server behind it anymore - and this goes for most of the listed domain names)The more serious blacklisted domains that are actually active and these are the ones you want to be blocked as they really serve adds, like "googleaddservices" won't have this treatment, you'll see an ugly web browser error. No nice 10.10.10.1 page.
Ok, fine, I went back :
Set :
and then a Force Reload All
New test on the PC command line :
C:\Users\Gauche>nslookup adxprtz.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:dead:beef:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : adxprtz.com Address: 0.0.0.0Note that I even didn't do a
ipconfig /flushdnsas the TTL was probably already set very low by pfBlockerng so it vanishes from the local PC DNS cache in no time.
Can you show the last Reload sequence (at the bottom) of this log :
?
-
G Gertjan referenced this topic on
-
I did read the post and understand that the webserver may be a thing of the past. I am fine to change that setting BUT while I am trying to learn pfblocker better, I'd like to figure out why the results do not seem to match the settings I have. Working through figuring that out will help me better understand pfblocker.
Here is the reload section of my log from yesterday:
I just went back and changed my feed group setting to null block (logging) and did a reload under update. I flushed DNS client cache and tried the nslookup to collector.github.com, which is on the EasyPrivacy list. It came back 0.0.0.0. So I then reverted the setting back to the webserver and did the force reload again I ran the same nslookup and it also goes to 0.0.0.0. So I do not understand why its getting 0.0.0.0 and not my 10.100.x.x webserver.
-
The complete logs is at a minimum 3, 4 of 5 hundred lines.
Instead of posting a 1 Mega byte pixel size image that won't fit nowhere, post it like this :Here is mine : 2 k of text :
[ Force Reload Task - All ] UPDATE PROCESS START [ v3.2.1_20 ] [ 01/10/25 07:29:37 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... enabled Loading DNSBL Whitelist... completed Blacklist database(s) ... exists. [ UT1_agressif ] Reload [ 01/10/25 07:29:38 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 265 265 0 0 0 265 ---------------------------------------------------------------------- [ UT1_astrology ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 28 28 0 0 0 28 ---------------------------------------------------------------------- [ EasyList ] Reload . completed .. Whitelist: adnxs.net|adsafeprotected.com|amazon-adsystem.com|pips.taboola.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 29589 29589 0 4 0 29585 ---------------------------------------------------------------------- [ StevenBlack_ADs ] Reload [ 01/10/25 07:29:39 ] . completed .. Whitelist: 15.taboola.com|aax-eu.amazon-adsystem.com|addthis.com|adsafeprotected.com|am-match.taboola.com|am-sync.taboola.com|am-trc-events.taboola.com|am-vid-events.taboola.com|am-wf.taboola.com|amazon-adsystem.com|api-s2s.taboola.com|api.taboola.com|assets.taboola.com|beacon.taboola.com|bs.eyeblaster.akadns.net|bs.serving-sys.com|c2.taboola.com|c3.taboola.com|cdn-yjp.taboola.com|cdn.taboola.com|cds.taboola.com|ch-match.taboola.com|ch-sync.taboola.com|ch-vid-events.taboola.com|ch-wf.taboola.com|control.kochava.com|convchmp.taboola.com|convhkmp.taboola.com|convlatbmp.taboola.com|convnjmp.taboola.com|device-metrics-us-2.amazon.com|fls-na.amazon.com|geolocation.onetrust.com|googletagmanager.com|hk-match.taboola.com|hk-sync.taboola.com|hk-vid-events.taboola.com|hk-wf.taboola.com|images-dl.taboola.com|images.taboola.com|impr.taboola.com|imprammp.taboola.com|imprchmp.taboola.com|imprhkmp.taboola.com|imprlatbmp.taboola.com|imprnjmp.taboola.com|imprsgmp.taboola.com|la-match.taboola.com|la-sync.taboola.com|la-trc-events.taboola.com|la-vid-events.taboola.com|la-wf.taboola.com|localhost.localdomain|match.taboola.com|mb.taboola.com|mpg.taboola.com|nr-events.taboola.com|nr.taboola.com|opps.taboola.com|parkingcrew.net|pi.pardot.com|pixel.adsafeprotected.com|popup.taboola.com|px.moatads.com|resources.taboola.com|secure-gl.imrworldwide.com|sg-match.taboola.com|sg-sync.taboola.com|sg-trc-events.taboola.com|sg-vid-events.taboola.com|sg-wf.taboola.com|sync-t1.taboola.com|sync.taboola.com|taboola.com|timeinc.trc.taboola.com|trace.svc.ui.com|tracking.taboola.com|trc-events.taboola.com|trc.taboola.com|us-match.taboola.com|us-sync.taboola.com|us-vid-events.taboola.com|us-wf.taboola.com|vidanalytics.taboola.com|vidstat.taboola.com|vidstatb.taboola.com|vidutils.taboola.com|wf.taboola.com|wildcard.moatads.com.edgekey.net|www.addthis.com|www.api.taboola.com|www.c2.taboola.com|www.cdn.taboola.com|www.googletagmanager.com|www.images.taboola.com|www.parkingcrew.net|www.popup.taboola.com|www.taboola.com|www.trc.taboola.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 117135 117135 806 99 0 116230 ---------------------------------------------------------------------- ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 01/10/25 07:29:46 ] TLD: TLD analysis.. completed [ 01/10/25 07:29:49 ] TLD finalize... ---------------------------------------- Original Matches Removed Final ---------------------------------------- 146108 65425 44417 101691 ----------------------------------------- TLD finalize... completed [ 01/10/25 07:29:52 ] Saving DNSBL statistics... completed [ 01/10/25 07:29:53 ] Reloading Unbound Resolver (DNSBL python). Stopping Unbound Resolver. Unbound stopped in 2 sec. Additional mounts (DNSBL python): No changes required. Starting Unbound Resolver... completed [ 01/10/25 07:29:56 ] Resolver cache restored [ 01/10/25 07:29:58 ] DNSBL update [ 101691 | PASSED ]... completed ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ [ pfB_Europe_v4 ] Changes found... Updating Aggregation Stats: ------------------ Original Final ------------------ 35609 14269 ------------------ ===[ IPv4 Process ]================================================= [ MS_1_v4 ] Reload [ 01/10/25 07:29:59 ] . completed .. Aggregation Stats: ------------------ Original Final ------------------ 2866 2679 ------------------ ------------------------------ Original Master Final ------------------------------ 2876 2679 2679 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_Europe_v4 no changes. Updating: pfB_PRI5_v4 no changes. Saving config changes... completed ===[ Kill States ]================================================== Firewall state(s) validation for [ 19 ] IPv4 address(es)... Firewall state(s) validation for [ 38 ] IPv6 address(es)... No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 38485 ] [ Final IP Count ] [ 2679 ] ===[ Permit List IP Counts ]========================= 14269 /var/db/pfblockerng/permit/pfB_Europe_v4.txt ===[ Deny List IP Counts ]=========================== 2679 /var/db/pfblockerng/deny/MS_1_v4.txt ===[ DNSBL Domain/IP Counts ] =================================== 146108 total 116230 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt 29585 /var/db/pfblockerng/dnsbl/EasyList.txt 265 /var/db/pfblockerng/dnsbl/UT1_agressif.txt 28 /var/db/pfblockerng/dnsbl/UT1_astrology.txt ====================[ IPv4/6 Last Updated List Summary ]============== Dec 31 12:37 MS_1_v4 Jan 10 07:29 pfB_Europe_v4 ====================[ DNSBL Last Updated List Summary ]============== Jan 6 00:05 EasyList Jan 6 00:15 UT1_agressif Jan 6 00:15 UT1_astrology Jan 6 00:15 StevenBlack_ADs =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 16948 total 14269 /var/db/aliastables/pfB_Europe_v4.txt 2679 /var/db/aliastables/pfB_PRI5_v4.txt pfSense Table Stats ------------------- table-entries hard limit 800000 Table Usage Count 19757 UPDATE PROCESS ENDED [ 01/10/25 07:30:00 ]