pfblockerNG Question(s)
-
Ok so something behind the scenes must pickup that DNS query and resolve it to the VIP instead of sending it on to a DNS server to get resolved as the real IP. Is that right? What about the URI's in the dnsbl blacklist? For example, there are entries like "/ad/" or something like that. Is there like a script that does a regex or something before the DNS query gets sent to the resolver?
I did a quick test and did a nslookup on a domain name that I know is on the list and getting blocked via he pfblocker reports. The IP return is 0.0.0.0 but I expected it to be the VIP IP of the dnsbl webserver. Is that expected? Here are my settings:
I am unclear on when a request would get sent to the dnsbl webserver or the "blocked webpage".
-
@djtech2k The unbound DNS server loads the DNSBL lists and essentially uses them as overrides.
Not sure about your other questions offhand.
-
This "10.10.10.1" is an option that will get ditched soon.
edit 2024-01-10 : That is me saying that, as this functionality only works well with http interception.
The thing is : there are no public https (se no https) web sites any more ... right ?
https interception : won't work, as per design.
end edit.
I use this :
A browser on your LAN wants to visit facebook.com
But you, as the pfSense admin, have blocked "facebook.com" (as a DNSBL)Do you think that a page like this :
will pop up to warn the user in its browser that 'facebook' has been blocked ?
Now way ! Because you, me and nobody can't break TLS. Remember, https is used. The browser want to visit facebook.com but it gets an answer back from some guy called "10.10.10.1". That is interception, and that is bad. The browser will bark.
And not showing any pages - no way, it will throw the complex and scary messages on the screen .... (we all know them)So, the admin that start to understand what TLS (= https) is, doesn't care about what the pfBlockerng web server "10.10.10.1" has to offer.
"10.10.10.1" was nice in the good old http days.
There are no http server anymore, they have been shut down (actually : they are still there doing just one thing : redirecting you to the https counterpart, as http traffic can be redirected.That's why I "Null log".
Btw : Null logging means : return 0.0.0.0 (means : doesn't exist, don't insist - do nothing)
means : return "10.10.10.1" - which is, imho, as I exposed above, pretty useless.
-
Ok so just curious why my nslookup to a known blacklisted domain name would come back with 0.0.0.0 as the IP? I thought with my options as posted above, it would come back to the 10.x address of the webserver?
On a side note about DNS, I posted my config in DNS Resolved and the General DNS server settings. I understand the concepts of all the settings but I am unclear on which ones take precedence. So on the General DNS Server settings, I have the checkbox (Allow DNS server list to be overridden by DNCP..." and the DNS Resolution Behavior is set to Local then remote DNS. I then have the DNS Resolver setting "DNS Query Forwarding" enabled. I understand what all those things mean but I am unclear on how they work together.
I enabled the forwarding because my thought was that if I did not, pfsense would be running its own DNS server, which I felt like it would be unneeded functionality or attack surface, so I thought sending those requests to my ISP would be fine. I do not care where my DNS queries get resolved as long as its as secure as it can be AND I can continue to use the conditional forwarder I mentioned above. I just don't know the net effect of these 3 different DNS settings. I am not opposed to changing them as long as it meets the need in the best way I can.
-
@djtech2k Do you have the null block set, as noted above?
On pfSense only allowed (LAN etc.) clients can access it for DNS, and they need to do so if forwarding also. Unless you allow port 53 on WAN, all traffic arriving on WAN is blocked by default.
You can have LAN clients not use pfSense/Unbound at all, but then DNSBL won't work. (side note: if their browsers use DoH then they will bypass DNSBL)
If DNS forwarding is enabled then Unbound will forward the queries to the configured DNS servers instead of asking the root servers.
If you have allowed WAN DHCP to override the DNS server list then it will use the DNS servers your ISP says to use:
https://docs.netgate.com/pfsense/en/latest/config/general.html#dns-resolution-behavior
(there's a ? icon on each page that links to the doc page)Normally we set to forward because we forward to Quad9. Others have different opinions of what is preferable.
-
No, I have it set to go to the webserver but when I tried the nslookup to a known blacklisted domain, it came back with 0.0.0.0 as the IP, which surprised me based on my config.
As for DNS, the pfsense value seems to be DNS Override enabled, Local >then> Remote in the DNS Behavior settings in the General Setup section, which is exactly how mine is set. So the only deviation I have from the default is the forwarding enabled in the DNS Resolver config. So I guess the real difference is that my DNS queries will be sent to the DNS servers from my ISP DHCP lease and if I uncheck the forwarder box, pfsense will try to resolve it from cache and then look for root hints for resolution. Does that sound accurate?
All of my DNS discussion is on the LAN side. Its all blocked on the WAN by default.
I think my gap in knowledge on this is mainly around the "Unbound" part and how DNSBL works, along with a little but about how the DNS Resolver (and its options) work in pfsense. As long as my override/conditional forwarder will work, it sounds like I can use the forwarder option or remove it. Like I said, I just initially thought it was a component/work that I did not need pfsense to do so i was thinking forwarding would be better. I could be wrong all along anyway.
-
@djtech2k said in pfblockerNG Question(s):
it came back with 0.0.0.0 as the IP, which surprised me based on my config.
Hummm.
DNSBL files - with the Webserver IP 10.100.1.1, or 0.0.0.0 = Null block, depending on your choice, are (re) build when you do a Force reload All.edit : as you said yourself :
-
I did a force reload and repeated the nslookup for a blacklisted site. It still comes back as 0.0.0.0. Its blocked, which is what I want, but I am expecting it to go to the 10.x webserver but it doesn't seem to be.
-
@djtech2k said in pfblockerNG Question(s):
but I am expecting it to go to the 10.x webserver
I agree. pfBockerng is out smarting you ;)
Not so bad, as You've read this, right ?I did a test :
I switched to :
Global setting is :
and I executed a Force Reload All.
I picked a test site from the "Easylist" (https://easylist-downloads.adblockplus.org/easylist_noelemhide.txt) : adxprtz.com and checked it :
C:\Users\Gauche>nslookup adxprtz.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:dead:beef:92ec:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : adxprtz.com Address: 10.10.10.1so that looks fine.
I pasted it in the browser
Cool.
The site doesn't do a http to https redirect, so it could be safely redirected.
As many domains, it's actually a dead site (the domain name is probably paid for, but abandoned, there is no real add server behind it anymore - and this goes for most of the listed domain names)The more serious blacklisted domains that are actually active and these are the ones you want to be blocked as they really serve adds, like "googleaddservices" won't have this treatment, you'll see an ugly web browser error. No nice 10.10.10.1 page.
Ok, fine, I went back :
Set :
and then a Force Reload All
New test on the PC command line :
C:\Users\Gauche>nslookup adxprtz.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:dead:beef:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : adxprtz.com Address: 0.0.0.0Note that I even didn't do a
ipconfig /flushdnsas the TTL was probably already set very low by pfBlockerng so it vanishes from the local PC DNS cache in no time.
Can you show the last Reload sequence (at the bottom) of this log :
?
-
G Gertjan referenced this topic on
-
I did read the post and understand that the webserver may be a thing of the past. I am fine to change that setting BUT while I am trying to learn pfblocker better, I'd like to figure out why the results do not seem to match the settings I have. Working through figuring that out will help me better understand pfblocker.
Here is the reload section of my log from yesterday:
I just went back and changed my feed group setting to null block (logging) and did a reload under update. I flushed DNS client cache and tried the nslookup to collector.github.com, which is on the EasyPrivacy list. It came back 0.0.0.0. So I then reverted the setting back to the webserver and did the force reload again I ran the same nslookup and it also goes to 0.0.0.0. So I do not understand why its getting 0.0.0.0 and not my 10.100.x.x webserver.
-
The complete logs is at a minimum 3, 4 of 5 hundred lines.
Instead of posting a 1 Mega byte pixel size image that won't fit nowhere, post it like this :Here is mine : 2 k of text :
[ Force Reload Task - All ] UPDATE PROCESS START [ v3.2.1_20 ] [ 01/10/25 07:29:37 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... enabled Loading DNSBL Whitelist... completed Blacklist database(s) ... exists. [ UT1_agressif ] Reload [ 01/10/25 07:29:38 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 265 265 0 0 0 265 ---------------------------------------------------------------------- [ UT1_astrology ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 28 28 0 0 0 28 ---------------------------------------------------------------------- [ EasyList ] Reload . completed .. Whitelist: adnxs.net|adsafeprotected.com|amazon-adsystem.com|pips.taboola.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 29589 29589 0 4 0 29585 ---------------------------------------------------------------------- [ StevenBlack_ADs ] Reload [ 01/10/25 07:29:39 ] . completed .. Whitelist: 15.taboola.com|aax-eu.amazon-adsystem.com|addthis.com|adsafeprotected.com|am-match.taboola.com|am-sync.taboola.com|am-trc-events.taboola.com|am-vid-events.taboola.com|am-wf.taboola.com|amazon-adsystem.com|api-s2s.taboola.com|api.taboola.com|assets.taboola.com|beacon.taboola.com|bs.eyeblaster.akadns.net|bs.serving-sys.com|c2.taboola.com|c3.taboola.com|cdn-yjp.taboola.com|cdn.taboola.com|cds.taboola.com|ch-match.taboola.com|ch-sync.taboola.com|ch-vid-events.taboola.com|ch-wf.taboola.com|control.kochava.com|convchmp.taboola.com|convhkmp.taboola.com|convlatbmp.taboola.com|convnjmp.taboola.com|device-metrics-us-2.amazon.com|fls-na.amazon.com|geolocation.onetrust.com|googletagmanager.com|hk-match.taboola.com|hk-sync.taboola.com|hk-vid-events.taboola.com|hk-wf.taboola.com|images-dl.taboola.com|images.taboola.com|impr.taboola.com|imprammp.taboola.com|imprchmp.taboola.com|imprhkmp.taboola.com|imprlatbmp.taboola.com|imprnjmp.taboola.com|imprsgmp.taboola.com|la-match.taboola.com|la-sync.taboola.com|la-trc-events.taboola.com|la-vid-events.taboola.com|la-wf.taboola.com|localhost.localdomain|match.taboola.com|mb.taboola.com|mpg.taboola.com|nr-events.taboola.com|nr.taboola.com|opps.taboola.com|parkingcrew.net|pi.pardot.com|pixel.adsafeprotected.com|popup.taboola.com|px.moatads.com|resources.taboola.com|secure-gl.imrworldwide.com|sg-match.taboola.com|sg-sync.taboola.com|sg-trc-events.taboola.com|sg-vid-events.taboola.com|sg-wf.taboola.com|sync-t1.taboola.com|sync.taboola.com|taboola.com|timeinc.trc.taboola.com|trace.svc.ui.com|tracking.taboola.com|trc-events.taboola.com|trc.taboola.com|us-match.taboola.com|us-sync.taboola.com|us-vid-events.taboola.com|us-wf.taboola.com|vidanalytics.taboola.com|vidstat.taboola.com|vidstatb.taboola.com|vidutils.taboola.com|wf.taboola.com|wildcard.moatads.com.edgekey.net|www.addthis.com|www.api.taboola.com|www.c2.taboola.com|www.cdn.taboola.com|www.googletagmanager.com|www.images.taboola.com|www.parkingcrew.net|www.popup.taboola.com|www.taboola.com|www.trc.taboola.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 117135 117135 806 99 0 116230 ---------------------------------------------------------------------- ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 01/10/25 07:29:46 ] TLD: TLD analysis.. completed [ 01/10/25 07:29:49 ] TLD finalize... ---------------------------------------- Original Matches Removed Final ---------------------------------------- 146108 65425 44417 101691 ----------------------------------------- TLD finalize... completed [ 01/10/25 07:29:52 ] Saving DNSBL statistics... completed [ 01/10/25 07:29:53 ] Reloading Unbound Resolver (DNSBL python). Stopping Unbound Resolver. Unbound stopped in 2 sec. Additional mounts (DNSBL python): No changes required. Starting Unbound Resolver... completed [ 01/10/25 07:29:56 ] Resolver cache restored [ 01/10/25 07:29:58 ] DNSBL update [ 101691 | PASSED ]... completed ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ [ pfB_Europe_v4 ] Changes found... Updating Aggregation Stats: ------------------ Original Final ------------------ 35609 14269 ------------------ ===[ IPv4 Process ]================================================= [ MS_1_v4 ] Reload [ 01/10/25 07:29:59 ] . completed .. Aggregation Stats: ------------------ Original Final ------------------ 2866 2679 ------------------ ------------------------------ Original Master Final ------------------------------ 2876 2679 2679 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_Europe_v4 no changes. Updating: pfB_PRI5_v4 no changes. Saving config changes... completed ===[ Kill States ]================================================== Firewall state(s) validation for [ 19 ] IPv4 address(es)... Firewall state(s) validation for [ 38 ] IPv6 address(es)... No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 38485 ] [ Final IP Count ] [ 2679 ] ===[ Permit List IP Counts ]========================= 14269 /var/db/pfblockerng/permit/pfB_Europe_v4.txt ===[ Deny List IP Counts ]=========================== 2679 /var/db/pfblockerng/deny/MS_1_v4.txt ===[ DNSBL Domain/IP Counts ] =================================== 146108 total 116230 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt 29585 /var/db/pfblockerng/dnsbl/EasyList.txt 265 /var/db/pfblockerng/dnsbl/UT1_agressif.txt 28 /var/db/pfblockerng/dnsbl/UT1_astrology.txt ====================[ IPv4/6 Last Updated List Summary ]============== Dec 31 12:37 MS_1_v4 Jan 10 07:29 pfB_Europe_v4 ====================[ DNSBL Last Updated List Summary ]============== Jan 6 00:05 EasyList Jan 6 00:15 UT1_agressif Jan 6 00:15 UT1_astrology Jan 6 00:15 StevenBlack_ADs =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 16948 total 14269 /var/db/aliastables/pfB_Europe_v4.txt 2679 /var/db/aliastables/pfB_PRI5_v4.txt pfSense Table Stats ------------------- table-entries hard limit 800000 Table Usage Count 19757 UPDATE PROCESS ENDED [ 01/10/25 07:30:00 ] -
This post is deleted! -
I thought you only wanted that specific section. Here is the entire latest force reload section.
[ Force Reload Task - DNSBL ] UPDATE PROCESS START [ v3.2.0_20 ] [ 01/10/25 08:35:44 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... disabled Loading DNSBL Whitelist... completed [ EasyList ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 29672 29672 0 0 0 29672 ---------------------------------------------------------------------- IPv4 count=101 [ EasyPrivacy ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 41424 41423 0 0 0 41423 ---------------------------------------------------------------------- IPv4 count=8 Saving DNSBL statistics... completed [ 01/10/25 08:35:45 ] ------------------------------------------------------------------------ Assembling DNSBL database...... completed Reloading Unbound Resolver (DNSBL python). Stopping Unbound Resolver. Unbound stopped in 2 sec. Additional mounts (DNSBL python): No changes required. Starting Unbound Resolver... completed [ 01/10/25 08:35:47 ] Resolver cache restored DNSBL update [ 71095 | PASSED ]... completed ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] exists. [ Abuse_SSLBL_v4 ] exists. [ CINS_army_v4 ] exists. [ ET_Block_v4 ] exists. [ ET_Comp_v4 ] exists. [ ISC_Block_v4 ] exists. [ Spamhaus_Drop_v4 ] exists. [ Talos_BL_v4 ] Downloading update .. 200 OK. completed .. [ pfB_PRI1_v4 Talos_BL_v4 ] No IPs found! Ensure only IP based Feeds are used! ] [ BDS_TOR_v4 ] exists. [ 01/10/25 08:35:48 ] [ DMe_TOR_EN_v4 ] exists. [ ET_TOR_All_v4 ] exists. [ ISC_TOR_v4 ] exists. [ PROJECT_TOR_EN_v4 ] exists. [ RUECKGR_TOR_All_v4 ] exists. [ SFS_IPs_v4 ] exists. [ DNSBLIP_v4 ] Downloading update .. completed .. Aggregation Stats: ------------------ Original Final ------------------ 109 72 ------------------ ------------------------------ Original Master Final ------------------------------ 109 72 72 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_DNSBLIP_v4 no changes. Saving config changes... completed ===[ Kill States ]================================================== No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 207257 ] [ Final IP Count ] [ 174653 ] ===[ Deny List IP Counts ]=========================== 174654 total 150429 /var/db/pfblockerng/deny/SFS_IPs_v4.txt 11928 /var/db/pfblockerng/deny/CINS_army_v4.txt 5404 /var/db/pfblockerng/deny/BDS_TOR_v4.txt 3651 /var/db/pfblockerng/deny/ET_TOR_All_v4.txt 1322 /var/db/pfblockerng/deny/ET_Block_v4.txt 565 /var/db/pfblockerng/deny/DMe_TOR_EN_v4.txt 562 /var/db/pfblockerng/deny/ET_Comp_v4.txt 403 /var/db/pfblockerng/deny/RUECKGR_TOR_All_v4.txt 294 /var/db/pfblockerng/deny/ISC_TOR_v4.txt 72 /var/db/pfblockerng/deny/DNSBLIP_v4.txt 9 /var/db/pfblockerng/deny/Spamhaus_Drop_v4.txt 7 /var/db/pfblockerng/deny/ISC_Block_v4.txt 6 /var/db/pfblockerng/deny/PROJECT_TOR_EN_v4.txt 1 /var/db/pfblockerng/deny/Abuse_SSLBL_v4.txt 1 /var/db/pfblockerng/deny/Abuse_Feodo_C2_v4.txt ====================[ Empty Lists w/127.1.7.7 ]================== Abuse_SSLBL_v4.txt ===[ DNSBL Domain/IP Counts ] =================================== 71204 total 41423 /var/db/pfblockerng/dnsbl/EasyPrivacy.txt 29672 /var/db/pfblockerng/dnsbl/EasyList.txt 101 /var/db/pfblockerng/dnsbl/EasyList_v4.ip 8 /var/db/pfblockerng/dnsbl/EasyPrivacy_v4.ip ====================[ IPv4/6 Last Updated List Summary ]============== Jan 3 06:40 Abuse_SSLBL_v4 Jan 9 00:30 ET_Block_v4 Jan 9 06:35 Spamhaus_Drop_v4 Jan 9 11:54 BDS_TOR_v4 Jan 9 17:15 ET_TOR_All_v4 Jan 9 17:15 ET_Comp_v4 Jan 10 06:40 ISC_TOR_v4 Jan 10 06:52 CINS_army_v4 Jan 10 07:00 ISC_Block_v4 Jan 10 07:30 PROJECT_TOR_EN_v4 Jan 10 07:31 DMe_TOR_EN_v4 Jan 10 07:55 Abuse_Feodo_C2_v4 Jan 10 07:55 SFS_IPs_v4 Jan 10 08:01 RUECKGR_TOR_All_v4 Jan 10 08:35 Talos_BL_v4 Jan 10 08:35 DNSBLIP_v4 ====================[ DNSBL Last Updated List Summary ]============== Jan 9 21:55 EasyPrivacy Jan 9 23:54 EasyList =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 174654 total 150429 /var/db/aliastables/pfB_SFS_v4.txt 13830 /var/db/aliastables/pfB_PRI1_v4.txt 10323 /var/db/aliastables/pfB_TOR_v4.txt 72 /var/db/aliastables/pfB_DNSBLIP_v4.txt pfSense Table Stats ------------------- table-entries hard limit 400000 Table Usage Count 174668 UPDATE PROCESS ENDED [ 01/10/25 08:35:50 ] -
S SteveITS referenced this topic on
-
I still have not figured out why my packets are resolving to 0.0.0.0 but its more of a curiosity thing at this point. resolving to 0.0.0.0 still blocks it, so that's my goal. Just would like to figure out why it is going there and not to the 10.x webserver.
I have been slowly adding more feeds to pfblocker and I have been trying to check out the logs to see how much is being blocked, and also how the logs look. I have been noticing that a few feeds are repeatedly failing to update both on the scheduled update and when I force it manually. When I go to the feed listed in pfsense, I am able to click the link and download the feed just fine but in the logs it says the updates fail because it can't download the feed.
Are any of you seeing this behavior and is it normal? For example, I am getting it on Talos_BL_v4 and DMe_TOR_EN_v4.
-
@djtech2k Talos basically isn't valid anymore, see thread
https://forum.netgate.com/topic/190566/pfb_pri1_v4-talos_bl_v4-download-fail -
OK thanks. Good to know. I will remove it then.
-
I recently added the ISC_Miner list and looks like it may be dead too.