pfBlockerNG not blocking everything in blocklists
-
You can do some more checking.
Locate the file /var/unbound/pfb_py_data.txt - load it into, for example, notepad++.
(a SSH access and SFTP with WinSCP comes in handy here)rw-r--r-- 1 root unbound 63782900 Apr 5 10:29 /var/unbound/pfb_py_data.txtA 63 Megabytes file (and I'm just using a couple of feeds).
When the oisd.nl feed file is loaded and parsed in, you'll find lines in this file like
.... ,ku6.com,,0,OISD,DNSBL_Compilation ... ,www.ku6.com,,0,OISD,DNSBL_Compilation .... etc. ....unbound, when it receive a DNS lookup request, first looks in the local cache. Part of the lookup is a call to the python script written for pfblockerNG. And this script uses /var/unbound/pfb_py_data.txt as it's local lookup table.
If a host name is found in that file, it will be blocked.I tend to say that when "ku6.com" resolves, it isn't present in var/unbound/pfb_py_data.txt or : the feed didn't get loaded.
-
I have checked the files and see the screenshots below:
Then I checked the entries for the domains in the file. They are there:
And then when I check with nslookup pfBlockerNG is not blocking them:
I think something seriously is broken and I hope that it is only on my end. I am planning to do a clean install tomorrow and restore my backup and hopefully that will fix this.
-
I can confirm that reloading (force or normal) of pfBlockerNG does not solve this problem. However (like said before) when I restart unbound service it start working properly and pfBlockerNG blocks those domains.
After restarting Unbound I go back to force reload pfblockerNG (to test) and it keeps working.
After that I hit pfSense again with GRC DNS benchmark tool. This tool generates a LOT of DNS requests. After running the benchmark tool pfBlockerNG is broken and pfSense unbound resolves those blocked domains. So it look like when you hit pfSense Unbound with a lot of DNS requests something breaks under the load and pfBlockerNG is not working anymore with unbound. The pfb_py_data.txt is still there containing the entries from the blocked feeds. Force reloading pfBlockerNG does nog fix the problem at this point. I have to restart Unbound service manually again fixes the problem :|
@Gertjan : can you test your side also with the GRC DNS Benchmark tool and see of pfsense with pfblocker hold up? I would appreciate it!
-
Actually, there is nothing to restart.
Check the unbound config :[2.5.0-RELEASE][root@mypfsense.net]/root: tail -n 3 /var/unbound/unbound.conf # Python Module python: python-script: pfb_unbound.pyunbound works - and uses the pfb_unbound.py script file to do some extra work.
Btw :
tail -f /var/unbound/var/log/pfblockerng/dns_reply.logshows the progress during DNS benchmark test. There are not that many DNS requests, a couple per seconds, and that is not much.
I've got another test for you :
cat /var/log/resolver.log | grep 'start'How often does your unbound restart ? per hour / per day ?
-
Here are the restarts from Unbound. I did some testing and restarting but nothing I think it looks good:
Apr 5 16:58:59 odin unbound[81926]: [81926:0] info: start of service (unbound 1.13.1). Apr 5 17:15:14 odin unbound[70024]: [70024:0] info: start of service (unbound 1.13.1). Apr 5 17:16:22 odin unbound[57071]: [57071:0] info: start of service (unbound 1.13.1). Apr 6 11:38:11 odin unbound[11398]: [11398:0] info: start of service (unbound 1.13.1). Apr 6 11:43:22 odin unbound[51771]: [51771:0] info: start of service (unbound 1.13.1). Apr 6 11:57:55 odin unbound[9894]: [9894:0] info: start of service (unbound 1.13.1). Apr 6 11:58:37 odin unbound[78660]: [78660:0] info: start of service (unbound 1.13.1). Apr 6 12:01:13 odin unbound[7024]: [7024:0] info: start of service (unbound 1.13.1). Apr 6 12:15:23 odin unbound[78621]: [78621:0] info: start of service (unbound 1.13.1). Apr 6 12:23:14 odin unbound[93856]: [93856:0] info: start of service (unbound 1.13.1).pfBlockerNG is not blocking those domains now and when I tail the unbound.conf I see this:
[2.5.0-RELEASE][admin@odin.vXXXXXX.lan]/root: tail -n 3 /var/unbound/unbound.conf # Python Module python: python-script: pfb_unbound.py [2.5.0-RELEASE][admin@odin.vXXXXXX.lan]/root:In the dns_reply.log I can see all the queries when using the GRC tool. Looks to me like a lot. I restart Unbound service to make sure that pfBlockerNG is blocking those domains. Here is the output when everything is working fine and the domain www.conduit.com is being blocked:
DNS-reply,Apr 6 12:46:02,reply,A,SOA,3600,www.conduit.com.vXXXXX.lan,192.168.100.25,SOA,unk DNS-reply,Apr 6 12:46:02,reply,AAAA,SOA,3600,www.conduit.com.vXXXXX.lan,192.168.100.25,SOA,unk DNS-reply,Apr 6 12:46:02,servfail,AAAA,AAAA,Unk,www.conduit.com,192.168.100.25,ServFail,unkThen I run GRC tool and see a lot of requests going by. Now when I query www.conduit.com I get an IP in nslookup and I see the following output and pfBlockerNG is broken:
DNS-reply,Apr 6 12:48:40,cache,A,A,3442,www.conduit.com.vXXXXXX.lan,192.168.100.25,NXDOMAIN,unk DNS-reply,Apr 6 12:48:40,reply,AAAA,SOA,3442,www.conduit.com.vXXXXXX.lan,192.168.100.25,SOA,unk DNS-reply,Apr 6 12:48:40,cache,A,A,86385,www.conduit.com,192.168.100.25,198.49.162.233,IL DNS-reply,Apr 6 12:48:40,servfail,AAAA,AAAA,Unk,www.conduit.com,192.168.100.25,ServFail,unk -
I see what you mean :
@vjizzle said in pfBlockerNG not blocking everything in blocklists:
DNS-reply,Apr 6 12:48:40,cache,A,A,86385,www.conduit.com,192.168.100.25,198.49.162.233,IL
dono where and when unbound (?) resolves "www.conduit.com".
Check these files :
dnsbl_error.log (mine is empty)
py_error.log (mine is empty)
dnsbl_parsed_error.log (some data, not related to this issue) -
@gertjan
Yes for some reason every time I run the GRC lookup tool and query my pfSense, pfblockerng is broken afterwards. I can reproduce it every time and when I restart unbound server it is fixed.I am using unbound on pfSense in resolver mode (forwarding option is disabled) and I have no DNS server entries in General Setup tab in pfSense. I have set the option DNS Resolution Behavior to Use local DNS (ignore remote DNS servers).
py_error.log is empty.
dnsbl_error.log shows a few entries[ DNSBL_Abuse_CH - Abuse_ch_hostfile ] Download Fail Firewall and/or IDS (Legacy mode only) are not blocking download. Restoring previously downloaded file [ DNSBL_Abuse_CH - Abuse_ch_hostfile ] Download Fail Firewall and/or IDS (Legacy mode only) are not blocking download. Restoring previously downloaded filednsbl_parsed_error.log has some entries like these:
04/6/21 14:16:12,No_Tracking_Hostnames,from,:: from 04/6/21 14:16:12,No_Tracking_Hostnames,hostwinds,0.0.0.0 hostwinds 04/6/21 14:16:12,No_Tracking_Hostnames,hostwinds,:: hostwinds 04/6/21 14:16:30,MS_2,beritanowtk,beritanowtk -
Hummm.
Everything looks fine to me / identical to me.
Except : I can run that GRC tool as hard as I can, it won't break 'unbound'.
Did you try the old method "unbound mode" ?
-
@gertjan
Good tip @Gertjan. I immediately tested it and low-and-behold. Everything keeps working even if I bombard pfsense with the GRC benchmarking tool. Here the output before and after the GRC testing:> server 192.168.100.1 Default Server: odin.vXXXX.lan Address: 192.168.100.1 > www.conduit.com Server: odin.vXXXXX.lan Address: 192.168.100.1 Name: www.conduit.com Address: 10.10.10.1 > www.vmn.net Server: odin.vXXXXX.lan Address: 192.168.100.1 Name: www.vmn.net Address: 10.10.10.1 > www.ku6.com Server: odin.vXXXXX.lan Address: 192.168.100.1 Name: www.ku6.com Address: 10.10.10.1 >Looks like there is something the matter with the python integration here. Also as an added bonus in Unbound mode I can see that pfSense is now faster in answering queries compared to the AdGuard home server I have running here. In python mode pfSense was always slower:
Not that the speed difference is that big but still ^^. So in the end something is broken here. I will initiate a reinstall tomorrow unless someone else has another good idea. Thank you very. much for helping Gertjan!
-
Ok I just did a clean install of pfsense and then restored a backup. I have the exact same behavior again. Python mode pfblockerng stops working after I hammer it with GRC DNS benchmark.
For now I am switching over to AdGuard Home. Maybe better luck with pfSense in the next release :(
-
I'm at home now.
Did some testing.
The dns benchmark running, doing it thing.In a console/SSH, I ran this :
tail -f /var/unbound/var/log/pfblockerng/dns_reply.log | grep 'conduit'According the https://dbl.oisd.nl/ feed ( the oisd.nl feed) :
,www.conduit.com,,0,OISD,DNSBL_Compilationand also :
,conduit.com,,0,MS_2,DNSBL_BBcan177I was spamming
www.conduit.comin a nslookup (Windows 10) session.
Suddenly, during the test :
...... > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0 > www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Nom : www.conduit.com Address: 0.0.0.0When the 0.0.0.0 came back, I saw this in the 'tail' console :
DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:08,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:09,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:10,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unk DNS-reply,Apr 6 23:59:11,servfail,AAAA,AAAA,Unk,www.conduit.com,2001:470:1f13:65c::2001,ServFail,unkand then :
> www.conduit.com. Serveur : priv.my-domain.net Address: 2001:470:xxxx:65c::1 Réponse ne faisant pas autorité : Nom : www.va.conduit.com Address: 198.49.162.233 Aliases: www.conduit.comDNS-reply,Apr 6 23:59:12,cache,A,A,86399,www.conduit.com,2001:470:xxxx:65c::2001,198.49.162.233,IL198.49.162.233 popped up.
I guess I can confirm what you saw.
Btw :
I saw :
Assembling DNSBL database...... completed [ 04/7/21 00:05:47 ] TLD: TLD analysis..xxxxxxxxxxx completed [ 04/7/21 00:05:51 ] ** TLD Domain count exceeded. [ 200000 ] All subsequent Domains listed as-is **during the force update. The OISD feed is BIG : 1,2 million domain names.
I'll test tommorow agagin without the OISD feed, as the base domain name itself "conduit.com" is already in the BBCan 77 feed - as shown above.
edit : without the oisd.nl feed, and thus a rather (for me) small set of blocked domains, env 16962, the "0.0.0.0" blocking of "conduit.com" works.
www.conduit.com is resolved..... -
Interesting! So maybe I am on to something :p.
I got curious and did the test with the BBCan77 feed. The domain conduit.com is indeed there. So I decided to disable the oisd.nl feed and force reload pfblockerNG. After that I bombarded pfSense with the DNS benchmarking tool. Guess what...everything keeps running fine for conduit.com. Multiple reloads of pfBlockerNG, multiple restart of Unbound but everything kept working as expected:
> conduit.com Server: odin.vXXXXX.lan Address: 192.168.100.1 Name: conduit.com Address: 0.0.0.0Then I enabled the https://dbl.oisd.nl feed did a force reload of pfBlockerNG. I don't have TLD blocking enabled so I don't get that message you have. You can see the log about loading the oisd.nl list:
[ dbl_list ] Downloading update .. 200 OK. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 1251871 1251871 154015 0 0 1097856 ---------------------------------------------------------------------- Saving DNSBL statistics... completed [ 04/7/21 00:31:48 ] ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 04/7/21 00:31:49 ] Reloading Unbound Resolver (DNSBL python). Stopping Unbound Resolver. Unbound stopped in 2 sec. Additional mounts (DNSBL python): No changes required. Starting Unbound Resolver... completed [ 04/7/21 00:31:53 ] Resolver cache restored DNSBL update [ 1379690 | PASSED ]... completedAnd of course now pfBlockerNG is broken and not blocking the domain anymore BUT conduit.com keeps getting blocked by pfblockerng fine:
> server 192.168.100.1 Default Server: odin.vXXXXXX.lan Address: 192.168.100.1 > conduit.com Server: odin.vXXXXXX.lan Address: 192.168.100.1 Name: conduit.com Address: 0.0.0.0 > www.conduit.com Server: odin.vXXXXXX.lan Address: 192.168.100.1 Non-authoritative answer: Name: www.va.conduit.com Address: 198.49.162.233 Aliases: www.conduit.com > www.ku6.com Server: odin.vXXXXXX.lan Address: 192.168.100.1 Non-authoritative answer: Name: 3a1d905b3d90ef4e-cl2.qcloudcjgj.com Address: 118.89.206.86 Aliases: www.ku6.com fa2a686c758fa3d110164df993a0b8c4.qcloudzygj.com > www.vmn.net Server: odin.vXXXXXX.lan Address: 192.168.100.1 Non-authoritative answer: Name: www.vmn.net Address: 54.146.142.3 >So www.conduit.com is broken but conduit.com is being blocked fine. Even after I run the DNS benchmark tool multiple times conduit.com keeps working.
I am glad you are seeing the same behavior like me. Looks like something is crashing when stressed with multiple (and fast) DNS queries .
-
Same test, back on work.
Home is a 1 Mbytes 'small' VM, work use its own i5, with 8 Mbyyes - although I don't think this matters.First test :
nslookup with full debug :set db2> www.conduit.com. Serveur : pfsense.my-work-domain.net Address: 2001:470:xxxx:5c0:2::1 ------------ SendRequest(), len 33 HEADER: opcode = QUERY, id = 20, rcode = NOERROR header flags: query, want recursion questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = A, class = IN ------------ ------------ Got answer (49 bytes): HEADER: opcode = QUERY, id = 20, rcode = NOERROR header flags: response, auth. answer, want recursion, recursion avail. questions = 1, answers = 1, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = A, class = IN ANSWERS: -> www.conduit.com type = A, class = IN, dlen = 4 internet address = 0.0.0.0 ttl = 60 (1 min) ------------ ------------ SendRequest(), len 33 HEADER: opcode = QUERY, id = 21, rcode = NOERROR header flags: query, want recursion questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = AAAA, class = IN ------------ ------------ Got answer (33 bytes): HEADER: opcode = QUERY, id = 21, rcode = SERVFAIL header flags: response, want recursion, recursion avail. questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = AAAA, class = IN ------------ Nom : www.conduit.com Address: 0.0.0.0Fine, "www.conduit.com" yields 0.0.0.0
Now I launch a DNS benchmark, and have it finished (30 seconds later)
I "nslookup" again :
> www.conduit.com. Serveur : pfsense.my-work-domain.net Address: 2001:470:xxxx:5c0:2::1 ------------ SendRequest(), len 33 HEADER: opcode = QUERY, id = 22, rcode = NOERROR header flags: query, want recursion questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = A, class = IN ------------ ------------ Got answer (70 bytes): HEADER: opcode = QUERY, id = 22, rcode = NOERROR header flags: response, want recursion, recursion avail. questions = 1, answers = 2, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = A, class = IN ANSWERS: -> www.conduit.com type = CNAME, class = IN, dlen = 9 canonical name = www.va.conduit.com ttl = 86227 (23 hours 57 mins 7 secs) -> www.va.conduit.com type = A, class = IN, dlen = 4 internet address = 198.49.162.233 ttl = 86227 (23 hours 57 mins 7 secs) ------------ Réponse ne faisant pas autorité : ------------ SendRequest(), len 33 HEADER: opcode = QUERY, id = 23, rcode = NOERROR header flags: query, want recursion questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = AAAA, class = IN ------------ ------------ Got answer (33 bytes): HEADER: opcode = QUERY, id = 23, rcode = SERVFAIL header flags: response, want recursion, recursion avail. questions = 1, answers = 0, authority records = 0, additional = 0 QUESTIONS: www.conduit.com, type = AAAA, class = IN ------------ Nom : www.va.conduit.com Address: 198.49.162.233 Aliases: www.conduit.comNote the difference :
www.conduit.com resolves to the CNAME "www.va.conduit.com" (domain not listed), and that one resolves to "198.49.162.233" -
Hey Gertjan. This is correct about eventually resolving to a CNAME. I can see it on my end also that after running the GRC benchmark testing tool the domain resolves to a CNAME (www.va.conduit.com) and so pfblockerNG is not blocking it anymore. I think the same thing is happening with "www.ku6.com" and "www.vmn.net".
Can we conclude that the CNAME blocking on pfBlockerNG is not working as expected?
Because the same test on AdguardHome server always gets the domains blocked. I think that is also the reason why I am getting the low percentage in the pfblockerNG dashboard widget (AdguardHome is at 10% blocked DNS requests at the moment).I also think the same thing is happening with other domains as well. Just found out that "diagnostics.meethue.com" gets blocked but after benchmarking with GRC it is not blocked anymore.
I have never seen "diagnostics.meethue.com" in the pfblockerNG DNSBL Block Stats tab being in the list. After running my whole network for 24 hrs on AdGuardhome it is the domain being blocked the most. -
Thanks for the report! Its appreciated!
The issue is that GRC is using a mixture of lowercase and uppercase in their DNS Tests.
I have fixed the code to force everything to lowercase before the domains are validated.
A fix can be downloaded here until the next version is available:
curl -o /var/unbound/pfb_unbound.py "https://gist.githubusercontent.com/BBcan177/1e400f170b7ef7e8cb76b548d6524b31/raw"Then Restart Unbound
Thanks!
-
Thank you for being so fast in fixing this and the great work you are doing with pfBlockerNG!
With the modifications all is working well :).
