@smolka_J said in Download failed for certain Lists "PFB_FILTER - 17": @Beerman I stumbled upon this same error on a dozen er so feeds when I recently upgraded to 24.03. Mr BBcan177 posted a temporary workaround on https://forum.netgate.com/topic/187931/pfblockerng-v3-2-0_10-unable-to-download-txt-blocklists until the next stable or long term fix is implemented. Editing /usr/local/pkg/pfblockerng/pfblockerng.inc file around line #259, added a line in the list of mime types adding 'text/x-file', and then running an Update>Force>Reload ALL takes care of this error for now until if pfBlockerNG package version 3.2.0_10 is re-installed, it does survive reboots I have the error since July of this year, and the log sadly gives no indication of which list it might be, I checked the file and the line is already there, its a shame but pfblockerng feels like its just breaking more and more, I have been fixing multiple issues at this point manually on it.

@Patch The Spamhaus lists were combined and converted to the new json format in the latest pfB update (25.07). Agree on Talos, may need a redmine...?

Based on the Timings it seams that KEA stuff stopped the Unbound and don't start it again? or just coincidence? i currently not understand by kea is involved, because i didn't have activated dhcp lease to dns ... (DNS Registration) But its also strange the unbound gets two time restarted on pfblockerng cron-task General Log: Sep 28 00:00:00 php 12862 [pfBlockerNG] Starting cron process. Sep 28 00:00:31 php 12862 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 01:00:00 php 71023 [pfBlockerNG] Starting cron process. Sep 28 01:00:14 php 71023 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 02:00:00 php 5372 [pfBlockerNG] Starting cron process. Sep 28 02:00:22 php 5372 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 03:00:00 php 37595 [pfBlockerNG] Starting cron process. Sep 28 03:00:31 php 37595 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 03:01:00 root 31193 rc.update_bogons.sh is starting up. Sep 28 03:01:00 root 32722 rc.update_bogons.sh is sleeping for 66003 Sep 28 04:00:00 php 49188 [pfBlockerNG] Starting cron process. Sep 28 04:20:17 php 49188 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 05:00:00 php 29707 [pfBlockerNG] Starting cron process. Sep 28 05:05:16 php 29707 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload Sep 28 06:00:00 php 3228 [pfBlockerNG] Starting cron process. Sep 28 06:00:00 php-cgi 4046 [Suricata] Checking for updated MaxMind GeoLite2 IP database file... Sep 28 06:01:00 php-cgi 4046 [Suricata] ERROR: GeoLite2-Country IP database download failed. The HTTP Response Code was . Sep 28 06:01:00 php-cgi 4046 [Suricata] ERROR: GeoLite2-Country IP database update check failed. The GeoIP database was not updated! Sep 28 06:01:00 php-cgi 4046 [Suricata] Cleaning up temp files after GeoLite2-Country database update. Sep 28 06:05:16 php 3228 [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload DNS-Resolver Log: Sep 28 00:00:23 unbound 27650 [27650:0] info: service stopped (unbound 1.23.0). Sep 28 00:00:23 unbound 27650 [27650:0] info: [pfBlockerNG]: pfb_unbound.py script exiting Sep 28 00:00:24 unbound 29770 [29770:0] notice: init module 0: python Sep 28 00:00:24 unbound 29770 [29770:0] info: [pfBlockerNG]: pfb_unbound.py script loaded Sep 28 00:00:24 unbound 29770 [29770:0] info: [pfBlockerNG]: init_standard script loaded Sep 28 00:00:24 unbound 29770 [29770:0] notice: init module 1: validator Sep 28 00:00:24 unbound 29770 [29770:0] notice: init module 2: iterator Sep 28 00:00:24 unbound 29770 [29770:0] info: start of service (unbound 1.23.0). Sep 28 00:05:46 unbound 29770 [29770:0] info: service stopped (unbound 1.23.0). Sep 28 00:05:46 unbound 29770 [29770:0] notice: Restart of unbound 1.23.0. Sep 28 00:05:46 unbound 29770 [29770:0] info: [pfBlockerNG]: pfb_unbound.py script exiting Sep 28 00:05:46 unbound 29770 [29770:0] notice: init module 0: python Sep 28 00:05:46 unbound 29770 [29770:0] info: [pfBlockerNG]: pfb_unbound.py script loaded Sep 28 00:05:47 unbound 29770 [29770:0] info: [pfBlockerNG]: init_standard script loaded Sep 28 00:05:47 unbound 29770 [29770:0] notice: init module 1: validator Sep 28 00:05:47 unbound 29770 [29770:0] notice: init module 2: iterator Sep 28 00:05:47 unbound 29770 [29770:0] info: start of service (unbound 1.23.0). Sep 28 03:13:53 unbound 29770 [29770:0] info: service stopped (unbound 1.23.0). DHCP Log: Sep 28 03:13:53 kea2unbound 88766 Unbound reloaded: /var/unbound/unbound.conf Sep 28 03:13:53 kea2unbound 88766 Include updated: /var/unbound/leases/leases4.conf (a7cfad6c13eb8df1) Sep 28 03:13:53 kea2unbound 88766 Unbound lease include is missing or inconsistent: /var/unbound/leases/leases4.conf Sep 28 00:05:46 kea2unbound 11822 Unbound reloaded: /var/unbound/unbound.conf Sep 28 00:05:46 kea2unbound 11822 Include updated: /var/unbound/leases/leases4.conf (a7cfad6c13eb8df1) Sep 28 00:05:46 kea2unbound 11822 Unbound lease include is missing or inconsistent: /var/unbound/leases/leases4.conf

The commit didn't work for me. pfBlockerNG fullfilling the pfsense config history, reverting the patch now.

@tinfoilmatt I don't understand, where and what to add?

@jrey those pfb_*.sh files are the executables Thanks for clarifying the executables. I do not think that the hardware and table entry size won't be problem as the current configuration provides sufficient computation and memory for its operations. Pfsense is running in different hardware, e.g., Intel Xenon Platinum 8272CL processor and maximum table entries is set to 10 times than the default. pfSense Table Stats ------------------- table-entries hard limit 4000000 Table Usage Count 465931

@High_Voltage I don't think there is anything to do.. Pretty sure those are the defaults.. When you query pihole for something that is blocked.. By default it returns some sort of answer - I am not aware of a setting that would wait for a timeout. [image: 1757768975895-blockmode.jpg] So I really have no idea he would of been doing. If something is blocked in pfblocker you would either get back the vip (so you could see a block page) or you would get all zeros. Same goes for pihole - I am not aware of a setting that would just time out and not send an answer if you asked for something that was blocked. The only reason pihole would time out on sending you a response is what you were asking for actually just never responded to pihole. If some fqdn you were asking for was being blocked you would get the answer almost instantly. If pihole was forwarding to unbound which I believe is a common sort of setup for stuff that is not blocked. Again if pfblocker was blocking it, you for sure should get a response right away of either the vip or all zeros. Its possible maybe pihole doing rebind protection, and pfblocker handing back the rfc1918 of your vip maybe causes pihole some sort of hangup, but normally when that happens it just returns a null to the client since it got an answer, it just not suppose to hand it back to the client.. But again no time out. Only time I could see a timeout issue is when unbound didn't answer the pihole, again if something unbound doesn't get an answer from how its resolving/forwarding. Maybe he was sending back nodata, and the client didn't take that as an answer and kept asking for the same thing until it gave up?

@marchand.guy you understand that an update would of cleared the cache - so for sure the numbers would be lower after this. Wait a few days to let your normal browsing habits stabilize.

@Gertjan Thanks for your reply – that’s also my impression. The point is: I don’t really see any lists right now that are actually “maintained” in the sense of being actively cleaned up, checked for dead domains, categorized, etc. That’s why my main interest is more about the demand: Would curated lists really be a game changer for admins? Would they be more helpful than what’s available today, or are most people already using other alternatives? If so, which ones? And from your perspective, what would be your expectation towards “community lists”? (e.g. reliability, update frequency, categories, fewer false positives?)

@BiloxiGeek said in DNSBL and IPv6: Does it just follow the IPv4 address that is listed above that? In my case it would end up being ::10.0.0.86 Yes. In this specific context that's the notation being used. (Full IPv6 web server address, for reference then, would be: http://[0000:0000:0000:0000:0010:0000:0000:0086]) Nota bene: I use 0.0.0.0 which renders the DNSBL webserver useless and inaccessible, but otherwise returns 0.0.0.0 or ::/NOERROR answers to all blocked lookups.

@mOrbo O.k. i see. Under such circumstances i would also stay on the internal DNS. Well just give it a try with @BBcan177 said in PFBlockerNG Python-Mode - Source-IP in Reports: For Python mode, when you use an internal dns server, you can either null block or check the option "DNSBL Event Logging", which will provide a workaround for this issue. So as far i remember, it did not work with Python mode and DNSBL Null block (logging). But i surely did not test it with checking "DNSBL Event Logging" and DNSBL Webserver / VIP.

@jrey Would you mind sharing a bit about that setup? I understand your reluctance to promise anything if you are looking into pfblockerNG package maintenance.

@Arowe95 How have you set up pfBlockerNG? For me if I do a basic setup using the Wizard the Steve Black Hosts list is already included. That would explain the duplicates :). Check Firewall / pfBlockerNG / DNSBL / DNSBL Groups, ADs_Basic. Click edit for that group and it contains one list, Steve Black Hosts.

@LowKnee Just out of curiosity are you referring to the Database Sanity Check reporting that "these two counts should match" it the count is off by 1 (which I suspect is your case) there was a fix (manual code change) to change masterfile to mastercat in pfblolckerng.sh you want to change this change the line from s1="$(grep -cv ^${ip_placeholder2}$ ${masterfile})" to s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})" There is also an edge case if the count is greater than one, here is how that goes if in the deny directory you have say two flies (because of the list / file selection you have and they have repeat addresses file 1 has say 100 lines file 2 has say 10 lines (but those 10 lines are also in file 1, file 2 is a subset) you get two uniquely named deny files and then when the "count" is calculated on the deny directory it sees 110 entries when the "count:" is calculated on the "mastercat" file it only contains 100 entries the count doesn't match in my case the issue was caused by full list I had selected, also having an available subset lists (I had inadvertently selected one of) this causing two deny files with some of the same (overlapping data) I unselected the subset and bingo matched again, was a "my bad" selection. Edit: this applied to 25.07 (and 25.07.1) and pfblockerng 3.2.7 as it is labelled on those versions of pfSense

@tinfoilmatt said in Failed or invalid Mime Type: [application/SIMH-tape-data|0]: (ASN data is IPinfo, not Maxmind) Thats correct but "GeoLite2-Country" is from Maxmind ... (that confused me) I'm considering simply adding "application/SIMH-tape-data" to the list to test. Thats what i tought too ... I'll try when I have the time for it ... Edit: I can confirm - adding "application/SIMH-tape-data" to the list at line 257 in /usr/local/pkg/pfblockerng/pfblockerng.inc did the trick - no more error! Edit: OK, problem resolved but I would like to know, whats the cause for that error! (SIMH-tape-data sounds like a "blast from the past" ...) Thanks a lot!

@jrey Thank you so much for the detailed explanation and help. I will adapt and apply the patch to move the job timing like suggested at 01:35 Are you just a user or are you also involved in package maintenance on one or more packages?

@andrebrait will you be able to rebase pfblockerng-adblock-clean on top of devel in the foreseeable future? I have been able to make use of patches until I upgraded to 25.07-RELEASE. The conflicts are deep. Oddly the pfblockerNG-devel package is 3.2.7 despite the current refs having 3.2.9 in the Makefile.

I finally got around installing a new pfsense firewall and the first connections I am seeing right of the bat are lets say strange. I don't know what they are: https://otx.alienvault.com/indicator/ip/178.250.1.11 https://www.abuseipdb.com/check/178.250.1.11 https://www.virustotal.com/gui/ip-address/178.250.1.11/community https://viz.greynoise.io/ip/178.250.1.11 Aug 10 11:07:09 WAN Default deny rule IPv4 (1000000103) 178.250.1.11:443 192.168.178.21:18414 TCP:PA Its incoming from WAN trying to get to the firewall. Very mixed results here. Never heard of criteo and it is flagged by some people despite being whitelisted on otx alienvault. I remember seeing the same the first time I installed pfsense on my other machine I think. Any idea what it could be? I also did a packetcapture and there are losts of ACKed Unseen segments. Does this indicate anything? On my other firewall I don't see anything coming from WAN to LAN but on the new one there is so much IPs. What can it be?

So you're using the CARP IP address for the pfBlockerNG redirects? May I ask why that's necessary?