pfBlockerNG-devel v3.0.0_9

BBcan177

A Pull request has been submitted to the pfSense devs for approval. Hope to have this released today.

Continue to follow in the pfSense forum and on Twitter [ u/BBcan177 ], Reddit [ r/pfBlockerNG ]
and Patreon ( https://www.patreon.com/pfBlockerNG ) for pfBlockerNG news and support.

Thank you for the Support!

Link to PR#
https://github.com/pfsense/FreeBSD-ports/pull/1035

Showing with 4,151 additions and 1,820 deletions.

CHANGE LOG:

Add a Unified Log Report (ip_deny.log, ip_permit.log, ip_match.log, dnsbl.log, dns_reply.log)
Refactored Reports tab to utilize the new Unified Log, Add additional Report Settings, and Improve Alert Filtering
Add an IP Cache sqlite3 DB to improve the loading of the Reports tab and more efficient to log repeated IP events
Add additional DoH/DoT DNS Servers that can be blocked (SafeSearch Tab)
DuckDuckGo / Pixabay use CNAME for SafeSearch
DNSBL Global Logging/Blocking option which will override all DNSBL Logging/Blocking settings.
Clog is removed from pfSense 2.5 and above. Add additional validation to switch to Tail when pfSense is upgraded to pfSense 2.5.
Utilize non-zero padded Day format for all log events. (IE: Feb 04 vs Feb 4) (External Syslog parsers might need to be reviewed)
Reports tab - add a DNSBL Cache sqlite3 DB to improve the loading of the Reports tab
Reports tab - Show DHCPv6 Hostnames (contributed by Gertjan)
Fix issue that would cause Unbound to restart during CRON/Force CMD events when DNSBL was disabled.
BGPView seems to be rate-limiting and causing connectivity issues. On failure, record the Cloudflare response to the ASN download.
See: https://twitter.com/BBcan177/status/1357161876812087297
DNSBL Default Block page - Improvements to Blocked Feed/Group reporting
Widget - Click on widget title will open new Unified Log page

Feeds:
Removed: Malware Domain List, BadIPs,
Added: FireBog - 5 New DNSBL Groups
https://github.com/pfsense/FreeBSD-ports/pull/982

Unbound Mode Changes:

Safe Search in Unbound mode, add safety belts to prevent TLD Blacklist entries from conflicting with DNSBL blocked domains. When SS is enabled, it will not allow any SS TLDs to be TLD Wildcard blocked.
When the DNSBL Interface is set to use Localhost, Lighttpd will be bound to the DNSBL VIP address (and port 80/443) instead of Localhost. There are no NAT Rules created in this scenario.

Unbound Python Mode Changes:

Workaround Unbound regressions for callbacks to allow for the logging of the Query IP
SafeSearch, utilize the Python integration instead of the traditional Unbound local-data/local-zone entries.
Add a DNSBL Cache sqlite3 DB to improve the loading of the Reports tab
Add Unbound Python_control feature. This will allow sending TXT records (only from pfSense localhost IP) to control DNSBL features. (Enable/Disable/Add Bypass, Remove Bypass)
noAAAA, allow domains to be wildcard noAAAA by prefixing a "." before the domain in the noAAAA Customlist.
Log noAAAA events in the logs
Fix issue with CNAME validation and improve logging to show both the Domain and CNAME
Add Threat Lookup query to DNS Reply events
Add Domain to DNSBL Customlist for DNS Reply Events
Fix issue with TLD Allow and sort option
Log RRcode result on DNS reply logging resolution failures
Fix issue with DNSBL IDN Blocking option always enabled
Add Suffix to DNSBL Modes (TLD/DNSBL) ie: _A, _AAAA, _CNAME

jdeloach

@bbcan177

Just updated from v3.0.0_8 to v3.0.0_9 with no issues other than I notice that I still had to manually restart Unbound. I don't consider that a problem, just a minor nuisance. I didn't see any errors and everything appears to be working as before.

Good update. Thanks for all your hard work in supporting this package.

Bob.Dig

Thank you @BBcan177

There was again one problem with the Description for one feed.

jdeloach

@bob-dig

That feed downloads for me when I click the link in the Feeds list. Maybe their servers were down/offline/overloaded when you tried it.

Have you tired downloading it again?

edit: That list looks like it is not formatted correctly for pfBlockerNG_devel? Looks to have a lot of email addresses in it, never seen a feed list with email addresses in it.

Bob.Dig

@jdeloach I renamed the description, then it worked. I don't think it downloads anything if you just add feeds, but I could be wrong.

fireodo

@bbcan177

Hello,

after update from 3.0.0_8 to 3.0.0_9 unbound brings this warning: "warning: duplicate local-zone use-application-dns.net."
I have the settings "DoH/DoT Blocking" on for Firefox only. If I deactivate that the warning disappeared.

PS. I found the culprit: it was a "pfb_dnsbl.firefoxdoh.conf" file that leftover from the previous version in /var/unbound/!
I let this post here for who may run into the same issue.

Thanks und a good weekend,
fireodo

@bbcan177 said in pfBlockerNG-devel v3.0.0_9:

When the DNSBL Interface is set to use Localhost, Lighttpd will be bound to the DNSBL VIP address (and port 80/443) instead of Localhost. There are no NAT Rules created in this scenario.

This one solves many Problems when using pfBlckerNG over VPN.

BBcan177

@fireodo said in pfBlockerNG-devel v3.0.0_9:

after update from 3.0.0_8 to 3.0.0_9 unbound brings this warning: "warning: duplicate local-zone use-application-dns.net."
I have the settings "DoH/DoT Blocking" on for Firefox only. If I deactivate that the warning disappeared.

When you enable the Firefox option it will add "application-dns.net" to your Unbound configuration. I assume that you also manually added the same line to the DNS Resolver Adv. Settings. So you can't do both at the same time, hence the "Duplicate zone" error.

fireodo

@bbcan177 said in pfBlockerNG-devel v3.0.0_9:

@fireodo said in pfBlockerNG-devel v3.0.0_9:

after update from 3.0.0_8 to 3.0.0_9 unbound brings this warning: "warning: duplicate local-zone use-application-dns.net."
I have the settings "DoH/DoT Blocking" on for Firefox only. If I deactivate that the warning disappeared.

When you enable the Firefox option it will add "application-dns.net" to your Unbound configuration. I assume that you also manually added the same line to the DNS Resolver Adv. Settings. So you can't do both at the same time, hence the "Duplicate zone" error.

I found the file "pfb_dnsbl.firefoxdoh.conf" in /var/unbound that was not deleted when i updated and thats why I got that entry twice. (I had "pfb_dnsbl.doh.conf" AND "pfb_dnsbl.firefoxdoh.conf")

Thanks,
fireodo

BBcan177

@fireodo said in pfBlockerNG-devel v3.0.0_9:

I found the file "pfb_dnsbl.firefoxdoh.conf" in /var/unbound that was not deleted when i updated and thats why I got that entry twice. (I had "pfb_dnsbl.doh.conf" AND "pfb_dnsbl.firefoxdoh.conf")

Ok Thanks for reporting, I will check it out asap.

scorpoin

@bbcan177 Thanks for updates , well I have a suggestion can we have an option for white listing the specific website instead of white list ingthe ip for entire list /websites.

Regareds

giminik

Is it safe to use this devel version in production?

@giminik In this case yes. The dev version is the version being updated with new features and bug fixes. Think of the non dev version as legacy.

I won't speak for the developer, @BBcan177, but I think the extended development cycle for pfSense 2.5 is at the root of this unusual naming situation and that it will be resolved when 2.5 is released.

fireodo

@bbcan177

There is something else I do not really understand:
I have only 1 TLD in the DNSBL Whitelist and in the "DNS over HTTPS/TLS Blocking" I have Firefox an 4 other server highlighted.
When I disable the "DNS over HTTPS/TLS Blocking" I have 3 entries in the "pfbdnsblsuppression.txt" (accordingly 3 in the widget). When I activate "DNS over HTTPS/TLS Blocking" I got 179 entries in the pfbdnsblsuppression.txt regardless what servers I highlighted in the "DNS over HTTPS/TLS Blocking". When I look in the pfbdnsblsuppression.txt all servers are in the whitelist regardless what I have chosen in "DNS over HTTPS/TLS Blocking".

Can you please explain me that behavior a simple as possible?

Thanks a lot for you excellent work!
fireodo

BBcan177

@fireodo said in pfBlockerNG-devel v3.0.0_9:

There is something else I do not really understand:
I have only 1 TLD in the DNSBL Whitelist

First, click on the Blue infoblock Icons for the TLD Blacklist and TLD Whitelist.

The TLD Blacklist is used to block a whole TLD like "ru" or "top" etc
The TLD Whitelist, is used to allow a domain that is being TLD Blacklisted. IE: "example.ru". The TLD Whitelist is not required for Unbound Python mode, as those domains can now be whitelisted in the same fashion as all other whitelisting.

The TLD Whitelist, is not same thing as the DNSBL Whitelist.

The DNSBL Whitelist is where you want to add domains to be whitelisted, and its best to whitelist by clicking on the "+" icon in the Reports tab as those take effect immediately. Adding a domain manually to the whitelist, will require a Force Reload - DNSBL to take effect.

Can you please explain me that behavior a simple as possible?
Thanks a lot for you excellent work!

Thanks, and hope that helps!

fireodo

@bbcan177
It is very much clearer now!

Thanks a lot,
fireodo

fireodo

@bbcan177 said in pfBlockerNG-devel v3.0.0_9:

@fireodo said in pfBlockerNG-devel v3.0.0_9:

The TLD Whitelist, is not same thing as the DNSBL Whitelist.

OK. But from where come those 181 Whitelist-Entrys shown in the pfblocker widget? (see atach)

I have nothing introduced deliberately in the DNSBL Whitelist.

RonpfS

@fireodo Click on it, it brings you to DNSBL Tab, how many entries do you have in Custom Whitelist?

Gertjan

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

how many entries do you have in Custom Whitelist?

Let me answer that question : 90 entries (close to).
This number is doubled (www is prepended for every host name if it isn't starting with www) and the final ",localhost.localdomain,," is added.

Source : /var/db/pfblockerng/pfbdnsblsuppression.txt

fireodo

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

@fireodo Click on it, it brings you to DNSBL Tab, how many entries do you have in Custom Whitelist?

Only 1 "ONE"

fireodo

@gertjan said in pfBlockerNG-devel v3.0.0_9:

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

how many entries do you have in Custom Whitelist?

Let me answer that question : 90 entries (close to).
This number is doubled (www is prepended for every host name if it isn't starting with www) and the final ",localhost.localdomain,," is added.

No, only 1!

Source : /var/db/pfblockerng/pfbdnsblsuppression.txt

The rest of the entrys in the file you mention are all the DoH/DoT Servers from the "DNS over HTTPS/TLS Blocking" -List ;-)

RonpfS

@fireodo said in pfBlockerNG-devel v3.0.0_9:

re all the DoH/DoT Servers from the "DNS over HTTPS/TLS Blocking" -List ;-)

Post you Whitelist and the pdbdnsblsuppresstion.txt so we can help.

fireodo

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

@fireodo said in pfBlockerNG-devel v3.0.0_9:

re all the DoH/DoT Servers from the "DNS over HTTPS/TLS Blocking" -List ;-)

Post you Whitelist and the pdbdnsblsuppresstion.txt so we can help.

Whitelist: youtube-nocookie.com

Suppresion List:
pfbdnsblsuppression.txt

RonpfS

@fireodo You are using Unbound mode or Python Unbound Mode ?

fireodo

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

@fireodo You are using Unbound mode or Python Unbound Mode ?

Unbound Mode.

Thanks

RonpfS

@fireodo I am with Unbound Python mode, so I can't verify the difference in file between mode.

But this may be normal,

fireodo

@ronpfs said in pfBlockerNG-devel v3.0.0_9:

@fireodo I am with Unbound Python mode, so I can't verify the difference in file between mode.

But this may be normal,

Hmmm, if I deactivate the DNS over HTTPS/TLS Blocking the Whitelist is reduced to 3 (in the pfblocker Widget - and also in the pfbdnsblsuppression.txt)