pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!
-
Thank you for the update and the effort you put into this project. This add-on is one of the main reasons I stick with pfSense.
We have a bunch of remote Sites running non-pfSense VPN Router with integrated DSL Modem. This remotes Sites are connected to our Data Centers by routed IPsec VPN, terminate on the pfSense where the DNS-Resolver with pfb enabled is hosted. Each site has a local resolver and forwards all queries for public domains to the pfSense in the Data Center over the VPN Tunnel.
@BBcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
DNSBL Web Server is now bound to localhost instead of 0.0.0.0
Unfortunately this change breaks the usage of pfBlockerNG over IPsec for good. I cannot get the web server to reply to connections which come in over VPN.
Out of the box it doesn't work at all but with a custom NAT config it can be make functional.
When the Web Server is bounded on all Interfaces it was possible make this work by placing NAT Rules before the system generated rules to redirect the Traffic to the High Ports of the DNSBL VIP.rdr pass on enc0 inet proto tcp from any to 10.255.255.12 port = http -> 10.255.255.12 port 8081 rdr pass on enc0 inet proto tcp from any to 10.255.255.12 port = https -> 10.255.255.12 port 8443
This was enough to get it working for Policy based VPN. If using VTIs for IPsec an addition floating rule with no state is necessary to allow the return traffic.
pass out quick on enc0 inet proto tcp from 10.255.255.12 to <net_vpn_all> port >= 1024 no state label "USER_RULE: Allow Stateless Replies from DNSBL VIP"
I do understand that this is a limitation of pfSense/FreeBSD VPN and not a pfBlockerNG problem. Anyway it is possible to make the binding of Web Server to all interface or loopback a configurable option for future release?
If not - does anybody have an idea how to get this work when the service is bound to a loopback? -
@Artes
I can add this as an option in the next release as time permits.
In the meantime, you can edit the code to revert to 0.0.0.0.
Edit the file:
/usr/local/pkg/pfblockerng/pfblockerng.incAt line #1097 there is a function that builds the Lighttpd configuration file:
function pfb_create_lighttpd() {
}In that function you can change the lines with "127.0.0.1" to "0.0.0.0"
and change "::1" to "::/0" for IPv6. (haven't tested that myself, so hoping that this is the correct IPv6 address)
In the same file:
Edit line #1308:
From:
'target' => '127.0.0.1',
To:
'target' => '0.0.0.0',Edit line #1322:
From:
'target' => '::1',
To:
'target' => '::/0',Hope this works for you, Thanks for using the package, and and any support is appreciated.
-
I get to this...and then it takes forever...
[ DoH ] Downloading update [ 11/25/20 20:02:51 ] . cURL Error: 35
OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to heuristicsecurity.com:443 Retry in 5 seconds...
. cURL Error: 35
OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to heuristicsecurity.com:443 Retry in 5 seconds...
. cURL Error: 35
OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to heuristicsecurity.com:443 Retry in 5 seconds...
.. unknown http status code | 0[ DNSBL_DoH - DoH ] Download FAIL [ 11/25/20 20:03:06 ]
Firewall and/or IDS (Legacy mode only) are not blocking download.Restoring previously downloaded file
No Domains Found! Ensure only domain based Feeds are used for DNSBL!
Assembling DNSBL database...... completed [ 11/25/20 20:03:19 ]
TLD:
TLD analysis.............................. completed [ 11/25/20 20:08:33 ]
TLD finalize....................... -
From the CHANGELOG :
FEEDS: Removed discontinued Heuristic Security DoH Feed
Remove this URL. Use the Feeds tab to update your Groups
-
Hi,
Do we have CNAME blocking now? Thanks and as always amazing job!
-
@vjizzle
Yes its there in the DNSBL Tab with the other Unbound Python mode features. -
@BBcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Edit the file:
/usr/local/pkg/pfblockerng/pfblockerng.inc
At line #1097 there is a function that builds the Lighttpd configuration file:
function pfb_create_lighttpd() {
}Editing this function works fine for me. There is no need to edit lines #1308 and #1322 - These are for the NAT Rules and should stay as they are.
Thank you!
-
This post is deleted! -
First of all, great job on this release!
I'm currently running 3.0.0_1 with 'Unbound python mode' enabled in DNSBL.
While viewing the Reports tab I saw that the If and Source columns show 'Unknown' within the DNSBL Python alert entry's.Any suggestion how the fix this?
-
@SachaH said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Reports tab I saw that the If and Source columns show 'Unknown' within the DNSBL Python alert entry's.
https://forum.netgate.com/topic/158597/interface-and-source-is-unknown-running-devel-3-0-1/5
-
@RonpfS Thanks
-
also https://www.reddit.com/r/pfBlockerNG/comments/k1137y/reports_dnsbl_python_unknown/
-
This post is deleted! -
and https://www.patreon.com/posts/nov-update-43770381
-
Thanks BBcan177 for your great work,
For info, get yellow icon for DNSBL widget due to following error.
2020-11-27 21:33:25,736|ERROR| [pfBlockerNG]: Failed get_q_name_qinfo: in method '_get_qname', argument 1 of type 'struct query_info *'
-
@HuskerDu
Do you have any of these options enabled in the DNS Resolver?DHCP Registration:
- Static DHCP
- OpenVPN Clients
If so, disable those options, clear the py_error log and Force Reload - DNSBL and see if the error returns.
-
@BBcan177
The options were removed after I've read the doc, but first launch after the install must have triggered the error.Thanks, clearing the log file solved the yellow icon "issue".
Regards,
-
with v3.0.0 load on my Firewall is significant reduced when running large Set of DNSBL Lists (~3 Million Domains).
I don't see the unbound-control process spinning up every couple of seconds anymore.
Well Done
-
Thank you for your effort!
Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.
This causes the backup machine holding as CARP master when the Master machine comes online again.
I'm running 3.0.0_2
-
@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Thank you for your effort!
Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.
This causes the backup machine holding as CARP master when the Master machine comes online again.
I'm running 3.0.0_2Did you update the package on all Nodes?
Did you set the SKEW on all nodes correctly?The package should not be syncing the SKEW to the other nodes.
See the relevant code here:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9349-L9352 -
@bbcan177
I got an alert today that an update from 3.0.0_2 => 3.0.0_3 was available. I had already updated to 3.0.0_3.
But my package manager now shows 3.0.0_2 as current and wants me to revert.
Any problem here? Thanks for reading.
-
@provels
No Idea. That is outside of the pkg as its part of the pfSense package manager. -
@bbcan177 After upgrading to 3.0.0_3 whitelisting domains from the list of blocked domains (e.g.
/pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2
) no longer worked - getting 504 Gateway Time-out errors after selecting the domain to whitelist and providing the options (wildcard or not etc.)The entire UI is unresponsive after that and I can get it back up again via
ssh
with/etc/rc.php-fpm_restart
. I have rolled back to 3.0.0_2 but I still have this issue.Any ideas on what the issue might be?
-
@bbcan177
Must have been a personal problem. Rebooted the FW, GUI still offered upgrade to _2 when _3 was already installed. Did pkg upgrade from shell, offered _2 to _3. Went ahead and installed, seems fine now. Go figure. -
@asmodeus said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
@bbcan177 After upgrading to 3.0.0_3 whitelisting domains from the list of blocked domains (e.g. /pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2) no longer worked - getting 504 Gateway Time-out errors after selecting the domain to whitelist and providing the options (wildcard or not etc.)
The entire UI is unresponsive after that and I can get it back up again via ssh with /etc/rc.php-fpm_restart. I have rolled back to 3.0.0_2 but I still have this issue.
Any ideas on what the issue might be?There are two different things here.
- Whitelisting
- Filtering the Alerts tab.
The URL that you posted is for Filtering events, not whitelisting? Write out the steps you took so I can try to reproduce.
It could just be a browser timeout issue if you are trying to view a lot of events? In Alert Settings, what did you set the DNSBL Events number to? -
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Thank you for your effort!
Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.
This causes the backup machine holding as CARP master when the Master machine comes online again.
I'm running 3.0.0_2Did you update the package on all Nodes?
Did you set the SKEW on all nodes correctly?The package should not be syncing the SKEW to the other nodes.
See the relevant code here:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9349-L9352Yes.
I'm running the latest version and My main node ovewrites the skew to the secondary one.
-
Ok I see the issue.
As reference:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9350-L9351Can you edit the file: /usr/local/pkg/pfblockerng/pfblockerng.inc, Lines 9350 and 9351
From:
// Remove CARP Advskew setting, if found if (isset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew'])) { unset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew']); }
To:
// Remove CARP Advskew setting, if found if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) { unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']); }
I missed adding ['config'][0] to each of those two lines... Sorry about that...
-
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
The URL that you posted is for Filtering events, not whitelisting? Write out the steps you took so I can try to reproduce.
Sure:
- Coming from the pfBlockerNG dashboard on my Status/Dashboard page, I use the alerts link to review blocked domains in a list. The link points to
/pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2
- Using this list (which takes a while to load) I want to whitelist a domain, so I click on the 'plus' sign to start the process.
- I specify wildcard no, add description no.
- Page loads for a while and ultimately I am getting the nginx timeout. (Even when this worked it takes 110 seconds or more to complete)
- The page I attempted to whitelist is NOT whitelisted.
- The entire pfsense UI is unresponsive at this point and I resolve by restarting php-fpm via ssh
The Log Settings are set to default (20000 lines), and the Alert Settings are set to default:1000 Stat Table entries and 100 Filter Limit. Are these the Alert Settings you were asking about?
- Coming from the pfBlockerNG dashboard on my Status/Dashboard page, I use the alerts link to review blocked domains in a list. The link points to
-
@asmodeus said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
and the Alert Settings are set to default:1000 Stat Table entries and 100 Filter Limit. Are these the Alert Settings you were asking about?
From the Reports/Alerts tab, click the "+" on the right side of the page to open the "Alert Settings", the default for "Filter Limit Entries" is 100.
If you set it too high, you need to wait for the whole page to finish loading before trying to Whitelist.
There is also a new option in the Reports tab to "Pause" the auto page refresh. Its at the left side of each table heading.
-
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
From the Reports/Alerts tab, click the "+" on the right side of the page to open the "Alert Settings", the default for "Filter Limit Entries" is 100.
Got it, that's what Filter Limit Entres is set to for me - 100. Is there a chance to speed up loading of this page before/after whitelisting?
-
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Ok I see the issue.
As reference:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9350-L9351Can you edit the file: /usr/local/pkg/pfblockerng/pfblockerng.inc, Lines 9350 and 9351
From:
// Remove CARP Advskew setting, if found if (isset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew'])) { unset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew']); }
To:
// Remove CARP Advskew setting, if found if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) { unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']); }
I missed adding ['config'][0] to each of those two lines... Sorry about that...
Thank you.
Is this fix going to be released in the next version?
-
@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Thank you.
Is this fix going to be released in the next version?Once you confirm that it works :)
-
@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) {
unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']);Made the changes you suggested but it is not working.
Skew still gets to 0 on the secondary machine. -
This post is deleted! -
Since the update to 3.0, cpu usage went down considerable, also the sg-3100 temps.
Really nice work.. -
Hi BBcan177,
Thanks for the 3.0 release!
I'm currently not using the Python module since I have unbound views set up to only include the DNSBL configuration for certain subnets. However, I also have a custom Python module loaded to filter out AAAA responses from a number of Netflix domains and subdomains (works around an issue with IPv6 tunnelbrokers being blocked by Netflix as proxies).
Presently, when pfBlockerNG-devel is set to Unbound mode it resets my Unbound Python module settings every hour when it refreshes. Is there any way to get it to coexist with my own Python module and remain in Unbound mode? (I understand that this will preclude me from using the pfBlockerNG Python module in the future since pfSense's unbound configuration only supports one Python module, but for the moment I'd rather live with that limitation and use pfBlockerNG in Unbound mode while maintaining my current custom Python configuration).
Thanks!
-
This is what you can do : edit : /usr/local/pkg/pfblockerng/pfblockerng.inc : lines 2025 and up :
// Remove python settings from DNS Resolver configuration if (isset($config['unbound']['python'])) { // unset($config['unbound']['python']); // $config['unbound']['python_order'] = ''; // $config['unbound']['python_script'] = ''; $log = 'KEEPING DNSBL Unbound admin added python script';
Add in front of lines 2027 - 2028 - 2029 the "//"
Change line 2031 so it shows a text in the update log what it is doing.So, 4 lines need minor editing. The last one is even not strictly needed.
Now your unbound python settings like :
persists when pfBlockerNG 'does it things'.
Your own no-aaaa script gets loaded by unbound as before :
This line shows in the log because I modified the original no-aaaa script to show this line at init/start of the script.
When "views" gets integrated into pfBlockerNG, you should be able to switch to the new Python mode, uses views and use the build in 'no-aaaa' facility.
See this as a temporary patch that gives you the possibility to use your own no-aaaa script while you use pfBlockerNG in unbound mode != python mode.
Right now, true, if pfBlockerNG is put in unbound mode, it removes / deactivates unbound own python settings. -
@axellarsson Or use the Python mode with no AAAA setting
-
@BBcan177 thanks for this release. I just installed it. Looks like it is working BUT in the reports tab suddenly Source and IF is "unknown" which was not the case before the upgrade. Any idea?
I am on pfsense 2.4.5 and pfblocker-ng 3.0.0_7
-
pfSense 2.4.5 uses Unbound v1.10.1 which has a regression that fails to pass some information to the python modules. It has been fixed, but there is no way to upgrade Unbound to v.1.12.0 in pfSense 2.4.5.
In pfSense 2.5, it has Unbound v1.13.0.
For the DNSBL Blocking part, you can enable the checkbox in the DNSBL Tab > DNSBL Event Logging , and that will stop the python integration from logging, and use the DNSBL Webserver to log the events. Unfortunately, that is only limited to HTTP events.
And for DNS Reply logging, there is no other workaround.
Not much I can do unfortunately.