pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!
-
Since the update to 3.0, cpu usage went down considerable, also the sg-3100 temps.
Really nice work..
-
Hi BBcan177,
Thanks for the 3.0 release!
I'm currently not using the Python module since I have unbound views set up to only include the DNSBL configuration for certain subnets. However, I also have a custom Python module loaded to filter out AAAA responses from a number of Netflix domains and subdomains (works around an issue with IPv6 tunnelbrokers being blocked by Netflix as proxies).
Presently, when pfBlockerNG-devel is set to Unbound mode it resets my Unbound Python module settings every hour when it refreshes. Is there any way to get it to coexist with my own Python module and remain in Unbound mode? (I understand that this will preclude me from using the pfBlockerNG Python module in the future since pfSense's unbound configuration only supports one Python module, but for the moment I'd rather live with that limitation and use pfBlockerNG in Unbound mode while maintaining my current custom Python configuration).
Thanks!
-
This is what you can do : edit : /usr/local/pkg/pfblockerng/pfblockerng.inc : lines 2025 and up :
// Remove python settings from DNS Resolver configuration if (isset($config['unbound']['python'])) { // unset($config['unbound']['python']); // $config['unbound']['python_order'] = ''; // $config['unbound']['python_script'] = ''; $log = 'KEEPING DNSBL Unbound admin added python script';Add in front of lines 2027 - 2028 - 2029 the "//"
Change line 2031 so it shows a text in the update log what it is doing.So, 4 lines need minor editing. The last one is even not strictly needed.
Now your unbound python settings like :
persists when pfBlockerNG 'does it things'.
Your own no-aaaa script gets loaded by unbound as before :
This line shows in the log because I modified the original no-aaaa script to show this line at init/start of the script.
When "views" gets integrated into pfBlockerNG, you should be able to switch to the new Python mode, uses views and use the build in 'no-aaaa' facility.
See this as a temporary patch that gives you the possibility to use your own no-aaaa script while you use pfBlockerNG in unbound mode != python mode.
Right now, true, if pfBlockerNG is put in unbound mode, it removes / deactivates unbound own python settings. -
@axellarsson Or use the Python mode with no AAAA setting
-
@BBcan177 thanks for this release. I just installed it. Looks like it is working BUT in the reports tab suddenly Source and IF is "unknown" which was not the case before the upgrade. Any idea?
I am on pfsense 2.4.5 and pfblocker-ng 3.0.0_7
-
pfSense 2.4.5 uses Unbound v1.10.1 which has a regression that fails to pass some information to the python modules. It has been fixed, but there is no way to upgrade Unbound to v.1.12.0 in pfSense 2.4.5.
In pfSense 2.5, it has Unbound v1.13.0.
For the DNSBL Blocking part, you can enable the checkbox in the DNSBL Tab > DNSBL Event Logging , and that will stop the python integration from logging, and use the DNSBL Webserver to log the events. Unfortunately, that is only limited to HTTP events.
And for DNS Reply logging, there is no other workaround.
Not much I can do unfortunately.
"Experience is something you don't get until just after you need it."
Website: http://pfBlockerNG.com
Twitter: @BBcan177 #pfBlockerNG
Reddit: https://www.reddit.com/r/pfBlockerNG/new/ -
@bbcan177 Thanks for the prompt reply.
I am willing to experiment but from what I just read 2.5 is not that close to be production ready (or is your experience different?).
Sad that no solution exists for upgrading unbound but that is not you fault of course. I changed to DNSBL Event Logging. What are typical "non http events" and am I missing them completely then (which would make debugging quit interesting).
What do you mean with DNS Reply logging?
Sorry for the stupid questions.
-
@j-koopmann said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
I am willing to experiment but from what I just read 2.5 is not that close to be production ready (or is your experience different?).
pfSense 2.5 is nearing RC status.
Sad that no solution exists for upgrading unbound but that is not you fault of course. I changed to DNSBL Event Logging. What are typical "non http events" and am I missing them completely then (which would make debugging quit interesting).
What do you mean with DNS Reply logging?
Sorry for the stupid questions.DNS Reply logging will not show the Source IP/Hostname in pfSense < 2.5 as there is a regression in Unbound.
-
@bbcan177 I might have found a bug with IPv6 DNSBL. When I have it enabled, it creates a VIP on the LAN interface, but it seems to block the ability for "track interface" to work. If I disable IPv6 DNSBL, the LAN gets an IPv6 address as expected.
-
@bruor
Set the DNSBL Interface to use "Localhost" -
@bbcan177 Awesome thanks!
-
Anybody an Idea why the pfb widget stopped to count the total queries resolved by unbound since v3.0.0? I'm currently running v3.0.0_7.
-
@artes said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
v3.0.0_7.
Upgrade .... 3.0.0_x versions are "work in progress".
3.0.0_8 for me right now :
edit :
This
that there are no lists / feeds loaded, so it's normal nothing else is listed.
If have 5 list loaded - with 1968 unique IP/DNSBL.
No "help me" PM's please. Use the forum, the community will thank you.
Edit : and where are the logs ?? -
@gertjan said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
Upgrade .... 3.0.0_x versions are "work in progress".
3.0.0_8 for me right now :The last one was a Copyright update : https://github.com/pfsense/FreeBSD-ports/commits/devel/net/pfSense-pkg-pfBlockerNG-devel
-
If there were no feeds the blocked counter wouldn't be at ~250k ;-)
here is a screenshot of my full widget
-
Just noticed that with this setup I get DNS SERVFAIL responses if pfBlockerNG matches. Should this not point to the virtual IP so that an error page has chances of being displayed? I am probably missing something.
-
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
pfSense 2.4.5 uses Unbound v1.10.1 which has a regression that fails to pass some information to the python modules. It has been fixed, but there is no way to upgrade Unbound to v.1.12.0 in pfSense 2.4.5.
In pfSense 2.5, it has Unbound v1.13.0.
For the DNSBL Blocking part, you can enable the checkbox in the DNSBL Tab > DNSBL Event Logging , and that will stop the python integration from logging, and use the DNSBL Webserver to log the events. Unfortunately, that is only limited to HTTP events.
And for DNS Reply logging, there is no other workaround.
Not much I can do unfortunately.
Hey @BBcan177 we already talked about that back when we exchanged mails but: Are you planning on switching trees with the upcoming 2.5 release? Make 3.0.x finally the stable version and go on developing in the devel branch again, so the customers can have a "stable" version again?
I get asked about that over and over and over and with all the changes in the devel up to 3.x I think it would be time to switch it back to stable so people finally ditch the "oldstable" and get the new one which would make support easier, too :)
What are your plans on that? And can we perhaps get together again about the whole HA/CARP thing?
Best wishes!
Cheers
\jens -
Hi,
I'm on pfSense 2.4.5 and pfBlockerNG 3.0.0.8.
After enabling the Unbound python mode for DNSBL and doing the Force Reload-DNSBL Unbound Resolver was stopped and did not start again.
I found the following information in the pfBlocker logfile:
| ...
| Starting Unbound Resolver... Not completed. [ 01/22/21 15:41:10 ]
| error: SSL handshake failed
| ...Saving DNSBL statistics... completed [ 01/22/21 15:41:05 ] ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 01/22/21 15:41:07 ] Added DNSBL Unbound python integration settings Adding DNSBL Unbound python mounts: Creating: /var/unbound/usr/local/bin Mounting: /usr/local/bin Creating: /var/unbound/usr/local/lib Mounting: /usr/local/lib Removing DNSBL SafeSearch mode (Resolver adv. setting) DNS Resolver ( enabled ) unbound.conf modifications: Added DNSBL Unbound Python mode Removed DNSBL SafeSearch mode Added DNSBL Unbound Python mode script Saving new DNSBL web server configuration to port [ 8081 and 8443 ] Stop Service DNSBL VIP address(es) configured Restarting DNSBL Service Stopping Unbound Resolver Unbound stopped in 1 sec. Starting Unbound Resolver... Not completed. [ 01/22/21 15:41:10 ] error: SSL handshake failed Restarting DNSBL Service (DNSBL python) DNSBL update [ 143616 | PASSED ]... completed ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] Reload . completed .. ------------------------------ Original Master Final ------------------------------ 1337 1337 1337 [ Pass ]**** ----------------------------------------------------------------- [ Abuse_IPBL_v4 ] Reload . completed .. Empty file, Adding '127.1.7.7' to avoid download failure. ------------------------------ Original Master Final ------------------------------ 0 1 1 [ Pass ] ----------------------------------------------------------------- [ Abuse_SSLBL_v4 ] Reload . completed .. ------------------------------ Original Master Final ------------------------------ 123 109 109 [ Pass ] ----------------------------------------------------------------- [ BBC_C2_v4 ] Downloading update [ 01/22/21 15:41:11 ] . cURL Error: 28 Resolving timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Resolving timed out after 15000 milliseconds Retry in 5 seconds... . cURL Error: 28 Resolving timed out after 15003 milliseconds Retry in 5 seconds... .. unknown http status code | 0(Re-)starting Unbound Resolver under Services/DNS Resolver/General Settings is also not possible. I get the rerror
| can't open file pfb_unbound.py for readingJan 22 15:47:55 unbound 77509:0 fatal error: failed to setup modules Jan 22 15:47:55 unbound 77509:0 error: module init for module python failed Jan 22 15:47:55 unbound 77509:0 error: pythonmod: can't open file pfb_unbound.py for reading Jan 22 15:47:55 unbound 77509:0 notice: init module 0: python Jan 22 15:47:15 filterdns failed to resolve host pool.ntp.org will retry later again. Jan 22 15:47:15 filterdns failed to resolve host time.windows.com will retry later again. Jan 22 15:47:15 filterdns failed to resolve host time.nist.gov will retry later again. Jan 22 15:45:15 filterdns failed to resolve host pool.ntp.org will retry later again. Jan 22 15:45:04 filterdns merge_config: configuration reload Jan 22 15:44:15 filterdns failed to resolve host time.windows.com will retry later again. Jan 22 15:44:15 filterdns failed to resolve host time-nw.nist.gov will retry later again. Jan 22 15:44:15 filterdns failed to resolve host time-b.nist.gov will retry later again. Jan 22 15:44:15 filterdns failed to resolve host time.nist.gov will retry later again. Jan 22 15:44:15 filterdns failed to resolve host pool.ntp.org will retry later again. Jan 22 15:44:15 filterdns failed to resolve host time-a.nist.gov will retry later again. Jan 22 15:43:15 filterdns failed to resolve host time-nw.nist.gov will retry later again. Jan 22 15:43:15 filterdns failed to resolve host time.windows.com will retry later again. Jan 22 15:43:15 filterdns failed to resolve host time-b.nist.gov will retry later again. Jan 22 15:43:15 filterdns failed to resolve host time.nist.gov will retry later again. Jan 22 15:43:15 filterdns failed to resolve host time-a.nist.gov will retry later again. Jan 22 15:43:15 filterdns failed to resolve host pool.ntp.org will retry later again. Jan 22 15:42:15 filterdns failed to resolve host time-nw.nist.gov will retry later again. Jan 22 15:42:15 filterdns failed to resolve host time.windows.com will retry later again. Jan 22 15:42:15 filterdns failed to resolve host time-b.nist.gov will retry later again. Jan 22 15:42:14 filterdns failed to resolve host pool.ntp.org will retry later again. Jan 22 15:42:14 filterdns failed to resolve host time-a.nist.gov will retry later again. Jan 22 15:42:14 filterdns failed to resolve host time.nist.gov will retry later again. Jan 22 15:41:10 unbound 38108:0 notice: init module 0: python Jan 22 15:41:09 unbound 61187:0 info: 2.000000 4.000000 5 Jan 22 15:41:09 unbound 61187:0 info: 1.000000 2.000000 2 Jan 22 15:41:09 unbound 61187:0 info: 0.524288 1.000000 5 Jan 22 15:41:09 unbound 61187:0 info: 0.131072 0.262144 3 Jan 22 15:41:09 unbound 61187:0 info: 0.032768 0.065536 2 Jan 22 15:41:09 unbound 61187:0 info: 0.016384 0.032768 2 Jan 22 15:41:09 unbound 61187:0 info: 0.008192 0.016384 1 Jan 22 15:41:09 unbound 61187:0 info: lower(secs) upper(secs) recursions Jan 22 15:41:09 unbound 61187:0 info: [25%]=0.065536 median[50%]=0.714573 [75%]=2 Jan 22 15:41:09 unbound 61187:0 info: histogram of recursion processing times Jan 22 15:41:09 unbound 61187:0 info: average recursion processing time 0.996802 sec Jan 22 15:41:09 unbound 61187:0 info: server stats for thread 1: requestlist max 36 avg 6.6 exceeded 0 jostled 0 Jan 22 15:41:09 unbound 61187:0 info: server stats for thread 1: 26 queries, 6 answers from cache, 20 recursions, 0 prefetch, 0 rejected by ip ratelimiting Jan 22 15:41:09 unbound 61187:0 info: 1.000000 2.000000 1 Jan 22 15:41:09 unbound 61187:0 info: 0.524288 1.000000 2 Jan 22 15:41:09 unbound 61187:0 info: 0.262144 0.524288 2 Jan 22 15:41:09 unbound 61187:0 info: 0.016384 0.032768 3 Jan 22 15:41:09 unbound 61187:0 info: 0.008192 0.016384 2 Jan 22 15:41:09 unbound 61187:0 info: lower(secs) upper(secs) recursions Jan 22 15:41:09 unbound 61187:0 info: [25%]=0.0191147 median[50%]=0.032768 [75%]=0.643216 Jan 22 15:41:09 unbound 61187:0 info: histogram of recursion processing times Jan 22 15:41:09 unbound 61187:0 info: average recursion processing time 0.391598 sec Jan 22 15:41:09 unbound 61187:0 info: server stats for thread 0: requestlist max 8 avg 3.3 exceeded 0 jostled 0 Jan 22 15:41:09 unbound 61187:0 info: server stats for thread 0: 12 queries, 2 answers from cache, 10 recursions, 0 prefetch, 0 rejected by ip ratelimiting Jan 22 15:41:09 unbound 61187:0 info: service stopped (unbound 1.10.1). Jan 22 15:40:15 unbound 61187:0 info: generate keytag query _ta-4f66. NULL IN Jan 22 15:40:15 unbound 61187:1 info: generate keytag query _ta-4f66. NULL IN Jan 22 15:40:09 unbound 61187:0 info: start of service (unbound 1.10.1). Jan 22 15:40:09 unbound 61187:0 notice: init module 1: iterator Jan 22 15:40:09 unbound 61187:0 notice: init module 0: validator Jan 22 15:40:02 unbound 44212:0 info: 4.000000 8.000000 1 Jan 22 15:40:02 unbound 44212:0 info: 2.000000 4.000000 3 Jan 22 15:40:02 unbound 44212:0 info: 1.000000 2.000000 17 Jan 22 15:40:02 unbound 44212:0 info: 0.524288 1.000000 43 Jan 22 15:40:02 unbound 44212:0 info: 0.262144 0.524288 144 Jan 22 15:40:02 unbound 44212:0 info: 0.131072 0.262144 146 Jan 22 15:40:02 unbound 44212:0 info: 0.065536 0.131072 116 Jan 22 15:40:02 unbound 44212:0 info: 0.032768 0.065536 71 Jan 22 15:40:02 unbound 44212:0 info: 0.016384 0.032768 287 Jan 22 15:40:02 unbound 44212:0 info: 0.008192 0.016384 132 Jan 22 15:40:02 unbound 44212:0 info: 0.004096 0.008192 8 Jan 22 15:40:02 unbound 44212:0 info: 0.000000 0.000001 39 Jan 22 15:40:02 unbound 44212:0 info: lower(secs) upper(secs) recursions Jan 22 15:40:02 unbound 44212:0 info: [25%]=0.0205371 median[50%]=0.050075 [75%]=0.222867 Jan 22 15:40:02 unbound 44212:0 info: histogram of recursion processing times Jan 22 15:40:02 unbound 44212:0 info: average recursion processing time 0.167938 sec Jan 22 15:40:02 unbound 44212:0 info: server stats for thread 1: requestlist max 26 avg 1.89474 exceeded 0 jostled 0 Jan 22 15:40:02 unbound 44212:0 info: server stats for thread 1: 2826 queries, 1819 answers from cache, 1007 recursions, 0 prefetch, 0 rejected by ip ratelimiting Jan 22 15:40:02 unbound 44212:0 info: 2.000000 4.000000 3 Jan 22 15:40:02 unbound 44212:0 info: 1.000000 2.000000 13 Jan 22 15:40:02 unbound 44212:0 info: 0.524288 1.000000 29 Jan 22 15:40:02 unbound 44212:0 info: 0.262144 0.524288 49 Jan 22 15:40:02 unbound 44212:0 info: 0.131072 0.262144 68 Jan 22 15:40:02 unbound 44212:0 info: 0.065536 0.131072 59 Jan 22 15:40:02 unbound 44212:0 info: 0.032768 0.065536 48 Jan 22 15:40:02 unbound 44212:0 info: 0.016384 0.032768 167 Jan 22 15:40:02 unbound 44212:0 info: 0.008192 0.016384 52 Jan 22 15:40:02 unbound 44212:0 info: 0.004096 0.008192 1 Jan 22 15:40:02 unbound 44212:0 info: 0.000000 0.000001 24 Jan 22 15:40:02 unbound 44212:0 info: lower(secs) upper(secs) recursions Jan 22 15:40:02 unbound 44212:0 info: [25%]=0.021412 median[50%]=0.0413013 [75%]=0.196126 Jan 22 15:40:02 unbound 44212:0 info: histogram of recursion processing times Jan 22 15:40:02 unbound 44212:0 info: average recursion processing time 0.167664 sec Jan 22 15:40:02 unbound 44212:0 info: server stats for thread 0: requestlist max 29 avg 1.26511 exceeded 0 jostled 0 Jan 22 15:40:02 unbound 44212:0 info: server stats for thread 0: 1484 queries, 971 answers from cache, 513 recursions, 0 prefetch, 0 rejected by ip ratelimiting Jan 22 15:40:02 unbound 44212:0 info: service stopped (unbound 1.10.1).Any ideas about this problem?
Regards Jürgen
-
@cantor Reboot your box
-
@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:
@cantor Reboot your box
Doesn't work. Unbound is still down after reboot and can only be restarted after I uncheck the option "Eable Python Module".
