PfBlockerNG v2.0 w/DNSBL
-
Hey BBcan,
2.0 is working great for me, but I did have to make one tweak after install:
The LightHTTPD config for DNSBL has debug enabled by default. This causes a lot of file write I/O for some reason, and on systems with poor disk performance (like the hypervisor I run PFSense on) it has a nasty side-effect of making the 1x1 image take around 5 seconds to be served.
I monitored the LightHTTPD process and saw lots of write activity to /var/log/pfblockerng/dnsbl_error.log every time a client loaded a blocked url.
I was able to fix the issue by disabling log-condition-handling in the pfb_dnsbl_lighty.conf file ( Modified pfblockerng.inc and restarted )
debug.log-condition-handling = "disable"Did you intend to leave this setting on for release of the package? Or should it be updated default to disable?
-
The LightHTTPD config for DNSBL has debug enabled by default. This causes a lot of file write I/O for some reason, and on systems with poor disk performance (like the hypervisor I run PFSense on) it has a nasty side-effect of making the 1x1 image take around 5 seconds to be served.
I monitored the LightHTTPD process and saw lots of write activity to /var/log/pfblockerng/dnsbl_error.log every time a client loaded a blocked url.
I was able to fix the issue by disabling log-condition-handling in the pfb_dnsbl_lighty.conf file ( Modified pfblockerng.inc and restarted )
debug.log-condition-handling = "disable"Did you intend to leave this setting on for release of the package? Or should it be updated default to disable?
Hi ODVPFS,
The debug.log-condition-handling is used to collect HTTPs alerts. Without that option enabled, HTTPs alerts will still be blocked but without the logging.
When a client makes an HTTPs request to a domain that is in the blocked lists, the Resolver (Unbound) forwards the client to the DNSBL VIP address where the DNSBL Lighttpd web server is listening. The browser tries to make an SSL handshake with Lighttpd, but quickly sees that the certificate doesn't match, and the browser should quickly drop the connection and terminate the browser request. Since it doesn't establish the SSL handshake, the 1x1 gif never gets sent, and there is nothing to log. It sounds counter intuitive being an error log, but this debug error log captures the HTTPs domain which is then logged to the Alerts tab. As stated above, you can disable it, however, you will not get any alerts for HTTPs rejected domains without it…
In regards to your delay in serving the 1x1 image, are you on a single LAN subnet or a multi LAN network? Ensure that you are able to ping and browse to the VIP. The HTTP alerts do not use this Lighttpd error log. For multi-lan segments, there is a checkbox in the DNSBL tab to create a floating rule to allow traffic from other LAN subnets.
Here is a tech link to the Lighttpd error conditional log...
http://blog.lighttpd.net/articles/2006/04/02/log-condition-handling-the-hidden-feature-for-debuging/ -
An update to pfBNG was recently merged. The pkg is now v2.0.3.
PR #1217
https://github.com/pfsense/pfsense-packages/pull/1217-
Changes have been made to the DNSBL Service. If you experience any issues with the DNSBL Service not starting after re-install please let me know.
-
"Alias Native" option added to TOP20 tab.
-
Improved the Syslog (system.log) notices when Cron/Force commands are run.
-
Added a set_time_limit(0) to hopefully fix the issue with the 900 secs timeout that some users were experiencing.
-
-
on the DNSBL FEEDS , below on the page have Custom Block List,
I've added some domains there but I can still access them even they are on the block list .
how is this possible ?When you add to an existing custom list, you need to select the "Update custom list" setting at the bottom of the page. Otherwise the list is only updated as per the Alias frequency setting.
i already did those steps however it didn't works,
even rebooted the firewall.Add both example.com and www.example.com to the list…
(Change example.com)thank you for your answer.
yes both domains are added , using sex.com as test
www.sex.com
sex.comafter I chosed on update custom list 'Update Custom List' and clicked saved, after I even forced to reload the pfblocked on the update page , but when I chose sex.com it opens
-
I have hard coded some domains so that they can't be suppressed ;) <kidding>Run the following command to see if the domain is in the DNSBL list:
grep "example.com" /var/unbound/pfb_dnsbl.confpinging those domains should also resolve to the DNSBL VIP address.
If the domain is listed there, then check to see if your device DNS is pointed to the Resolver, and is not resolving from to a different DNS server…</kidding>
-
@BBcan177
First I wanted to thank you for the work you have put into this and as usual no real problems. I just wanted to ask about the gif image being pushed to the browser. When I browse to the VIP address I get a 1x1 pixel image. Is this image a placecard holder that would be populated with info or should it be in a different scale. I run my browser locked down as much as possible so not having seen the image is not on my priority list. I can ping and browse to it but was wondering about the single pixel size. If this is normal unpopulated size no worries but if not where do you hide it and what size is the default. -
Hi webtyro,
The 1x1 pixel is used so that it fills in where an ADvert is placed in a browser. This could be a warning message "Blocked by Admin" or something like that… but its hard to seperate the Feeds of ADVert domains vs Malicious domains, so its just easier to use the 1x1 pixel.
-
Hi webtyro,
The 1x1 pixel is used so that it fills in where an ADvert is placed in a browser. This could be a warning message "Blocked by Admin" or something like that… but its hard to seperate the Feeds of ADVert domains vs Malicious domains, so its just easier to use the 1x1 pixel.
Sometimes, when clicking a blocked AD URL, The browser was brought to VIP page with a white 1x1 pixel in the middle.
-
Hi webtyro,
The 1x1 pixel is used so that it fills in where an ADvert is placed in a browser. This could be a warning message "Blocked by Admin" or something like that… but its hard to seperate the Feeds of ADVert domains vs Malicious domains, so its just easier to use the 1x1 pixel.
Sometimes, when clicking a blocked AD URL, The browser was brought to VIP page with a white 1x1 pixel in the middle.
Note to self: "Don't click on ADs" :) lol….
-
Hi ODVPFS,
The debug.log-condition-handling is used to collect HTTPs alerts. Without that option enabled, HTTPs alerts will still be blocked but without the logging.
When a client makes an HTTPs request to a domain that is in the blocked lists, the Resolver (Unbound) forwards the client to the DNSBL VIP address where the DNSBL Lighttpd web server is listening. The browser tries to make an SSL handshake with Lighttpd, but quickly sees that the certificate doesn't match, and the browser should quickly drop the connection and terminate the browser request. Since it doesn't establish the SSL handshake, the 1x1 gif never gets sent, and there is nothing to log. It sounds counter intuitive being an error log, but this debug error log captures the HTTPs domain which is then logged to the Alerts tab. As stated above, you can disable it, however, you will not get any alerts for HTTPs rejected domains without it…
In regards to your delay in serving the 1x1 image, are you on a single LAN subnet or a multi LAN network? Ensure that you are able to ping and browse to the VIP. The HTTP alerts do not use this Lighttpd error log. For multi-lan segments, there is a checkbox in the DNSBL tab to create a floating rule to allow traffic from other LAN subnets.
Here is a tech link to the Lighttpd error conditional log...
http://blog.lighttpd.net/articles/2006/04/02/log-condition-handling-the-hidden-feature-for-debuging/Thanks, that makes sense. The issue I have is clearly disk I/O related (and self-induced) so I'm willing to sacrifice the logging in this case. I'm running PFSense on a Smartos/KVM Hypervisor (which uses ZFS) and I'm doing it without log drives - which has the consequence of horribly slow write sync performance.
Everything network related is fine - once I disabled the debug the response times went down to <100ms easily. I diagnosed the issue by running truss on the Lighthttpd process while hitting it with test requests from the browser - could see a ton of strings written to error log (even though the content of the error log never changed, which is strange, its almost like its overwriting the file repeatedly). Anyway I checked fstat for the pid to get the inode, and used find to identify it was the log file, which led me to the debug setting.
The only thing I would suggest as a very very minor enhancement (which I'm happy to try to do and submit a PR on github if you are open to it) is to throw a checkbox on the DNSBL config section to disable logging (explaining the caveat of losing HTTPS logging) so I don't have to redo my hack whenever I download updates.
Thanks again for the support, this is a great package!
-
An update to pfBNG was recently merged. The pkg is now v2.0.3.
PR #1217
https://github.com/pfsense/pfsense-packages/pull/1217-
Changes have been made to the DNSBL Service. If you experience any issues with the DNSBL Service not starting after re-install please let me know.
-
"Alias Native" option added to TOP20 tab.
-
Improved the Syslog (system.log) notices when Cron/Force commands are run.
-
Added a set_time_limit(0) to hopefully fix the issue with the 900 secs timeout that some users were experiencing.
Got this today
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 Connection timed out after 15007 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... .. unknown http status code [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ] The Following list has been REMOVED [ SEC1_OpenBL ]However I see nothing blocked in firewall log
nothing in Suricata alert/log nor Pfblocker alert/log :o -
-
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ]There is an issue as its displaying "Array"…
You can either edit the line in pfblockerng.inc Line 1329 or wait for the next PR…
Original: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result} ]\n";
Modified: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result[1]} {$result[0]} ]\n";[updated]
-
The only thing I would suggest as a very very minor enhancement (which I'm happy to try to do and submit a PR on github if you are open to it) is to throw a checkbox on the DNSBL config section to disable logging (explaining the caveat of losing HTTPS logging) so I don't have to redo my hack whenever I download updates.
Thanks again for the support, this is a great package!
Ok I will try to get that option into the package. Thanks!
-
I did the modification, waiting for the next block.
grep "^69\.60\.103" /var/db/aliastables/*.txt /var/db/aliastables/pfB_SeedBox.txt:69.60.103.0/24This is an alias_deny table.
I had a floating rule defined for it using this alias, but I just noticed that the rules no longer exist ???But during the day I made a few modifications, like disable LAN interface where the floating applied, updapted pfBlocker, and a few reboot, disable pfBlocker (do I have to disable dnsbld as well ) for doing the editing of pfblockerng.inc , so I am not sure the rules was there as 21:29:54.
I just did a Force Update
[ SEC1_OpenBL ] Downloading update . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15002 milliseconds Retry in 5 seconds... .. unknown http status code [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 23:36:22 ] [ 69.60.103.158 ] Firewall IP block found in: [ SB_PrimThreats 69.60.103.0/24 ]So I guess the fix is working now :)
But it is probably just timing out as the alias is not in used and there is no block in the logs.
From http://www.isitdownrightnow.com
Openbl.org is DOWN for everyone. It is not just you. The server is not responding… -
This might sound like a bizarre bug, but just reporting what happened.
Had everything working great for weeks now. Survived updates, everything good.
Today noticed that Chameleon had 0 entries on the pfsense home. Figured no matter, but thought to debug I'd change under DNSBL Feeds, "Chameleon" to "chameleon".
After hitting saved pfsense complained of HTTP_REFERRER issues and my router was inaccessible even after reboot.
Was odd, hence my reporting.
Reinstalled and recovered from backup and everything is fine now.
I should note that the Chamelon list is not relevant to me.
-
Hi yea,
When you hit "save" to update that DNSBL list, do you know if the pfBNG cron task might have been running in the background at the same time? Can you see the pfblockerng.log?
-
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ]There is an issue as its displaying "Array"…
You can either edit the line in pfblockerng.inc Line 1329 or wait for the next PR…
Original: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result} ]\n";
Modified: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result[1]} | {$result[0]} ]\n";The following issue is fixed in the package. So users can either edit the file to manually make this change, or re-install. I didn't bump the package version for this single issue.
https://github.com/pfsense/pfsense-packages/commit/d48c16f2bf89810c45d75fe50912fb6bd7edf019
-
How can I make a whitelist for pfblocker or more for dnsbl?
I created a alias with 2 test domains and set it as the first floating rule but that didn't work the 2 test domains can't be reached.How do i do this the best way?
-
@Gé:
How can I make a whitelist for pfblocker or more for dnsbl?
I created a alias with 2 test domains and set it as the first floating rule but that didn't work the 2 test domains can't be reached.How do i do this the best way?
The IP rules are independent of DNSBL…
Currently, you can't create access-control type refinements in DNSBL. If you want to have some devices to bypass DNSBL, then those devices need to use a different DNS server. I have plans for improvement in this area, but that involves enabling python support in Unbound and designing a python script for DNSBL
-
Sounds complicated.
Good luck with this future improvement.Using a alternative DNS is a good enough solution to reach a site I can't reach thru dnsbl wen needed.
Thank you!
