PfBlockerNG v2.0 w/DNSBL
-
Hi ODVPFS,
The debug.log-condition-handling is used to collect HTTPs alerts. Without that option enabled, HTTPs alerts will still be blocked but without the logging.
When a client makes an HTTPs request to a domain that is in the blocked lists, the Resolver (Unbound) forwards the client to the DNSBL VIP address where the DNSBL Lighttpd web server is listening. The browser tries to make an SSL handshake with Lighttpd, but quickly sees that the certificate doesn't match, and the browser should quickly drop the connection and terminate the browser request. Since it doesn't establish the SSL handshake, the 1x1 gif never gets sent, and there is nothing to log. It sounds counter intuitive being an error log, but this debug error log captures the HTTPs domain which is then logged to the Alerts tab. As stated above, you can disable it, however, you will not get any alerts for HTTPs rejected domains without it…
In regards to your delay in serving the 1x1 image, are you on a single LAN subnet or a multi LAN network? Ensure that you are able to ping and browse to the VIP. The HTTP alerts do not use this Lighttpd error log. For multi-lan segments, there is a checkbox in the DNSBL tab to create a floating rule to allow traffic from other LAN subnets.
Here is a tech link to the Lighttpd error conditional log...
http://blog.lighttpd.net/articles/2006/04/02/log-condition-handling-the-hidden-feature-for-debuging/Thanks, that makes sense. The issue I have is clearly disk I/O related (and self-induced) so I'm willing to sacrifice the logging in this case. I'm running PFSense on a Smartos/KVM Hypervisor (which uses ZFS) and I'm doing it without log drives - which has the consequence of horribly slow write sync performance.
Everything network related is fine - once I disabled the debug the response times went down to <100ms easily. I diagnosed the issue by running truss on the Lighthttpd process while hitting it with test requests from the browser - could see a ton of strings written to error log (even though the content of the error log never changed, which is strange, its almost like its overwriting the file repeatedly). Anyway I checked fstat for the pid to get the inode, and used find to identify it was the log file, which led me to the debug setting.
The only thing I would suggest as a very very minor enhancement (which I'm happy to try to do and submit a PR on github if you are open to it) is to throw a checkbox on the DNSBL config section to disable logging (explaining the caveat of losing HTTPS logging) so I don't have to redo my hack whenever I download updates.
Thanks again for the support, this is a great package!
-
An update to pfBNG was recently merged. The pkg is now v2.0.3.
PR #1217
https://github.com/pfsense/pfsense-packages/pull/1217-
Changes have been made to the DNSBL Service. If you experience any issues with the DNSBL Service not starting after re-install please let me know.
-
"Alias Native" option added to TOP20 tab.
-
Improved the Syslog (system.log) notices when Cron/Force commands are run.
-
Added a set_time_limit(0) to hopefully fix the issue with the 900 secs timeout that some users were experiencing.
Got this today
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 Connection timed out after 15007 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... .. unknown http status code [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ] The Following list has been REMOVED [ SEC1_OpenBL ]However I see nothing blocked in firewall log
nothing in Suricata alert/log nor Pfblocker alert/log :o -
-
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ]There is an issue as its displaying "Array"…
You can either edit the line in pfblockerng.inc Line 1329 or wait for the next PR…
Original: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result} ]\n";
Modified: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result[1]} {$result[0]} ]\n";[updated]
-
The only thing I would suggest as a very very minor enhancement (which I'm happy to try to do and submit a PR on github if you are open to it) is to throw a checkbox on the DNSBL config section to disable logging (explaining the caveat of losing HTTPS logging) so I don't have to redo my hack whenever I download updates.
Thanks again for the support, this is a great package!
Ok I will try to get that option into the package. Thanks!
-
I did the modification, waiting for the next block.
grep "^69\.60\.103" /var/db/aliastables/*.txt /var/db/aliastables/pfB_SeedBox.txt:69.60.103.0/24This is an alias_deny table.
I had a floating rule defined for it using this alias, but I just noticed that the rules no longer exist ???But during the day I made a few modifications, like disable LAN interface where the floating applied, updapted pfBlocker, and a few reboot, disable pfBlocker (do I have to disable dnsbld as well ) for doing the editing of pfblockerng.inc , so I am not sure the rules was there as 21:29:54.
I just did a Force Update
[ SEC1_OpenBL ] Downloading update . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15001 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15002 milliseconds Retry in 5 seconds... .. unknown http status code [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 23:36:22 ] [ 69.60.103.158 ] Firewall IP block found in: [ SB_PrimThreats 69.60.103.0/24 ]So I guess the fix is working now :)
But it is probably just timing out as the alias is not in used and there is no block in the logs.
From http://www.isitdownrightnow.com
Openbl.org is DOWN for everyone. It is not just you. The server is not responding… -
This might sound like a bizarre bug, but just reporting what happened.
Had everything working great for weeks now. Survived updates, everything good.
Today noticed that Chameleon had 0 entries on the pfsense home. Figured no matter, but thought to debug I'd change under DNSBL Feeds, "Chameleon" to "chameleon".
After hitting saved pfsense complained of HTTP_REFERRER issues and my router was inaccessible even after reboot.
Was odd, hence my reporting.
Reinstalled and recovered from backup and everything is fine now.
I should note that the Chamelon list is not relevant to me.
-
Hi yea,
When you hit "save" to update that DNSBL list, do you know if the pfBNG cron task might have been running in the background at the same time? Can you see the pfblockerng.log?
-
[ SEC1_OpenBL ] Downloading update [ 12/14/15 21:28:54 ] . cURL Error: 28 [ pfB_SEC1 - SEC1_OpenBL ] Download FAIL [ 12/14/15 21:29:54 ] [ 69.60.103.158 ] Firewall IP block found in: [ Array ]There is an issue as its displaying "Array"…
You can either edit the line in pfblockerng.inc Line 1329 or wait for the next PR…
Original: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result} ]\n";
Modified: $log = " [ {$ip} ] Firewall IP block found in: [ [b]{$result[1]} | {$result[0]} ]\n";The following issue is fixed in the package. So users can either edit the file to manually make this change, or re-install. I didn't bump the package version for this single issue.
https://github.com/pfsense/pfsense-packages/commit/d48c16f2bf89810c45d75fe50912fb6bd7edf019
-
How can I make a whitelist for pfblocker or more for dnsbl?
I created a alias with 2 test domains and set it as the first floating rule but that didn't work the 2 test domains can't be reached.How do i do this the best way?
-
@Gé:
How can I make a whitelist for pfblocker or more for dnsbl?
I created a alias with 2 test domains and set it as the first floating rule but that didn't work the 2 test domains can't be reached.How do i do this the best way?
The IP rules are independent of DNSBL…
Currently, you can't create access-control type refinements in DNSBL. If you want to have some devices to bypass DNSBL, then those devices need to use a different DNS server. I have plans for improvement in this area, but that involves enabling python support in Unbound and designing a python script for DNSBL
-
Sounds complicated.
Good luck with this future improvement.Using a alternative DNS is a good enough solution to reach a site I can't reach thru dnsbl wen needed.
Thank you!
-
@Gé:
…Using a alternative...
pfBlockerNG: DNSBL: Settings (Custom Domain Suppression) ? No ?
-
Hi yea,
When you hit "save" to update that DNSBL list, do you know if the pfBNG cron task might have been running in the background at the same time? Can you see the pfblockerng.log?
Hi BBcan177 nothing warned me of such. Couldn't prove either way I'm afraid.
-
I am seeing an issue where the pages with ads can take up to 20 seconds to timeout prior to loading the page…
Example: landsend.com will sit blank for 20 seconds waiting on assets.adobedtm.com, then loads immediately.
Our DNSBL webserver is set at 192.168.192.168 and i can ping it from my computer, it is also set to allow all traffic to/from this address in the floating rules.
My internal network is 10.0.0.0/12 and Guest network is in 172.16.0.0/16 range.
I have followed all the instructions in this forum, and all is working well, except this delay between DNSBL and the computer.
Are there any more tests i can do to check the connection to the DNSBL webserver? If I browse to the IP or request a jpg file it times out.
Thanks for any help or suggestions.
-Chad
-
I am seeing an issue where the pages with ads can take up to 20 seconds to timeout prior to loading the page…
If I browse to the IP or request a jpg file it times out.Hi Chad,
You have to check your Firewall/NAT rules to see if a rule is blocking access to the DNSBL VIP from your LAN. Without fixing that you will get timeouts as you are experiencing. You have to ensure that you can ping and browse to the VIP.
-
I have hard coded some domains so that they can't be suppressed ;) <kidding>Run the following command to see if the domain is in the DNSBL list:
grep "example.com" /var/unbound/pfb_dnsbl.confpinging those domains should also resolve to the DNSBL VIP address.
If the domain is listed there, then check to see if your device DNS is pointed to the Resolver, and is not resolving from to a different DNS server…</kidding>
if I ping the domains its resolve the DNSBL IP 10.10.10.1
now somehow stuff start working and it blocks the sites.
maybe it needed time ? I didn't change anything from side . -
I have posted a Pull Request #1223:
-
Fix 'Match Outbound' rule variable name
-
Remove duplicate Alexa variable as its already in pfb_global()
-
Determine if a DNSBL background reload is running before updating DNSBL
-
Change "${cmd}" variable name to "{$cmd}"
-
Add $pfb['ccdir'] explicitly due to file caching issue.
-
Fix dShield URL
-
Add tcpiputils to DNSBL Domain threat lookup
-
Add log size settings (5000 & 10000 lines) for limited storage installations. (Not recommended)
-
Add Paypal 'support the Developer' link
-
Remove extra whitespace
Also, the following issue was included in Pull Request #1219
-
-
-
I have a question of running VPN next to/ together with DNSBL.
On my laptop and Iphone/ Ipad i have client software running PIA VPN. With the VPN on nothing seem to be blocked, when switched of DNSBL is working. This is confirmed in the alerts tab and log.
Is it not possible to run VPN and DNSBL ? My understanding of the system is still limited. Hope you guys could point me in the right direction.
Greets,
HJ
-
Is it not possible to run VPN and DNSBL ? My understanding of the system is still limited.
I don't personally use a VPM like PIA, but the only way for DNSBL to function, is if the pfSense Resolver is used for the DNS requests. If the VPN uses a different DNS server, then DNSBL will not block any domains…
