PfBlockerNG v2.0 w/DNSBL
-
Any1 has the issue about failed to download Snort rules from Snort website since Nov. 24? It seemed this is an issue after upgrading to pfBlockerNG v2.0. but I'm not sure, don't know which dns feed block it, and I don't have any clue from the Alert tab either.
EDIT: Its the DNS: s3.amazonaws.com, was blocked by the DNSBL MPatrol feed. Adding it to the suppression list solves.
-
I've sorted the issue with the snortBL by setting the state to FLEX. It had a curl ssl error code 200 . This could be seen in view log in the update tab. So setting to FLEX also resolves a curl 200 status code.
-John-
-
Any1 has the issue about failed to download Snort rules from Snort website since Nov. 24? It seemed this is an issue after upgrading to pfBlockerNG v2.0. but I'm not sure, don't know which dns feed block it, and I don't have any clue from the Alert tab either.
I posted in the IDS forum… Try to avoid double posting...
I've sorted the issue with the snortBL by setting the state to FLEX.
The issue that pfcode is having, is with the Snort VRT rulesets for the IDS/IPS Snort… not the snort IP list. But yes the Snort IP list needs to use 'Flex'... Would be nice if Snort actually had a proper cert for their HTTPS site :)
-
Thank you BBCAN,
I've tested it and it works fine really clean internet.
the only think I notice is the DHCP server stops sharing IPS,
is this related to the DNSBL ?
please advise.Not related to DNSBL… Only thing that I suggest, is not to use the two "DHCP Registration" checkboxes in the Resolver, as that seems to have issues on Reload.
-
One extra that would be very useful is the ability to export (automatic) the lists to the webserver of PFSense, that way I could use the same list on other computers and only maintain it in pfsense. This is useful because when I'm not at home I like to use on my personal laptop the lists with PeerBlock!
Nice Job
Peerblock would probably need the files to be formatted in a certain way? There is a file in: /var/db/pfblockerng/mastercat which is a complete list of IPs (Deny).
-
Passing my comment from the other topic to this one
Somtimes the reporting of the Alerts misses the list matching! I had this problem today.
List 1 - Alias_Deny (De-Duplication enabled)
List 2 - Alias_NativeInterface 1
Block List 1The IP1 is in both lists, because Alias_Native does not passes by the De-Duplication process.
I had a hit on Interface 1 of IP1 and in the alert tab appeared the information that the hit is from List 2 (that does not even appears on the Interface)
Thank You all
This is a hard one to solve when "Alias Native" lists are used since multiple lists can have the same IP.. Alerts tab is not superhuman :)
-
Trouble with whitelist
Since the update to pfBlockerNG v2.0 w/DNSBL I'm having a problem whitelisting some IPs. I'm not sure exactly if the problem started before I tried using the DNS blocklist option, or purely from updating pfBlockerNG.
I turned off the dns forwarder and turned on the dns resolver and the DNSBL worked nicely, some alerts appeared in the alert tab. Besides updating PfBlockerNG and trying the DNSBL option I did not change anything to my IP blocklists or whitelist.
I noticed that my whitelist wasn't working anymore. No more green icons in the firewall log dashboard gadget and nothing in the permit section of the pfBlockerNG: Alerts tab. I disabled the DNSBL and went back to the dns forwarder. This didn't help. I tried rebooting, flusing the firewall state table and reinstalling pfBlockerNG (keeping the settings tab checked). This didn't help either.When I look at the iplist update log, my whitelist shows up in the Permit List with the correct number of IPs.
The blocked IPs that I want to whitelist are blocked by the country list.Does anyone have the same problem? Searching the forum I get the impression this is not a universal problem.
Suggestions would be greatly appreciated.
- John -
-
JohnPFsense,
For this whitelist, did you select "permit_outbound" in the alias? Does it create a firewall permit rule in the outbound firewall tab? Also ensure that the 'Rule order' is set to order the Permit rules above the block/reject rules.
-
JohnPFsense,
For this whitelist, did you select "permit_outbound" in the alias? Does it create a firewall permit rule in the outbound firewall tab? Also ensure that the 'Rule order' is set to order the Permit rules above the block/reject rules.
I did select permit_outbound, I also tried permit_both just to check. That did not help. I can see a permit rule in the Firewall: Rules tab. After some fiddling around I figured it out thanks to your suggestion. It was te "Rule order" The original setting apparently doesn't work as it used to. I left it at default after my pfBlockerNG update.
I also figured out that using the pfSense Pass/Match first order totally negates any block rule if "default pass LAN to any rule" is present. This could not be seen in the firewall dashboard widget. The last blocked packet was shown from before the settings change. Some vague memories about ipchains are coming to the surface, but I am not a networking expert.Thanks a lot for your quick response, knowing what to do and teaching me something on top of it.
- John -
-
I get an error when updating the BBC_DGA list as follows:
[ DNSBL FAIL ] [ Skipping : BBC_DGA ]
/var/unbound/check.conf:6: error: cannot open include file '/var/db/pfblockerng/dnsbl/BBC_DGA': No such file or directory
/var/unbound/check.conf:6: error: unknown keyword '.bk'
read /var/unbound/check.conf failed: 2 errors in configuration fileAnyone got a clue ?
-
I get an error when updating the BBC_DGA list as follows:
[ DNSBL FAIL ] [ Skipping : BBC_DGA ]
/var/unbound/check.conf:6: error: cannot open include file '/var/db/pfblockerng/dnsbl/BBC_DGA': No such file or directory
/var/unbound/check.conf:6: error: unknown keyword '.bk'
read /var/unbound/check.conf failed: 2 errors in configuration fileAnyone got a clue ?
BBC DGA is a large GZ download (16M Extracted - I don't readily have the gz file size). Maybe its timing out on you? Do you have a slow connection?
-
What about setting your virtual address to 0.0.0.0? As that whole /8 is non-routable, you wouldn't get the cert error. It would just silently fail to load ads.
While that might hide the cert error, there would be downsides. First, some some pages will hang as they wait for the ad server to respond. Second, blackholing the traffic completely by setting the VIP to something other than the DNSBL webserver will disable alerts. That would create problems, because the regular ad-blocking lists WILL break some sites and mobile apps. The alerts are, by far, the easiest way to track down problems.
There are better plausible workarounds, but we need to see some examples to understand under what circumstances people run into cert errors.
I manually set ad domains to 127.0.0.1, which results in a "Non-existent domain" response from Unbound. No waiting for timeouts. A website or app that breaks if ads can't be reached is a site or app I don't care about. Others may not be able to say that.
Every website that has annoying ads, I've been blackholing DNS for over a year without issues. Results may vary.
-
Thank you BBCAN,
I've tested it and it works fine really clean internet.
the only think I notice is the DHCP server stops sharing IPS,
is this related to the DNSBL ?
please advise.Not related to DNSBL… Only thing that I suggest, is not to use the two "DHCP Registration" checkboxes in the Resolver, as that seems to have issues on Reload.
i've noticed the LAN had the same IP as the DNSBL.
change it to 10.10.30.1 now let test.
i'll report back
one notice MAC book users keep getting error certificate when they visit youtube.
certificate can't be identified " Safari can't identify the website certificate "
anyway of avoiding this ? is kinda frustrating.
thank you
thank you -
When I select Match Both, with Floating Rules enabled, 1 rule is generated Source on the Wan interface. Same as Match Inbound. If I select Match Outbound, no rule is created ?
With Deny Both 2 rules are created: Source on WAN and Destination on LAN.
I was expecting that Match Both would also create 2 rules.The other odd thing is that the match rules are shown as Block with the Red X, maybe because there is no Match icon in the System Logs/Firewall GUI. This is confusing as it is not blocked at that stage.
pfBlockerNG 2.0.1
2.2.5-RELEASE (i386) -
I manually set ad domains to 127.0.0.1, which results in a "Non-existent domain" response from Unbound. No waiting for timeouts. A website or app that breaks if ads can't be reached is a site or app I don't care about. Others may not be able to say that.
Every website that has annoying ads, I've been blackholing DNS for over a year without issues. Results may vary.
Truth be told, I suspect blackholing undesirable traffic along those lines should work for most things. I haven't tried it myself to see what could go wrong. But, that kind of blackholing would kill the logging system pfBlockerNG uses for the Alerts page. For non-HTTPS traffic, pfBlockerNG logs blocked traffic when an endpoint pulls down the php page that its redirected to. This logging method has the advantage of logging blocked traffic even when DNS entries are cached by an endpoint, and also captures other useful diagnostic information, such as the referrer URL, which tells you what page you were looking at that had the blocked traffic.
Redirection doesn't work for HTTPS traffic, as the browser won't complete a TLS handshake with the DNSBL webserver due to the certificate issue. So, BBcan177 came up with a workaround that basically logs the failed HTTPS connection. But, while alerts are still logged, this method doesn't provide the same amount of information.
Getting the Alerts working reliably is very important. The common DNS blocklists include things that definitely will break stuff. Many include various AWS domains, which breaks all kinds of things. Separate from that, I found that the Amazon and HBO Go apps did not work properly until I unblocked a few domains. I'm sure those aren't the only examples.
-
What lists are you guys using with pfblockerNG DNSBL?
iblocklist.com hosts premium
http://hphosts.gt500.org/hosts.txt
http://winhelp2002.mvps.org/hosts.txt
http://malwaredomains.lehigh.edu/files/justdomains
http://www.malwaredomainlist.com/hostslist/hosts.txt
http://www.hostsfile.org/Downloads/hosts.txt -
I manually set ad domains to 127.0.0.1, which results in a "Non-existent domain" response from Unbound. No waiting for timeouts. A website or app that breaks if ads can't be reached is a site or app I don't care about. Others may not be able to say that.
Every website that has annoying ads, I've been blackholing DNS for over a year without issues. Results may vary.
Truth be told, I suspect blackholing undesirable traffic along those lines should work for most things. I haven't tried it myself to see what could go wrong. But, that kind of blackholing would kill the logging system pfBlockerNG uses for the Alerts page. For non-HTTPS traffic, pfBlockerNG logs blocked traffic when an endpoint pulls down the php page that its redirected to. This logging method has the advantage of logging blocked traffic even when DNS entries are cached by an endpoint, and also captures other useful diagnostic information, such as the referrer URL, which tells you what page you were looking at that had the blocked traffic.
Redirection doesn't work for HTTPS traffic, as the browser won't complete a TLS handshake with the DNSBL webserver due to the certificate issue. So, BBcan177 came up with a workaround that basically logs the failed HTTPS connection. But, while alerts are still logged, this method doesn't provide the same amount of information.
Getting the Alerts working reliably is very important. The common DNS blocklists include things that definitely will break stuff. Many include various AWS domains, which breaks all kinds of things. Separate from that, I found that the Amazon and HBO Go apps did not work properly until I unblocked a few domains. I'm sure those aren't the only examples.
My guess is my limited list doesn't cause issues, but when you feed a list of 1k+ records, one of those may break a page. In general, web pages should function on their own, except when referencing CDN resources.
The only time I care about logging is when debugging an issue. Being a home user, I have the convenience of debugging after the fact instead of digging through logs for a one-off that needs to be solved.
What I really don't want is yet another service listening to a port on my firewall. But if there is a high enough risk of breaking popular sites, then I guess I'd rather not blackhole. I'd be interested to know what that risk is. How many of the top 1000 sites completely break when blackholing DNS instead of redirecting?
There is also the issue that Chrome and others are looking at making self-signed certs virtually useless in the near future. I'd rather not have to install a self-signed CA to every device on my network, especially when visitors come over.
"Hey bro, I have this ad block on my network, you'll need to add my self-signed root-CA to your computer, otherwise your browser is going to throw a tantrum on every site you visit"
Just saying.
-
"Hey bro, I have this ad block on my network, you'll need to add my self-signed root-CA to your computer, otherwise your browser is going to throw a tantrum on every site you visit"
Hi Harvey, there is nothing in the Instructions to require adding any self-signed CAs to any browser? As reggie14 has stated, adding a Self-Signed CA to a browser will not achieve anything either.
There is a self-signed CA for DNSBL but that is only there so that the Browser will terminate the connection when it sees that it is not a valid cert for HTTPs. Without a DNSBL Cert, the Browser will take more time to terminate the session. DNSBL then uses the Lighttpd error.conditional log to find the Domains that failed and log those requests to the Alerts Tab.
I think its best not to base everything on a few exceptions that break browsing, or cause some nuisance alerts. Most of those seem to be on Safari anyways. Suppressing those domains is also easy to accomplish.
And if you have a Guest network, and you wanted to bypass DNSBL, then you would just assign a different DNS server (ISP or Google) for that LAN segment via DHCP and the issue is moot.
-
Getting back to the main issue, I'd be curious to see some examples of websites that generate a HTTPS/TLS warning/error message. I haven't had any recently, except for when I click on a link that redirects me through tracking domains (SlickDeals.net are a good example of this). I haven't gotten any error messages when there's simply an ad on a page that gets blocked.
-
Hi, I'm a bit confused as how to configure pfBlockerNG.
I run a server, just a Mumble and a game server. I only wish them to be port-forwarded for IPs of my country (Sweden).
For instance, Mumble runs on the default 64738. I have it port-forwarded in firewall NAT. With pfBlockerNG running, I select 'Sweden' under Countries->Europe. Under 'List Action' I select 'Permit Both'. I Force Update.
Rule order is default pfB_Pass/Match | pfB_Block/Reject | pfSense Pass/Match.
My UK-based friend has no issues connecting, which is not my intention. How exactly would I go about to configure this?
Thanks a bunch in advance.
