PfBlockerNG v2.0 w/DNSBL

simby

Hi!

i have error:

There were error(s) loading the rules: /tmp/rules.debug:40: cannot define table pfB_Malicious: Cannot allocate memory - The line in question reads [40]: table persist file "/var/db/aliastables/pfB_Malicious.txt" @ 2016-04-26 07:10:37

and i have 16GB memory on i7

What is the problem?

cmaxwell04

@BBcan177:

@cmaxwell04:

Help please,

I'm having a strage problem in pfsense 2.3. Whenever I enable pfBlockerNG DNSBL it stops DNS Resolver, and I have to start it manually. when I rebot pfsense 2.3 I have to manualy start DNS Resolver. if I desable DNSBL, DNS Resolver will always run automaticaly.

thanks

Hi cmaxwell04,

Can you review the pfblockerng.log and see if there are any error messages in regards to DNSBL. Try to Re-Enable DNSBL, and try to send any relevant error messages… Do you see any errors in the resolver.log or system.log?

After a reboot of the pfsense, DNS Resolver was not running, so I disabled DNSBL and got this error messages.

Notices
Filter Reload

There were error(s) loading the rules: /tmp/rules.debug:214: macro 'pfB_DNSBLIP' not defined - The line in question reads [214]: block return in log quick on $LAN inet from any to $pfB_DNSBLIP tracker 1770009088 label "USER_RULE: pfB_DNSBLIP auto rule" @ 2016-04-26 12:58:40

Crash report begins. Anonymous machine information:

amd64
10.3-RELEASE
FreeBSD 10.3-RELEASE #4 05adf0a(RELENG_2_3_0): Mon Apr 11 19:09:19 CDT 2016 root@factory23-amd64-builder:/builder/factory-230/tmp/obj/builder/factory-230/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP Stack trace:
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 1. {main}() /etc/rc.update_urltables:0
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 2. process_alias_urltable() /etc/rc.update_urltables:51
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 3. parse_aliases_file() /etc/inc/pfsense-utils.inc:2185
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 4. filesize() /etc/inc/pfsense-utils.inc:1932

Nachtfalke

@chris4916:

@BBcan177:

I don't use Squid, but a quick google search, do you think that "host" should be changed to "url" instead in this line below?

I know this is slightly nitpicking but it has nothing to do with Squid itself but rather the proxy.pac that will help your browser deciding whether access is direct or through proxy ;)

This said, in proxy.pac (or whatever alias), you may look at different kind of information in order to make the "routing" decision.

If goal is to route through proxy (or not) based on IP address or DNS related information, this should implement directive like:
dnsDomainIs(host,…
isInNet(dnsResolve(host)
localHostOrDomainIs(host...

This involves DNS resolution

if goal is to look at URL pattern (not involving DNS resolution) then syntax to be used is:
shExpMatch(url,
url.substring

However, shExpMatch(host, does work too if you match the fqdn part of URL only 8) (although this is quite confusing, I agree)

Hey chris,

thank you for helping with the PAC file. I had alook on the internet and found this:
http://findproxyforurl.com/pac-functions/
but for me this is all quite confusing.

Could you be so kind a create a PAC file which could help with DNSBL and squid?

Lets say we have these IPs:

pfsense LAN-IP: 192.168.10.1 /24
pfsense WAN-IP: 192.168.178.254 /24
DNSBL VIP: 10.123.123.123

Could you show me a PAC file which will make sure that every connection to the internet will be checked against DNSBL?

Don't know if this is "easy" to do but at the moment I am totally confused ;-)

Regards

BBcan177

@simby:

Hi!

i have error:

There were error(s) loading the rules: /tmp/rules.debug:40: cannot define table pfB_Malicious: Cannot allocate memory - The line in question reads [40]: table persist file "/var/db/aliastables/pfB_Malicious.txt" @ 2016-04-26 07:10:37

and i have 16GB memory on i7

What is the problem?

Hi simby,

The package defaults the System / Advanced / Firewall & NAT / "Firewall Maximum Table Entries" to 2million entries… Did you alter that setting? It's not related to the hardware...

If you hover-over this rule in the Firewall Tab, does it show the IPs? Does the widget have a Red Icon in the far right column?

Can you give some more context to the events that led to this error, and what happened after?

Take a look at the pfblockerng.log for additional clues.

BBcan177

@cmaxwell04:

After a reboot of the pfsense, DNS Resolver was not running, so I disabled DNSBL and got this error messages.

Is this a full install or Nano? Do you use RamDisks?

There were error(s) loading the rules: /tmp/rules.debug:214: macro 'pfB_DNSBLIP' not defined - The line in question reads [214]: block return in log quick on $LAN inet from any to $pfB_DNSBLIP tracker 1770009088 label "USER_RULE: pfB_DNSBLIP auto rule" @ 2016-04-26 12:58:40

I have a fix for this issue, and will add that to the next release. In my tests, this is just a warning, and can be discarded. When DNSBL is disabled, check to see if there are any Firewall rules that reference "pfB_DNSBL"…

Crash report begins. Anonymous machine information:

amd64
10.3-RELEASE
FreeBSD 10.3-RELEASE #4 05adf0a(RELENG_2_3_0): Mon Apr 11 19:09:19 CDT 2016 root@factory23-amd64-builder:/builder/factory-230/tmp/obj/builder/factory-230/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP Stack trace:
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 1. {main}() /etc/rc.update_urltables:0
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 2. process_alias_urltable() /etc/rc.update_urltables:51
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 3. parse_aliases_file() /etc/inc/pfsense-utils.inc:2185
[26-Apr-2016 12:59:13 Africa/Abidjan] PHP 4. filesize() /etc/inc/pfsense-utils.inc:1932

This error message is not related to pfBlockerNG directly, so it could also be from another issue… Do you have any pfSense URL Aliases defined?

Nachtfalke

Hi,

I noticed that resolving DNS takes sometimes several retries. This is one of the URLs in the DNSBL which I tried to resolve 2 time until I got the correct VIP.
Any ideas where to look?

Further I created a new PAC file with the help of some websites on the internet. Tested the PAC file with some "testing" tools I found on the web, too.


function FindProxyForURL(url, host) {

    // Normalize the URL for pattern macthing
    url = url.toLowerCase();
    host = host.toLowerCase();

    // diese Ziel-IPs, Domains oder Hosts gehen DIRECT
    if (shExpMatch(url, "*.microsoft.com/*") ||
        shExpMatch(url, "*.windowsupdates.com/*") ||
        shExpMatch(url, "*ap.bundesbank.de/*") ||
        shExpMatch(url, "*on4u3.buhl.de/*") ||
        shExpMatch(host, "*.local") ||
        shExpMatch(url, "*data-cdn.mbamupdates.com/*"))
    {
    return "DIRECT";
    }

    // If IP of the requested host falls within any of the ranges specified, send direct.
    if (isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
        isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
        isInNet(dnsResolve(host), "192.168.0.0", "255.255.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
    {
    return "DIRECT";
    }

    // Fuer alle anderen Requests den Proxy zurueckgeben
    return "PROXY 192.168.10.1:3128";
}

Unfortunately DNSBL is still not working. :-/

DNS_not_OK.png_thumb

DNS_OK.png_thumb

f34rinc

Not sure if anyone else saw the pfSense newsletter today ;D the package pfBlockerNG is going to be the topic for this months pfSense Hangout Friday, April 29th, 2016 at 1PM CDT/2PM EDT

BBCan177 will be providing an overview of the package, how-to's, and taking questions from the audience.

This should be really good reference material for anyone looking for a basic setup, overview of how the package works.

Panja

I'm having issues retrieving updates from dshield.org.

In the log I found:


Failed to connect to feeds.dshield.org port 80: Connection refused Retry in 5 seconds...
. cURL Error: 7
Failed to connect to feeds.dshield.org port 80: Connection refused Retry in 5 seconds...
. cURL Error: 7
Failed to connect to feeds.dshield.org port 80: Connection refused Retry in 5 seconds...
.. unknown http status code

I have set up pfBlocker to get updates from:
http://feeds.dshield.org/top10-2.txt

Anyone know why it is failing?

BBcan177

@Panja:

I'm having issues retrieving updates from dshield.org.

I have set up pfBlocker to get updates from:
http://feeds.dshield.org/top10-2.txt

That link loads ok for me…
Change the "State" to "Off" for a day, then re-enable it... Maybe they are rate-limiting you...
Also best to use the "Download Failure Threshold" and set it to say (4).

Panja

Ok, thanks. Gonna have a look!

fmaxwell

Is it possible to import and export individual pfBlockerNG IPV4 aliases?
I ask because I am in the process of creating an alphabetized-by-continent IPV4 alias for with one country per line, allowing each country to be toggled on/off. The beginning of the alias is shown below:

I want to be able to create multiple instances of it, each with different names for use with different rules (e.g. "No_Email_Countries", "No_FTP_Countries", "No_HTTP_Countries," etc.). With more than 250 lines when it is all done, I am not willing to re-enter the entire thing in order to make a duplicate for another purpose.

Nor do I want to have to reenter the entire list if Zimbabwe decides to become Aaritania or a new country is added to the ISO 3166 country code list. (It's easy to just delete a country. So, for example, I have no problem with Guinea invading, and taking the territory of, Sierra Leone – as long as it doesn't change its name and, hence, where it needs to sort in my alias. Unfortunately, I doubt that I will be able to force all countries to restrict their political and military actions to ones which minimize the impact on my pfBlockerNG IPV4 aliases.)

So I would like to be able to export the alias as XML, edit the XML file (change the alias name, add a country, etc.), and then re-import it.

It looks like this in the config.xml file:

 <config><aliasname>Countries_I_Do_Not_Trust</aliasname>

        <row><format>auto</format>
                <state>Disabled</state>
                <url>/usr/local/share/GeoIP/cc/Africa_v4.txt</url>

<header>__AFRICA_ALL__</header></row> 
        <row><format>auto</format>
                <state>Enabled</state>
                <url>/usr/local/share/GeoIP/cc/DZ_v4.txt</url>

<header>Algeria</header></row> 
        <row><format>auto</format>
                <state>Disabled</state>
                <url>/usr/local/share/GeoIP/cc/AO_v4.txt</url>

<header>Angola</header></row> 
        <row><format>auto</format>
                <state>Disabled</state>
                <url>/usr/local/share/GeoIP/cc/BJ_v4.txt</url>

<header>Benin</header></row> 
        <row><format>auto</format>
                <state>Disabled</state>
                <url>/usr/local/share/GeoIP/cc/BW_v4.txt</url>

<header>Botswana</header></row> 
        <row><format>auto</format>
                <state>Disabled</state>
                <url>/usr/local/share/GeoIP/cc/BF_v4.txt</url>

<header>Burkina_Faso</header></row> 
...</config>

What I don't want to do is backup my entire configuration, edit the XML file manually, pasting in 1.5K line sections at a time for each instance of this alias, and then restore the entire configuration. Not only is that time-consuming, I will eventually screw something up and then very bad things will result.

So is exporting/importing practical or should I just abandon this effort?

chris4916

@Nachtfalke:

Could you be so kind a create a PAC file which could help with DNSBL and squid?

Lets say we have these IPs:

pfsense LAN-IP: 192.168.10.1 /24
pfsense WAN-IP: 192.168.178.254 /24
DNSBL VIP: 10.123.123.123

Could you show me a PAC file which will make sure that every connection to the internet will be checked against DNSBL?

To be honest I even don't understand (but I've to admit that I didn't spent lot of time on this specific issue) what the problem is with .pac file.

Let me explain :

when you configure explicit proxy, .pac file will tell your browser when to go direct or when use proxy.
because you are using explicit proxy, DNS resolution, when using proxy, is done at proxy level and DNS relies on pfSense to resolve names and block or not IPs.

Thus because I don't understand what problem is, I can't really help with proxy.pac design yet. But feel free to explain and I'll do my best with proxy.pac ;)

trumee

I have disabled DNSBL in the pgblockerNG menu. However, i restart pfsense DNSBL service shows up as Running under Status>Services. Is this a bug?

chris4916

@Nachtfalke:

Further I created a new PAC file with the help of some websites on the internet. Tested the PAC file with some "testing" tools I found on the web, too.

Again I'm confused because I don't understand the link between pac file and pfBlockerNG.

Pac file will tell your browser when to go direct or through proxy
when going thought proxy, DNS resolution is handled by your proxy server (and therefore underlying system)

This means that for anything that is not "local" (plus perhaps some exceptions if you allow direct access for Windowsupdate e.g. ::) ), name resolution is done by proxy server.
BTW, when going direct, name resolution id most likely done by your main DNS that is, I guess pfSense too

Because of this, I don't understand why .pac file would impact pfBlockerNG. I'm not saing there is no link but I just don"t understand. If you could explain, it would be nice ;)

Harvy66

@JasonJoel:

Anyone else experience that pfBlockerNG doesn't work on 2.3 if using floating rules? After the last update, I noticed all my rule counts were ZERO hits on the pfBlocker widget. So I tried going to a website on a blocked country, and it did not block.

Then tried:
1. Disabled the floating rules option (saved), forced an update, and then tried the website again and it was blocked.
2. Then re-enabled floating rules (saved), forced an update, tried the website again and it was NOT blocked.
3. Disabled the floating rules option (saved), forced an update, and then tried the website again and it was blocked.

So, seems on my system at least the floating rules simply do not work. I can't tell if this is a 2.3 issue or a pfBlocker issue though.

Jason Bottjen

I do a bunch of traffic shaping with floating rules, and they all show hits. Remember, last rule wins for floating, which is the opposite of non-floating rules.

q54e3w

@Harvy66:

@JasonJoel:

Anyone else experience that pfBlockerNG doesn't work on 2.3 if using floating rules? After the last update, I noticed all my rule counts were ZERO hits on the pfBlocker widget. So I tried going to a website on a blocked country, and it did not block.

Then tried:
1. Disabled the floating rules option (saved), forced an update, and then tried the website again and it was blocked.
2. Then re-enabled floating rules (saved), forced an update, tried the website again and it was NOT blocked.
3. Disabled the floating rules option (saved), forced an update, and then tried the website again and it was blocked.

So, seems on my system at least the floating rules simply do not work. I can't tell if this is a 2.3 issue or a pfBlocker issue though.

Jason Bottjen

I do a bunch of traffic shaping with floating rules, and they all show hits. Remember, last rule wins for floating, which is the opposite of non-floating rules.

unless you have quick match enabled, right?

JasonJoel

@irj972:

unless you have quick match enabled, right?

Right.

Nachtfalke

Hi,

I am not sure if I found a little bug in the GUI:

In pfblockerng general settings I configured CRON settings like this:
once a day :15 0 3

When going to services –> cron I can see this - which is correct I assume:
15 3 * * * root /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron >> /var/log/pfblockerng/pfblockerng.log 2>&1

BUT when I go to pfblockerng --> Updates I can see this:
Status NEXT Scheduled CRON Event will run at 01:15 with 01:44:46 time remaining.
Refresh to update current status and time remaining.
PS: At the moment it is 11:30 pm on my system and pfsense GUI + CLI.

Is there something wrong or did I miss some configuration in pfblockerng which is related to this time ?

PPS:
The redirect of DNSBL to the https VIP always shows an invalid certificate. As far as I can see this is because the CN in the cert is not the same as the blocked domain. I tried it with creating a wildcard certificate for some domains in the DNSBL like ".smaato.net" which showed a correct certificate for "c09.smaato.net" and so on. Tried with a wilcard certificate like ".net" but this did not work.

So my question ist if it is possible after downloading all the DNSBL domains to copy all these domains into a file and the create one new certificate which contains all these domains as SAN in the certificate, them replace the "old" cert on "/var/unbound/dnsbl_cert.pem" with the new one. Further is it possible to add the "ssl.ca-file" parameter (https://redmine.lighttpd.net/projects/1/wiki/Docs_SSL) to the lighhtpd conf file?

Found some code here not sure if it can be used:


openssl req -batch -newkey rsa:2048 -sha1 -keyout priv-key.pem -out certreq.pem -config openssl-gwdg.cnf


####################################################################
[ req ]
distinguished_name    = req_distinguished_name

string_mask = nombstr

# The extensions to add to a certificate request
req_extensions = v3_req

# GWDG default options for certificate request
[ req_distinguished_name ]
countryName            = Country Name (2 letter code)
countryName_default        = DE
countryName_min            = 2
countryName_max            = 2

stateOrProvinceName        = State or Province Name (full name)
stateOrProvinceName_default    = NIEDERSACHSEN

localityName            = Your City
localityName_default        = GOETTINGEN

0.organizationName        = Organization Name (eg, company)
0.organizationName_default    = Gesellschaft fuer wissenschaftliche Datenverarbeitung

# Dieser Teil ist Optional und kann bei Bedarf einkommentiert werden
# Wichtig: Es dürfen keine Sonderzeichen wie z.B. auch ein - oder _ dort vorkommen!
#
# organizationalUnitName        = Organizational Unit Name (eg, section)
# organizationalUnitName_default    =  AG A

commonName            = YOUR NAME
commonName_max            = 64
commonName_default        = example.gwdg.de

emailAddress            = E-MAIL
emailAddress_max        = 64
emailAddress_default        = support@gwdg.de

[ v3_req ]
subjectAltName          = DNS:example.gwdg.de, DNS:www.example.gwdg.de, DNS:www2.example.gwdg.de

####################################################################

Don't know if there is a SAN limit per certificate but perhaps with wildcard it should reduce the amount.

Regards

BBcan177

@Nachtfalke:

Hi,

I am not sure if I found a little bug in the GUI:

In pfblockerng general settings I configured CRON settings like this:
once a day :15 0 3

When going to services –> cron I can see this - which is correct I assume:
15 3 * * * root /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron >> /var/log/pfblockerng/pfblockerng.log 2>&1

BUT when I go to pfblockerng --> Updates I can see this:
Status NEXT Scheduled CRON Event will run at 01:15 with 01:44:46 time remaining.
Refresh to update current status and time remaining.
PS: At the moment it is 11:30 pm on my system and pfsense GUI + CLI.

Is there something wrong or did I miss some configuration in pfblockerng which is related to this time ?

Thanks Nachtfalke, I posted a fix for this in pfBlockerNG v2.0.14 (for pfSense v2.3)

Nachtfalke

Hi again,

I am still facing problems with certificate errors on Google Chrome 50 and IE10 on Windows 7. I am running pfsense 2.3 and the most recent pfblockerng package.

To make sure that there aren't any side-effects with squid anymore I configured an "allow mc-pc to any" rule on the LAN interface, I disabled squid package and disabled squid configuration on my browser.
Further I uninstalled pfblockerng package with (Keep Settings enabled) and then removed all remaining files in "usr/local/pkg/pfblockerng/" and all pfblockerng related files (config + certificates) in "/var/unbound/".

Then I did a reinstall of the pfblockerng package and I did a "Force Update All".

So I attached some screenshots which show the following:
1.) Which show tha I do not get any "IF" or "SOURCE" displayed for HTTPS requests
2.) I get a certificate warning with Google Chrome 50 and IE10 when accessing a website with https directly which is in the DNSBL
3.) When I accept the "insecure/untrusted" certificate then it will display an "IF" and "SOURCE" in the "Alerts" tab.
4.) Sometimes I am getting a HSTS error message

I know that pfblockerng / DNSBL will still block access to the websites on DNS level but it will not give my any visibility in the Alerts tab as long as the certificate problem exists.

So do you have the same problems?
Any ideas how to fix this?

Kind regards :-)

browser_warning_1.png_thumb

browser_warning_2.png_thumb

browser_warning_hsts_3.png_thumb

Alerts.png_thumb

Alerts_after_accept_cert.png_thumb