PfBlockerNG v2.0 w/DNSBL

varazir

@varazir:

@BBcan177:

@varazir, are you on the latest version of pfBlockerNG? If not, please update and see if that fixes your issue.

I don't have any newer in the System: Package Manager, 2.0.4

Line 902 has code for the Alexa database conversion… If this only happened once, then discard it, but if its happening more often please provide some additional details on your hardware.

Do you see these two files:
ls /var/db/pfblockerng/alexa*

/var/db/pfblockerng/alexa-top1m.zip 
/var/db/pfblockerng/alexa_1m
Can you open the alexa_1m file?

When you run this command, it will show how many Alexa TLDs are being used… The count should match the Alexa count that you defined in the DNSBL tab (Number of Alexa Top Domains to Whitelisting):
wc -l /var/db/pfblockerng/alexa_1m
You can also review the error.log file, to see if the Alexa Database is failing…

Can't find the files and I get it several times a day. almost every time I check pfsense I see a crash log.

I'm running it as virtual so

Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
Current: 388 MHz, Max: 3109 MHz
4 CPUs: 2 package(s) x 2 core(s)

4GB ram

BBcan177

I have posted PR #1243, pfBlockerNG v2.0.5 (for pfSense v2.2.x)
and
I have also posted PR #87, pfBlockerNG v2.0.8 (for pfSense v2.3)

Changelog can be seen in the attached links. If you have any issues, post back in the forum.

I highly recommend installing pfSense 2.3. Its nearing Release Candidate (RC) and is really looking sharp. Its getting harder to maintain pfBlockerNG in two different platforms, so I may concentrate my future efforts in pfSense 2.3. So it will most likely see all of the new upcoming features. I will however maintain pfBlockerNG in 2.2.x that are bug fixes.

Thanks!

beatstick

Can the package be updated from the pfsense gui by clicking reinstall pfblockerng (pfsense 2.2.6)? Will my settings be kept? I can't find a download file on the github page on first look, which is normal, as github seems to make a point of confusing anyone who just wants to find a simple installer ;)

wiz561

@beatstick:

Can the package be updated from the pfsense gui by clicking reinstall pfblockerng (pfsense 2.2.6)? Will my settings be kept? I can't find a download file on the github page on first look, which is normal, as github seems to make a point of confusing anyone who just wants to find a simple installer ;)

Normally, I think packages get upgraded/updated through the package manager. You have to manually update the packages by clicking on the "reinstall the package" button. However, I did this and I'm still on 2.0.4.

Also, my settings didn't get removed, but it's always a good idea to backup the config first.

killmasta93

FOR PEOPLE WITH GIF ISSUES READ THIS

If your running WPAD and have on your navigator to auto detect proxy you WILL get the webGUI of pfSense instead of the GIF but disable the auto detect proxy on the navigator you then will see the GIF…It seems that WPAD overrides everything even when it comes to OpenVPN also...

BBcan177

@wiz561:

@beatstick:

Can the package be updated from the pfsense gui by clicking reinstall pfblockerng (pfsense 2.2.6)? Will my settings be kept? I can't find a download file on the github page on first look, which is normal, as github seems to make a point of confusing anyone who just wants to find a simple installer ;)

Normally, I think packages get upgraded/updated through the package manager. You have to manually update the packages by clicking on the "reinstall the package" button. However, I did this and I'm still on 2.0.4.

Also, my settings didn't get removed, but it's always a good idea to backup the config first.

The two PR (#87 and #1243) above are called "Pull Requests"… So basically, I submit my code changes to the pfSense Devs for review in Github. Each pfSense version has its own specific repository in Github.

Once the devs have time to review my changes, they can either request changes, or merge the code; at which time, you will see in pfSense packages: Installed Packages: that the package is available to be re-installed to the latest version.

There are some advanced methods to add the Github commits to System Patches, but its best to wait for the Devs to review and approve the changes…

I will also post a message once it has been approved.

Nachtfalke

@killmasta93:

FOR PEOPLE WITH GIF ISSUES READ THIS

If your running WPAD and have on your navigator to auto detect proxy you WILL get the webGUI of pfSense instead of the GIF but disable the auto detect proxy on the navigator you then will see the GIF…It seems that WPAD overrides everything even when it comes to OpenVPN also...

Thanks for that information. I am using WPAD with squid3 proxy. But I only get the WebUI with http websites. Is it working for you with httpS websites, too?
Can you describe if you configued something sepcidal to get pfblocker-NG + squid (http + https) working?

For me it looks like it is not working fpr https and it ios not working for http and https to see the bad websites in the "Alerts" tab of pfblopcker-ng.

Regards

killmasta93

Well I have pfBlockerNG working just for IP I had to disable DNSBL because WPAD was overriding everything. Because WPAD files are hosted on pfSense I had to revert to HTTP on the webGUI of pfSense. But by disabling auto detect proxy then works DNSBL but then WPAD wont work BUT transparent proxy will. I guess having WPAD and DNSBL wont work at the same time. UNLESS somehow on the wpad.dat a way to direct or ignore the 10.10.10.1 something that i wish it can do with OpenVPN. You get the HTTPS sites i guess because WPAD overlaps that but the http that would be the transparent proxy which DNSBL does not have an issue. Also depends if your webGUI is HTTPS or HTTP

Nachtfalke

@killmasta93:

Well I have pfBlockerNG working just for IP I had to disable DNSBL because WPAD was overriding everything. Because WPAD files are hosted on pfSense I had to revert to HTTP on the webGUI of pfSense. But by disabling auto detect proxy then works DNSBL but then WPAD wont work BUT transparent proxy will. I guess having WPAD and DNSBL wont work at the same time. UNLESS somehow on the wpad.dat a way to direct or ignore the 10.10.10.1 something that i wish it can do with OpenVPN. You get the HTTPS sites i guess because WPAD overlaps that but the http that would be the transparent proxy which DNSBL does not have an issue. Also depends if your webGUI is HTTPS or HTTP

Hi,

same situation in my environment. I changes from HTTPS WebUI to HTTP because WPAD is hosted on pfsense itself. But i defined all private IPs to go DIRECT and not using the proxy in the WPAD files. But unfortunately it does not work.

Further I disabled auto proxy configuration and hardcoded the proxy in my browser but DNSBL still does only work with HTTP (pfsense WebUI page) but not with HTTPS. Further it does not log anything :-(

PS:
Anybody here who could suggest a good DNSBL which focuses on command and control server and such stuff?

killmasta93

Hi,

same situation in my environment. I changes from HTTPS WebUI to HTTP because WPAD is hosted on pfsense itself. But i defined all private IPs to go DIRECT and not using the proxy in the WPAD files. But unfortunately it does not work.

Further I disabled auto proxy configuration and hardcoded the proxy in my browser but DNSBL still does only work with HTTP (pfsense WebUI page) but not with HTTPS. Further it does not log anything :-(

PS:
Anybody here who could suggest a good DNSBL which focuses on command and control server and such stuff?

hmm…are you getting the gif? does it log the gif? are you sure you disabled the WPAD?

Nachtfalke

@killmasta93:

Hi,

same situation in my environment. I changes from HTTPS WebUI to HTTP because WPAD is hosted on pfsense itself. But i defined all private IPs to go DIRECT and not using the proxy in the WPAD files. But unfortunately it does not work.

Further I disabled auto proxy configuration and hardcoded the proxy in my browser but DNSBL still does only work with HTTP (pfsense WebUI page) but not with HTTPS. Further it does not log anything :-(

PS:
Anybody here who could suggest a good DNSBL which focuses on command and control server and such stuff?

hmm…are you getting the gif? does it log the gif? are you sure you disabled the WPAD?

Hi killmasta93,

yes, as I said above DNSBL only works for HTTP butw not for HTTPS. Further it only shows me the WebUI (HTTP) but it does not log anything.
I am pretty sure that I disabled WPAD. I disabled the "Automatic discovery" option in chrome and IE (both browsers use the same proxy config menue) and then I closed my browsers and opened them again. At least I disbaled my network adapter and enabled it again to make sure that I am getting new information from DHCP and DNS.

So when I try to access the internet I still have to go through the proxy for http and https. I am NOT using TRANSPARENT proxy. Further I allowed port 8443 as an allowed port for squid SSL.

So the behavour is strange that it differs from yours even if it seems we have the same configuration.

killmasta93

Hi Nachtfalke,

So I have been trying to get it work and nothing, But this is what got my eye. I did a fresh install pfSense 2.2.4 installed the lasted pfBlockerNG and nothing wont log either the ads BUT it blocks them somehow. The GIF is the only alert that shows. Im thinking since i Have updated to the newest pfBlocker this is what happened. I also check on another pfSense box that was running pfBlocker 1.0 i updated also to the newest and wont work either. That box does no have any packages besides pfBlocker

BBcan177

@BBcan177:

I have posted PR #1243, pfBlockerNG v2.0.5 (for pfSense v2.2.x)
and
I have also posted PR #87, pfBlockerNG v2.0.8 (for pfSense v2.3)

Changelog can be seen in the attached links. If you have any issues, post back in the forum.

I highly recommend installing pfSense 2.3. Its nearing Release Candidate (RC) and is really looking sharp. Its getting harder to maintain pfBlockerNG in two different platforms, so I may concentrate my future efforts in pfSense 2.3. So it will most likely see all of the new upcoming features. I will however maintain pfBlockerNG in 2.2.x that are bug fixes.

Thanks!

Both PR #1243 and #87 have been merged for pfSense 2.2.x and 2.3.x respectively.

You can click on the PR links above to review the changes. Any questions, please let me know…

Looking forward to your feedback and hope you guys checkout pfSense 2.3 !!!

Wolf666

On 2.3 beta since the beginning, pfBlockerNG is working pretty fine, grazie for your work BBcan177!

Fesoj

Hi!

I read the info about 2.3 and I shall update asap. For the time being I'd like to update to 2.0.5 on an older 2.2. box. The installation aborted with the message

Downloading https://files.pfsense.org/packages/10/All/pfblockerng-1.6.6-amd64.pbi ...  could not download from there

It seems that the .pbi file is missing. Did someone else run into the same problem?

BBcan177

@Fesoj:

Hi!

I read the info about 2.3 and I shall update asap. For the time being I'd like to update to 2.0.5 on an older 2.2. box. The installation aborted with the message
Downloading https://files.pfsense.org/packages/10/All/pfblockerng-1.6.6-amd64.pbi ...  could not download from there
It seems that the .pbi file is missing. Did someone else run into the same problem?

I can download the file from the shell without issue:

fetch -o /tmp/pfblockerng-1.6.6-amd64.pbi https://files.pfsense.org/packages/10/All/pfblockerng-1.6.6-amd64.pbi

Fesoj

This time the file was found.

adx442

Haven't had issues with pfBlockerNG on 2.2.x, but I've upgraded to 2.3RC (and I've tried uninstalling/reinstalling pfB), but I get this error and a pfSense crash report every time I click the Alerts tab under pfBlockerNG's section:

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 72 bytes) in /usr/local/www/pfblockerng/pfblockerng_alerts.php on line 581 Call Stack: 0.0141 232168 1. {main}() /usr/local/www/pfblockerng/pfblockerng_alerts.php:0 1.4219 2882400 2. conv_log_filter_lite() /usr/local/www/pfblockerng/pfblockerng_alerts.php:362 1.4220 2883016 3. exec() /usr/local/www/pfblockerng/pfblockerng_alerts.php:581 PHP ERROR: Type: 1, File: /usr/local/www/pfblockerng/pfblockerng_alerts.php, Line: 581, Message: Allowed memory size of 268435456 bytes exhausted (tried to allocate 72 bytes)

This is on a Netgate C2758 box with 8GB of RAM and a small SSD with plenty of space, so this error doesn't make much sense to me. Is there a hard-set limit somewhere that I can adjust upwards?

BBcan177

Hi adx442,

The Alerts tab reads the pfSense Firewall logs. How many log entries do you have defined in the syslog settings? Also try to clear the firewall log and see if the error returns. The memory issue is related to PHP and not the hardware itself.

adx442

Yup, I'd already increased PHP's memory limit to 512M. Turns out, the syslog size was a little too large for even that amount of memory, cutting it in half and clearing the logs did fix the behavior.

Sorry, the only place I'd been seeing this issue exposed was in pfBlockerNG, though it was unrelated in the end.