PfBlockerNG v2.0 w/DNSBL

fookhk

[quote]
Hi - I'm having some trouble with pfBlockerNG updates when DNSBL is enabled. The update process spikes my CPU to 100% for more than an a hour. I've tried manual updates as well as CRON, but no success in completing the updates.  Any thoughts? 

**Saving configuration [ 06/23/16 21:16:18 ] ...

Adding Unbound Server:Include line... completed
Validating database... completed 
Reloading Unbound ... completed
DNSBL update [ 0 ]... completed 
------------------------------------------
DNSBL - Adding Unbound custom 'include' option

Saving new DNSBL web server configuration to port [ 8081 & 8443 ]
Saving pfSense config...
VIP address configured. Widget Packet statistics reset.
Restarting Service DNSBL...
 UPDATE PROCESS START [ 06/23/16 21:16:30 ]

From the portion of the log that you posted above, the DNSBL count is "0"... Did you add any DNSBL feeds?

Run a [b]top -SH[/b] command from the shell and see whats eating memory...
[/quote]

Hi BBcan177

I added another DNSBL feed and have started the update process.  Here is the results from top -SH 

last pid: 57837;  load averages:  1.10,  1.03,  0.69                                                                        up 1+21:05:30  18:56:13
202 processes: 6 running, 166 sleeping, 30 waiting
CPU: 25.3% user,  0.0% nice,  0.1% system,  0.4% interrupt, 74.2% idle
Mem: 380M Active, 2656M Inact, 755M Wired, 668M Buf, 4027M Free
Swap: 16G Total, 16G Free

  PID USERNAME   PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
32798 root       103    0   220M 33536K CPU0    0  11:35 100.00% php
   11 root       155 ki31     0K    64K CPU1    1  43.6H  88.67% idle{idle: cpu1}
   11 root       155 ki31     0K    64K RUN     0  42.6H  84.67% idle{idle: cpu0}
   11 root       155 ki31     0K    64K CPU2    2  43.7H  71.00% idle{idle: cpu2}
   11 root       155 ki31     0K    64K RUN     3  43.7H  63.18% idle{idle: cpu3}
98665 root        20    0  2023M  1872M uwait   1 109:34   1.07% suricata{FlowManagerThre}
55524 clamav      20    0   787M   701M select  2  15:30   0.00% clamd{clamd}
98665 root        20    0  2023M  1872M nanslp  1  13:59   0.00% suricata{suricata}
98665 root        20    0  2023M  1872M uwait   2  13:53   0.00% suricata{Detect2}
98665 root        20    0  2023M  1872M uwait   1  11:40   0.00% suricata{Detect1}
98665 root        20    0  2023M  1872M uwait   3   9:32   0.00% suricata{Detect3}

wolfsden3

I have multiple pfsense firewalls at multiple locations, on various versions. My newest ones are on 2.3.1 release p1, I see there is now a p5. My problem with pfBlocker is that on this newer version, pfblocker is on version 2.0.17 whereas I have an older version of pfsense, version 2.2.5 that has pfblocker 2.0.2. The problem I am having is that the newer pfsense is not offering an update to pfblocker, and I am comparing versions due to a problem with the DNSBL allowing me to sites on the newer pfblocker with the older pfsense but blocking my clients on the newer pfsense with the older pfblocker. Why would it not offer an update to the package for pfblocker?

BBcan177

@wolfsden3:

I have multiple pfsense firewalls at multiple locations, on various versions. My newest ones are on 2.3.1 release p1, I see there is now a p5. My problem with pfBlocker is that on this newer version, pfblocker is on version 2.0.17 whereas I have an older version of pfsense, version 2.2.5 that has pfblocker 2.0.2. The problem I am having is that the newer pfsense is not offering an update to pfblocker, and I am comparing versions due to a problem with the DNSBL allowing me to sites on the newer pfblocker with the older pfsense but blocking my clients on the newer pfsense with the older pfblocker. Why would it not offer an update to the package for pfblocker?

I am not actively updating the package for the 2.2.x branch… Just not enough time in the day to support multiple branches... 2.0.17 is only available for 2.3.x. If you are not seeing the new version in the 2.3.x branch, then try to execute a "pkg update -f" command... if your still having issues with that, then try to post in the pfSense "Installation and Upgrade" Threads and see if one of the Devs might have some answers for you...

wolfsden3

@BBcan177:

@wolfsden3:

I have multiple pfsense firewalls at multiple locations, on various versions. My newest ones are on 2.3.1 release p1, I see there is now a p5. My problem with pfBlocker is that on this newer version, pfblocker is on version 2.0.17 whereas I have an older version of pfsense, version 2.2.5 that has pfblocker 2.0.2. The problem I am having is that the newer pfsense is not offering an update to pfblocker, and I am comparing versions due to a problem with the DNSBL allowing me to sites on the newer pfblocker with the older pfsense but blocking my clients on the newer pfsense with the older pfblocker. Why would it not offer an update to the package for pfblocker?

I am not actively updating the package for the 2.2.x branch… Just not enough time in the day to support multiple branches... 2.0.17 is only available for 2.3.x. If you are not seeing the new version in the 2.3.x branch, then try to execute a "pkg update -f" command... if your still having issues with that, then try to post in the pfSense "Installation and Upgrade" Threads and see if one of the Devs might have some answers for you...

Okay, that did not work. So the newer pfblocker versions are only available for the older pfsense versions? For your records as well, we see all of our newer pfsense versions not properly logging the DNSBL alerts as well. Maybe we will back date our pfsense versions as the pfblocker is our most important tool.

BBcan177

@wolfsden3:

Okay, that did not work. So the newer pfblocker versions are only available for the older pfsense versions? For your records as well, we see all of our newer pfsense versions not properly logging the DNSBL alerts as well. Maybe we will back date our pfsense versions as the pfblocker is our most important tool.

@wolfsden3

For pfSense 2.2.x, the package version is 2.0.6
For pfSense 2.3.x, the package version is 2.0.17

If you are not seeing the package, then you are having some connectivity issues connecting to the pfSense repos.

Typically when DNSBL logging is not working, it's due to configuration issues. Ensure that the LAN devices only have the pfSense Resolver as its DNS server. Ensure that you can ping the DNSBL VIP, and also browse to the DNSBL VIP and get the 1x1 pix.

Pippin

@XmickS:

Just one thing I can't get done. Thats blocking those annoying ads on youtube. Is that possible with dnsbl? I use the easylists Cameleon, adaway yoyo and hostfile. I haven't configured ip blocklists yet.

@Deadpool:

same here. haven't found a way yet.
does somebody tried the blocking lists from example adblock plus addon for firefox/chrome?
would that work?

Is working here.

I have a Synology NAS on a remote location (family) with OpenVPN and DNS Server Package installed.
I use a script which uses the yoyo list to block ads when VPNed in. The yoyo list is updated through a scheduled task. Thats working pretty well for quite some time (also for the family).

On pfS it didn`t work at first.
With pfBlockerNG/DNSBL I just found the problem to be on the TAB DNSBL Easylist.
When I disabled those it started to work.
I use only the yoyo list on DNSBL Feeds.

Hope it helps.

Deadpool

@Pippin:

I use a script which uses the yoyo list to block ads when VPN`ed in.

how do you do that? can you explain?

nathulal

So I have setup two DNSBL feeds. One for AdBlock and another for Adult sites block. Both feeds work correctly and sites are blocked as appropriate. However I am using the following feed source for Adult sites block.

http://dsi.ut-capitole.fr/blacklists/download/adult.tar.gz

This file is very big and has a lot of hosts defined. When the cron job runs I notice that the update/download of the job is successful but sometimes (almost 50% of the times) the unbound service is stuck on stopped. Due to this all DNS resolutions fail until in go to the webgui and start up the unbound service again.

If I disable the Adult Block feed all is fine and there are not issues after update. Any ideas on what I can do to help figure out the issue?

Thanks

blewis

I've got pbBlockerNG v2.0.17 using the latest 2.3.1-RELEASE-p5 (i386) on an Alix2d3. Each time I Reboot or unplug the system and even after waiting for half an hour, I still cannot get online. I had to manually Select 'Force' option Run in pfBlockerNG inorder to get back online to the internet.

I use it to block ads only, nothing more and CRON Settings is set to Once a day.

Is it possible to store & restore the block lists from the CF card? And why do I lose Internet access without forcing a cron update? Shouldn't I get Internet access, but without the ad block? Is it because this particular Alix does not have a battery backed up Real-Time Clock (The Alix2D13 added a RTC battery)?

Thanks..

BBcan177

@Pippin:

I have a Synology NAS on a remote location (family) with OpenVPN and DNS Server Package installed.
I use a script which uses the yoyo list to block ads when VPN`ed in.
The yoyo list is updated through a scheduled task.

Hi Pippin,

For OpenVPN, as long as you configure the pfSense DNS servers for the VPN, then DNSBL should filter those DNS resolutions… Shouldn't need to do any other work-arounds...

With pfBlockerNG/DNSBL I just found the problem to be on the TAB DNSBL Easylist.
When I disabled those it started to work.
I use only the yoyo list on DNSBL Feeds.

You aren't going to get much filtering with just the yoyo feed by itself. Keep in mind that anything that is blocked is visible in the Alerts Tab (DNSBL window). You can also hit F12 in the Browser and goto "Console" to see whats being blocked.

BBcan177

@nathulal:

However I am using the following feed source for Adult sites block.
http://dsi.ut-capitole.fr/blacklists/download/adult.tar.gz

That feed has just over 1M domains and the balance (~100k) as IP addresses… I haven't tested it, but its going to tax the system depending on how many domains are in the other DNSBL feeds....

How much memory do you have in the pfSense Box? Can you bump it up?

BBcan177

@blewis:

I've got pbBlockerNG v2.0.17 using the latest 2.3.1-RELEASE-p5 (i386) on an Alix2d3. Each time I Reboot or unplug the system and even after waiting for half an hour, I still cannot get online. I had to manually Select 'Force' option Run in pfBlockerNG inorder to get back online to the internet.

A few versions ago, I added a feature to backup the /var/unbound/pfb_dnsbl.conf file and restore it on reboot. This is only needed for Nano and RAMDisk installs as on reboot, the /var and /tmp folders are wiped.

The only reason that I can think that would cause unbound not to start, would be that the /var/unbound/pfb_dnsbl.conf file is missing after reboot. Please confirm…

If you open the Resolver GUI config, you will see the following line "server:include: /var/unbound/pfb_dnsbl.conf"... If the file is missing, it will not load and you either need to remove that line in the Adv. field, or run a "Force Update" cmd...

Are you properly shutting down the box, or just pulling the plug?

Pippin

O.T.
@Deadpool:

how do you do that? can you explain?

Follow these links for the Synology NAS:
https://forum.synology.com/enu/viewtopic.php?f=39&t=78126&p=310721#p310721
https://synologytweaks.wordpress.com/2015/08/23/use-synology-as-an-ad-blocker

Deadpool

haven't read everything, but could this also be applied if you don't rock a synology nas?

has anyone tried that?

BBcan177

@Pippin:

O.T.
@Deadpool:

how do you do that? can you explain?

Follow these links for the Synology NAS:
https://forum.synology.com/enu/viewtopic.php?f=39&t=78126&p=310721#p310721
https://synologytweaks.wordpress.com/2015/08/23/use-synology-as-an-ad-blocker

I don't think its a good idea to have your NAS provide DNS resolution for your Network.

You are potentially opening yourself up to security implications by doing this…

theeddie

Not sure what thread to post this in
http://someonewhocares.org/hosts/hosts
A good list I have found that is updated. for DNSBL ADVERTS

Pippin

@BBcan177:

I don't think its a good idea to have your NAS provide DNS resolution for your Network.

You are potentially opening yourself up to security implications by doing this…

Appreciate the input.
The NAS is in a remote location and except for OpenVPN its not open to the world. Its my backup target and also provides add blocking DNS over OpenVPN that runs on it.

wolfsden3

@BBcan177:

@wolfsden3:

Okay, that did not work. So the newer pfblocker versions are only available for the older pfsense versions? For your records as well, we see all of our newer pfsense versions not properly logging the DNSBL alerts as well. Maybe we will back date our pfsense versions as the pfblocker is our most important tool.

@wolfsden3

For pfSense 2.2.x, the package version is 2.0.6
For pfSense 2.3.x, the package version is 2.0.17

If you are not seeing the package, then you are having some connectivity issues connecting to the pfSense repos.

Typically when DNSBL logging is not working, it's due to configuration issues. Ensure that the LAN devices only have the pfSense Resolver as its DNS server. Ensure that you can ping the DNSBL VIP, and also browse to the DNSBL VIP and get the 1x1 pix.

I updated to the newest package of Pfblocker. How do I clear the alerts currently showing, outdated as of July 1st, to see if they repopulate as new alerts?

wolfsden3

@BBcan177:

@wolfsden3:

Okay, that did not work. So the newer pfblocker versions are only available for the older pfsense versions? For your records as well, we see all of our newer pfsense versions not properly logging the DNSBL alerts as well. Maybe we will back date our pfsense versions as the pfblocker is our most important tool.

@wolfsden3

For pfSense 2.2.x, the package version is 2.0.6
For pfSense 2.3.x, the package version is 2.0.17

If you are not seeing the package, then you are having some connectivity issues connecting to the pfSense repos.

Typically when DNSBL logging is not working, it's due to configuration issues. Ensure that the LAN devices only have the pfSense Resolver as its DNS server. Ensure that you can ping the DNSBL VIP, and also browse to the DNSBL VIP and get the 1x1 pix.

Update - I was able to clear the alerts by deselecting the "enable/disable" and "keep settings" boxes, saving, then re-check them. I did get updated log alerts. However, I did not get new ones after that, its as if the log gets full, is that possible? The installation is on a 60gb SSD that is only 6 months old or so now, and previously ran on a 4gb flash drive and lasted for about 2 or 3 years. Not sure if the hardware really has anything to do with the logging not working or not.

-Update- Logging worked for about 20 minutes, then stopped. It would actively update on what seemed to be about 30 second intervals. Is there a logging setting that needs to change?

BBcan177

@wolfsden3:

-Update- Logging worked for about 20 minutes, then stopped. It would actively update on what seemed to be about 30 second intervals. Is there a logging setting that needs to change?

The IP Alerts are taken from the pfSense Firewall Logs. So you can clear the firewall log to start fresh.

For DNSBL the log is taken from /var/log/pfblockerng/dnsbl.log

If you are getting sporatic DNSBL logging, ensure that your LAN device(s) only have pfSense as its DNS server, as any other DNS server settings could be round-robin to other DNS servers and will bypass DNSBL.

Ensure that you can ping the DNSBL VIP, and browse to the DNSBL VIP and get the 1x1 pix.