PfBlockerNG v2.1 w/TLD

Andrew453

I've had that trouble before too. It happened when I was trying to change from the development thread for updates to the stable thread. I couldn't update anything. I eventually found some instructions to reinstall the main pfsense components from the command line. I ended up still on the development thread and didn't venture to try to change it back after that.

pftdm007

@Andrew453:

I've had that trouble before too. It happened when I was trying to change from the development thread for updates to the stable thread. I couldn't update anything. I eventually found some instructions to reinstall the main pfsense components from the command line. I ended up still on the development thread and didn't venture to try to change it back after that.

Not sure I understand that. I am not playing with development stuff, nor that I am configured to retrieve packages from development repos.. Just a vanilla pfsense install with pfblockerNG, snort and thats it. Not normal all of a sudden I lose connection to repos..

Also after a reboot I see these warnings in the main page:

There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:00
There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:11
There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:20

BBcan177

@lpallard:

There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:00
There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:11
There were error(s) loading the rules: /tap/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:20

These all seem to be related to the MaxMind IPv6 database. Looks like you will need to bump the pfSense max aliastable entries limit from 2M to 4M. If you enable aggregation in the general tab, it should condense the CIDRs and reduce the overall IP count. This changed due to using the new MaxMind Geolite2 database which seems to have smaller subsets of the data listed causing more IP entries to be added.

BBcan177

@CiscoX:

After updating to 2.1.1_2 i can't "clear DNSBL Packets" from the pfBlockerNG widge
The DNSBL_EasyList won't delete the packets

I am away for a few weeks but will check that out. Seems like some regression somewhere. Thanks for reporting.

java007md

@lpallard:

Not sure if this is related to pfblockerNG (2.1 w/ TLD) but I went to the package manager to install a package, and saw that my copy of pfblockerNG was outdated, so I clicked the yellow round arrow to update the package. It went well, but immediately after I returned to the package manager I was greeted with a red ribbon saying "Unable to retrieve package information". This happens for the "Installed Packages as well as "Available Packages" tabs!

On the main page, I see "Obtaining update status ", then it turns to "Unable to check for updates".

Tried disabling both pfblockerNG and DNSBL to no avail. Snort is disabled and the blocked hosts list is empty.

Now I cannot update, install or uninstall packages…. How do I remedy to this?

From the following thread:

https://forum.pfsense.org/index.php?topic=116019.0

I followed the ssh command line execution steps:

pkg update -f
pkg upgrade -f

and the same problem was resolved.

MoonKnight

@BBcan177:

@CiscoX:

After updating to 2.1.1_2 i can't "clear DNSBL Packets" from the pfBlockerNG widge
The DNSBL_EasyList won't delete the packets

I am away for a few weeks but will check that out. Seems like some regression somewhere. Thanks for reporting.

Hi, No problem. Have a nice Holiday :)

Qinn

Hi there I followed this guide, http://fredmerc.com/2016/07/15/pfsense-adblock-using-pfblockerng-guide/ a rather short setup, there is only DNSBL and no IP4 is that new or is this guide missing it? Thanks for any help.

RonpfS

@Qinn:

Hi there I followed this guide, http://fredmerc.com/2016/07/15/pfsense-adblock-using-pfblockerng-guide/ a rather short setup, there is only DNSBL and no IP4 is that new or is this guide missing it? Thanks for any help.

Here are the original pfBlockerNG thread https://forum.pfsense.org/index.php?topic=86212.0
and the pfBlockerNG v2.0 w/DNSBL thread https://forum.pfsense.org/index.php?topic=102470

Heimire

I am getting this error when I try to use the Spamhaus list in this tread.

===[ DNSBL Process ]================================================

[ EasywoElements ] exists.
[ SpamHouse_TLDS ] Downloading update .. 200 OK
Remote timestamp missing .
–--------------------------------------------------------------------
Orig. Unique # Dups # White # Alexa Final
----------------------------------------------------------------------
3 3 0 0 0 3
----------------------------------------------------------------------

[ DNSBL FAIL ] [ Skipping : SpamHouse_TLDS ]

[1470071701] unbound-checkconf[87654:0] error: error parsing local-data at 38 '(xmlhttp.readystate 60 IN A 10.10.10.1': Syntax error, could not parse the RR
[1470071701] unbound-checkconf[87654:0] error: Bad local-data RR (xmlhttp.readystate 60 IN A 10.10.10.1
[1470071701] unbound-checkconf[87654:0] fatal error: failed local-zone, local-data configuration
[ Malware_1month ] Downloading update [ 08/01/16 12:15:01 ] .. 200 OK
Remote timestamp missing .
–--------------------------------------------------------------------
Orig. Unique # Dups # White # Alexa Final
----------------------------------------------------------------------
1221 956 0 0 0 956
----------------------------------------------------------------------

[ Malware_1week ] Downloading update [ 08/01/16 12:15:04 ] .. 200 OK
Remote timestamp missing .
–--------------------------------------------------------------------
Orig. Unique # Dups # White # Alexa Final
----------------------------------------------------------------------
526 487 487 0 0 0
----------------------------------------------------------------------

[ Malware_1day ] Downloading update [ 08/01/16 12:15:05 ] .. 200 OK
Remote timestamp missing .
–--------------------------------------------------------------------
Orig. Unique # Dups # White # Alexa Final
----------------------------------------------------------------------
48 47 47 0 0 0
----------------------------------------------------------------------

[ Malware_1hour ] Downloading update .. 200 OK
Remote timestamp missing
No Domains Found

–----------------------------------------
Assembling database... completed
Executing TLD
TLD analysis. completed
Finalizing TLD... completed

Original Matches Removed Final

6062 5530 1 6061

Validating database... completed [ 08/01/16 12:15:08 ]
Reloading Unbound…. completed
DNSBL update [ 6061 | PASSED ]… completed

RonpfS

Which Spamhaus URL are you using ?
this https://www.spamhaus.org/statistics/tlds/ is just a web page, not a feed DNSBL can use.

as for the H3X, only one is needed
https://forum.pfsense.org/index.php?topic=115357.msg643896#msg643896

And do a Force Reload after making the modifications.

Heimire

@RonpfS:

Which Spamhouse URL are you using ?
this https://www.spamhaus.org/statistics/tlds/ is just a web page, not a feed DNSBL can use.

as for the H3X, only one is needed
https://forum.pfsense.org/index.php?topic=115357.msg643896#msg643896

And do a Force Reload after making the modifications.

Thank you.
i see my mistake now.
I was certain I had 2 feeds that contained data but I must have misplaced it?

RonpfS

Read the first posts (or more ;)) of each of these threads:
pfBlockerNG
pfBlockerNG v2.0 w/DNSBL
pfBlockerNG v2.1 w/TLD

You will find some posts about IP and DNSBL Feed.

minority

First of all thank you very much for your hard work and this awesome package!

I was just wondering is it possible to somehow change the Rule Order setting to something like:
pfB_Pass/Match | pfB_Block/Reject | All other Rules | (original format)
so the first IP-list would be the whitelist?

Right now I can't seem to figure out how to make custom LAN IPv4 whitelist (Permit_Outbound) rule to be the first in the rule list of the LAN interface. If I manually move it first. Next list update puts it bellow the blocklists (Deny_Outbound) again. Right now only the default setting | pfB_Block/Reject | All other Rules | (Original format) is partly usable for me (whitelist won't work) and all other rule order settings just mess my original LAN rules.

I use Traffic Shaper queues in the floating rules so prefer not to move pfBlockerNG's rules in there too.

Is this somehow possible or what am I missing, thanks?

RonpfS

Which version are you using ?

with pfBlockerNG 2.1.1_2 I have these choices.

And you can still use the Floating Rules, it won't affect the Traffic Shaper rules.

rulepass.jpg_thumb

RonpfS

@Heimire:

@RonpfS:

Which Spamhouse URL are you using ?
this https://www.spamhaus.org/statistics/tlds/ is just a web page, not a feed DNSBL can use.

as for the H3X, only one is needed
https://forum.pfsense.org/index.php?topic=115357.msg643896#msg643896

And do a Force Reload after making the modifications.

Thank you.
i see my mistake now.
I was certain I had 2 feeds that contained data but I must have misplaced it?

The https://www.spamhaus.org/statistics/tlds/ page can be useful to find TLD to put in the TLD Blacklist.

hulleyrob

There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:00
There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:11
There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_Europe_v6: Cannot allocate memory - The line in question reads [37]: table <pfB_Europe_v6> persist file "/var/db/aliastables/pfB_Europe_v6.txt" @ 2016-07-31 14:55:20

Check what is selected in this tab, as i had a similar problem and found since the update that the inverse of what i had previously selected had been selected causing over 1.5M IP's for this section and using up all the available memory.

Rob

RonpfS

PFBlockerNG 2.1.1_2 Memory Errors

coolspot

When I try to add a new TLD Blacklist i.e. "Google.com", I get the following error:

Clearing all DNSBL Feeds… completed
Executing TLD
Blocking full TLD/Sub-Domain(s)... |google.com| completed
TLD analysis completed
Finalizing TLD... head: 1: No such file or directory
tail: 1: No such file or directory
completed

Original Matches Removed Final

0 0 -1 1

Validating database... completed

DNSBL enabled FAIL - restoring Unbound conf
/var/unbound/pfb_dnsbl.conf:1: error: unknown keyword '.google.com'
/var/unbound/pfb_dnsbl.conf:1: error: unknown keyword '60'
read /var/unbound/unbound.tmp failed: 2 errors in configuration file

Any ideas why DNSBL is failing to add the TLD blacklist entries?

Thanks.

RonpfS

Do you have any DNSBL feeds defined and enabled?
I did put Google.com in the DNSBL TLD Blacklist, ran a Force Reload and things looks ok.

This is the part of pfblockerNG log after the last DNSBL feed

[ BBC_C2 ]		 Reload [ 08/08/16 15:25:16 ] . completed ..
  ----------------------------------------------------------------------
  Orig.    Unique     # Dups     # White    # Alexa    Final                
  ----------------------------------------------------------------------
  332      332        331        0          0          1                    
  ----------------------------------------------------------------------

[ DNSBL_IP ]		 Updating aliastable [ 08/08/16 15:25:22 ]... 
  no changes.
  Total IP count = 280

------------------------------------------
Assembling database... completed
Executing TLD
 Blocking full TLD/Sub-Domain(s)... |google.com| completed
TLD analysis...xxxxxxxxxxx completed
** TLD Domain count exceeded. [ 250000 ] All subsequent Domains listed as-is **
Finalizing TLD...  completed
 ----------------------------------------
 Original    Matches    Removed    Final     
 ----------------------------------------
 1323464     87716      169286     1154178   
 -----------------------------------------
Validating database... completed [ 08/08/16 15:31:20 ]
Reloading Unbound.... completed
DNSBL update [ 1154178 | PASSED  ]... completed [ 08/08/16 15:32:02 ]
------------------------------------------

===[  Continent Process  ]============================================

coolspot

@RonpfS:

Do you have any DNSBL feeds defined and enabled?
I did put Google.com in the DNSBL TLD Blacklist, ran a Force Reload and things looks ok.

No, I only want to block a couple domains and not use any DNSBL lists.

Must I have a DNSBL list for TLD to work?

PfBlockerNG v2.1 w/TLD

–---------------------------------------- Assembling database... completed Executing TLD TLD analysis. completed Finalizing TLD... completed

Original Matches Removed Final

6062 5530 1 6061

Clearing all DNSBL Feeds… completed Executing TLD Blocking full TLD/Sub-Domain(s)... |google.com| completed TLD analysis completed Finalizing TLD... head: 1: No such file or directory tail: 1: No such file or directory completed

Original Matches Removed Final

0 0 -1 1

–----------------------------------------
Assembling database... completed
Executing TLD
TLD analysis. completed
Finalizing TLD... completed

Clearing all DNSBL Feeds… completed
Executing TLD
Blocking full TLD/Sub-Domain(s)... |google.com| completed
TLD analysis completed
Finalizing TLD... head: 1: No such file or directory
tail: 1: No such file or directory
completed