PfBlockerNG v2.1 w/TLD

hulleyrob

@hulleyrob:

Check what is selected in this tab, as i had a similar problem and found since the update that the inverse of what i had previously selected had been selected causing over 1.5M IP's for this section and using up all the available memory.

Rob

Hi Rob,

The memory issue will be fixed with v2.1.1_3, however, you don't want to reverse the "Registered" vs the "Represented" entries. Please refer to the link in the GeoIP tabs "Whats new in GeoIP2" to help you understand the difference between those two types.

Not sure if we are talking about the same thing there, i was referring to the North American IPV6 tab which my selection from a few countries which i picked changed to unselecting those ones and selecting all the ones i had not picked.

This is what i meant by it had reversed the selection and seemed to be the cause of all my memory use as you have pointed out its is now several million.

Rob

BBcan177

pfBlockerNG v2.1.1_3 has been approved and merged and is now available to be installed/Upgraded.

I have noticed that some of the installation log messages are not appearing in the pkg install window. I am investigating that; however, the installation is still occurring in the background.

The MaxMind conversion will take a few mins to process, so wait for it to complete.

UPDATE

I pushed a fix for this just now. The pfBlockerNG version is now 2.1.1_4

Qinn

@BBcan177:

pfBlockerNG v2.1.1_3 has been approved and merged and is now available to be installed/Upgraded.

I have noticed that some of the installation log messages are not appearing in the pkg install window. I am investigating that; however, the installation is still occurring in the background.

The MaxMind conversion will take a few mins to process, so wait for it to complete.

UPDATE

I pushed a fix for this just now. The pfBlockerNG version is now 2.1.1_4

Thanks I noticed that the log while updating to 2.1.1_3 didn't gave sign it finished, after updating to 2.1.1_4 all seems well ;)

btw I would like to test a php /usr/local/www/pfblockerng/pfblockerng.php dc but as I have dramatically changed the hardware I cannot compare it to when the memory issues occurred (see https://forum.pfsense.org/index.php?topic=102470.750 )!

Cheers Qinn

BBcan177

@Qinn:

Thanks I noticed that the log while updating to 2.1.1_3 didn't gave sign it finished, after updating to 2.1.1_4 all seems well ;)

It was still running in the background (v2.1.1_3). The issue was that it wasn't printing the log messages to the installation window. So if you would have left it running for a min or so, it would have completed. Its now fixed in v2.1.1_4

btw I would like to test a php /usr/local/www/pfblockerng/pfblockerng.php dc but as I have dramatically changed the hardware I cannot compare it to when the memory issues occurred (see https://forum.pfsense.org/index.php?topic=102470.750 )!

The code was re-factored to not use as much PHP memory, so hopefully no one else runs into those issues :) Still hoping that MaxMind fixes the issues that caused those two Countries IPv6 entries to explode 5 fold….

Qinn

@BBcan177:

@Qinn:

Thanks I noticed that the log while updating to 2.1.1_3 didn't gave sign it finished, after updating to 2.1.1_4 all seems well ;)

It was still running in the background (v2.1.1_3). The issue was that it wasn't printing the log messages to the installation window. So if you would have left it running for a min or so, it would have completed. Its now fixed in v2.1.1_4

btw I would like to test a php /usr/local/www/pfblockerng/pfblockerng.php dc but as I have dramatically changed the hardware I cannot compare it to when the memory issues occurred (see https://forum.pfsense.org/index.php?topic=102470.750 )!

The code was re-factored to not use as much PHP memory, so hopefully no one else runs into those issues :) Still hoping that MaxMind fixes the issues that caused those two Countries IPv6 entries to explode 5 fold….

Thanks for the quick relpy! Yeah, that's the hardest part of coding, making it idiot proof for both users and resources. In this case it was the latter ;)

Qinn

Maybe it's n=1 and is it just me, but after update to 2.1.1_4 unbound won't come up, I did a reboot let's wait and see.

jrdnlc

Anyone have issues with pfBlocker and Playstation 4 online gaming? While playing online games lag a lot that the only fix was to disable pfb.
The logs show nothing of what exactly is blocking it.

Is there a way to exclude the PS4 to not use the service?

BBcan177

@jrdnlc:

Anyone have issues with pfBlocker and Playstation 4 online gaming? While playing online games lag a lot that the only fix was to disable pfb.
The logs show nothing of what exactly is blocking it.

Is there a way to exclude the PS4 to not use the service?

Did you review the pfBlockerNG Alerts Tab? If its being blocked via an IP List, it will show in the logs. For DNSBL it should also show in the Alerts Tab. For DNSBL there are some further instructions listed in the DNSBL tab which can be seen when you click on the blue infoblock icon in the INFO section. If it is being blocked by DNSBL and you can't find the Domain thats being blocked, you can set the DNS settings of the LAN device to a different DNS server to bypass DNSBL.

brandur

Hi @BBcan177
I just wanted to inform you that the info link/icon (to the right of the update ion), links to the wrong forum thread.

It's pointing to (pfBlockerNG v2.0 w/DNSBL): https://forum.pfsense.org/index.php?topic=102470.0
When it should be pointing to (pfBlockerNG v2.1 w/TLD): https://forum.pfsense.org/index.php?topic=115357.0

pfSense-update_link.JPG_thumb

BBcan177

@brandur:

Hi @BBcan177
I just wanted to inform you that the info link/icon (to the right of the update ion), links to the wrong forum thread

Thanks good catch! Will change that when I submit the next release :)

Wow! This thread had over 1000 views since last night ;)

Qinn

@BBcan177:

@brandur:

Hi @BBcan177
I just wanted to inform you that the info link/icon (to the right of the update ion), links to the wrong forum thread

Thanks good catch! Will change that when I submit the next release :)

Wow! This thread had over 1000 views since last night ;)

That's high 1k over night! For now 2.1.1_4 is running 22 hours without a flaw ;) . I did a php /usr/local/www/pfblockerng/pfblockerng.php dc and all went right (took about 25 min, but that is to be expected as of the dramatic raise of the resources of MaxMind).

pftdm007

First of all sorry if this is not in the right forum thread, there is now 3+ active threads for pfbng…

My problem is with the latest release (2.1.1_4) so I figured this is the right location to post.

This morning I got the notification that 2.1.1_4 was released which would fix the late php error problems caused by MaxMind. I immediately updated my package then started pfblockerNG. Then I went to the force update and did a force update. All went well, then I did a force reload. At this moment, the hard drive went crazy for 10min+ and I lost all network connectivity. Lost contact with pfsense, LAN connectivity and of course lost connectivity to the internet.

I rebooted the firewall (reset button) then it came back online. I immediately deactivated pfbng. After that I got these errors by email:


 There were error(s) loading the rules: /tmp/rules.debug:53: cannot define table pfB_Top_v6: Cannot allocate memory - The line in question reads [53]: table <pfb_top_v6>persist file "/var/db/aliastables/pfB_Top_v6.txt"

There were error(s) loading the rules: /tmp/rules.debug:199: macro 'pfB_Africa_v4' not defined - The line in question reads [199]: block log  quick  on {  em5  } inet from $pfB_Africa_v4 to any tracker 1770009617  label "USER_RULE: pfB_Africa_v4 auto rule"</pfb_top_v6>

RonpfS

Take a look a /var/log/pfblockerng/extras.log, /var/log/pfblockerng/pfblockerng.log, Status / System Logs / System / General, Status / System Logs / System / DNS Resolver, Dashboard for crash report.

Resolver log won't tell much. On reboot you have to go to Status / Services and restart the unbound service. After the restart, the log will have unbound messages.

rajl

@lpallard:

First of all sorry if this is not in the right forum thread, there is now 3+ active threads for pfbng…

My problem is with the latest release (2.1.1_4) so I figured this is the right location to post.

This morning I got the notification that 2.1.1_4 was released which would fix the late php error problems caused by MaxMind. I immediately updated my package then started pfblockerNG. Then I went to the force update and did a force update. All went well, then I did a force reload. At this moment, the hard drive went crazy for 10min+ and I lost all network connectivity. Lost contact with pfsense, LAN connectivity and of course lost connectivity to the internet.

I rebooted the firewall (reset button) then it came back online. I immediately deactivated pfbng. After that I got these errors by email:
 There were error(s) loading the rules: /tmp/rules.debug:53: cannot define table pfB_Top_v6: Cannot allocate memory - The line in question reads [53]: table <pfb_top_v6>persist file "/var/db/aliastables/pfB_Top_v6.txt"

There were error(s) loading the rules: /tmp/rules.debug:199: macro 'pfB_Africa_v4' not defined - The line in question reads [199]: block log  quick  on {  em5  } inet from $pfB_Africa_v4 to any tracker 1770009617  label "USER_RULE: pfB_Africa_v4 auto rule"</pfb_top_v6> 

I'm having a very similar problem. I had uninstalled pfblockerng using the package manager and was waiting for an update to fix the memory problems. When I installed the latest version, I began getting the following errors:


There were error(s) loading the rules: /tmp/rules.debug:27: cannot load "/var/db/aliastables/pfB_NAmerica_v4.txt": No such file or directory - The line in question reads [27]: table <pfB_NAmerica_v4> persist file "/var/db/aliastables/pfB_NAmerica_v4.txt" @ 2016-08-24 21:03:13
There were error(s) loading the rules: /tmp/rules.debug:27: cannot load "/var/db/aliastables/pfB_NAmerica_v6.txt": No such file or directory - The line in question reads [27]: table <pfB_NAmerica_v6> persist file "/var/db/aliastables/pfB_NAmerica_v6.txt" @ 2016-08-24 21:03:24
There were error(s) loading the rules: /tmp/rules.debug:178: macro 'pfB_NAmerica_v4' not defined - The line in question reads [178]: block in log quick on $WAN reply-to ( re0 174.49.92.1 ) inet from ! $pfB_NAmerica_v4 to any tracker 1770009560 label "USER_RULE: pfB_NAmerica_v4 auto rule" @ 2016-08-24 21:03:27
There were error(s) loading the rules: /tmp/rules.debug:178: macro 'pfB_NAmerica_v4' not defined - The line in question reads [178]: block in log quick on $WAN reply-to ( re0 174.49.92.1 ) inet from ! $pfB_NAmerica_v4 to any tracker 1770009560 label "USER_RULE: pfB_NAmerica_v4 auto rule" @ 2016-08-24 21:03:30

The end result for me is that my white list rule allowing only inbound traffic from the U.S. fails to load. However, I have no problems with other features, (e.g., adblocking). No errors show up in extras.log or pfblockerng.log.

BBcan177

When you uninstalled the pkg previously, did you uncheck "Keep Settings"… If not, some files may have remained.

I would suggest you goto the pfBlockerNG General tab, and uncheck "Enable pfBlockerNG" and uncheck "Keep Settings", followed by "Save"... Then reverse this by re-checking both options and "Save".

Goto the Dashboard and clear any notices so that you are starting fresh...

Then goto the Update tab and run a "Force Update".

Then review the pfblockerng.log for any issues (if any).

pftdm007

OK so I tried unchecking the "Keep settings" and "Enable pfb" checkboxes then saving. Then I checked them back on and did a force update. The process never ended. 45minutes later, everything was dead and the last thing I could see on the WebUI was "Rstsarting Unbound".

The hard drive gpoes completely off charts while this happens. I tried getting the system logs after the hard reset but it goes only up to 22:35 which is already 5 minutes after I manually reset the pfsense box.

Tomorrow I will try to simulate this once more, and gather all logs I will be able to find. My feeling, somehow, since I lose all network connectivity, is that unbound crashes hard probably due to lack of RAM?? Is it even possible? I am saying that because when this happens I have network connectivity for a few minutes then everything drops. Then I cant even connect to my internal clients (same subnet).

RonpfS

If outbound crashes, you should still be able to access the FW by it's IP. So open one tab in your browser using the FW IP and have Diagnostics / System activity open so you can see what is happening while you run Force Reload on another tab with the FW FQDN.
Again, you won't get any log from Resolver(unbound) if you do not restart it right after reboot.

Could you be running out of disk space ? Do you have /var in RAM Disk? Maybe your hard disk is failing.

Before enabling pfBlockerNG, disable the tables and enable them progressively to pinpoint the problem.

Then before enabling DSNBL, disable the tables and go progressively until the issue appears.

pftdm007

BBcan177, private email sent.

spittlbm

Same issues as lpallard. I mostly resolved this by bumping Max table entries above 2MM and disabling/enabling PFBNG.

blueduckdock

Trying to figure out pfblocker on CARP….

I've used this extensively on single installs but not via CARP. Are there any considerations I should take into account? I was told by pfsense support when I first installed that firewalls should mirror (ie. have pfblocker installed on both, etc.)
Am I ok to configure FW1 on 10.0.10.1 with whatever pfblocker stuff I want then simply sync to 10.0.10.2 (FW2?) I don't have to worry about the CARP interface or sync issues between this package and that right (let's say CARP interface is on 10.0.10.250)