Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfBlocker

    Scheduled Pinned Locked Moved pfSense Packages
    896 Posts 143 Posters 1.4m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      taryezveb
      last edited by

      After trying out a few things all is fine for now. Disabled the TBG List and left the Bluetack enabled. Also in System -> Advanced -> Firewall and NAT; Firewall Maximum Table Entries, removed that large number[9999999999999999] from there. Restarted pfBlocker and now there are no error messages and Squid works as transparent proxy, even with pfBlocker enabled. So it seems the TBG list is the cause of all these issues I was having.

      TBG list:

      http://list.iblocklist.com/?list=ijfqtofzixtwayqovmxn&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=ecqbsykllnadihkdirsh&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=tbnuqfclfkemqivekikv&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=ewqglwibdgjttwttrinl&fileformat=p2p&archiveformat=gz

      Bluetack list:

      http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_level2&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_spyware&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_proxy&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_templist&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_hijacked&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_dshield&fileformat=p2p&archiveformat=gz
      http://list.iblocklist.com/?list=bt_bogon&fileformat=p2p&archiveformat=gz

      1 Reply Last reply Reply Quote 0
      • marcellocM
        marcelloc
        last edited by

        You may try if you want add each TBG list until you find what list get errors.

        Treinamentos de Elite: http://sys-squad.com

        Help a community developer! ;D

        1 Reply Last reply Reply Quote 0
        • T
          taryezveb
          last edited by

          @marcelloc:

          You may try if you want add each TBG list until you find what list get errors.

          Yes, I'm going to try that. I might spring for the i-block lists, need to do some research on that. Are the i-block lists any better than the public lists?

          1 Reply Last reply Reply Quote 0
          • T
            taryezveb
            last edited by

            Did some testing.

            With the following in the TBG list, whenever I try to enable them[one at a time], get the Fatal error below.

            http://list.iblocklist.com/?list=ijfqtofzixtwayqovmxn&fileformat=p2p&archiveformat=gz
            http://list.iblocklist.com/?list=ecqbsykllnadihkdirsh&fileformat=p2p&archiveformat=gz

            Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8650655 bytes) in /usr/local/pkg/pfblocker.inc on line 268

            Note, the Bluetack list is still enabled. When testing the above.

            1 Reply Last reply Reply Quote 0
            • marcellocM
              marcelloc
              last edited by

              the total download exceeds php memory limit.

              try to apply one list at time

              Treinamentos de Elite: http://sys-squad.com

              Help a community developer! ;D

              1 Reply Last reply Reply Quote 0
              • T
                taryezveb
                last edited by

                @marcelloc:

                the total download exceeds php memory limit.

                try to apply one list at time

                @taryezveb:

                whenever I try to enable them[one at a time]

                I did, or do you mean something else?

                1 Reply Last reply Reply Quote 0
                • marcellocM
                  marcelloc
                  last edited by

                  I mean that.  Sorry, I didn't saw the "[one at a time]"

                  Treinamentos de Elite: http://sys-squad.com

                  Help a community developer! ;D

                  1 Reply Last reply Reply Quote 0
                  • T
                    taryezveb
                    last edited by

                    @marcelloc:

                    I mean that.  Sorry, I didn't saw the "[one at a time]"

                    No problem and guess no more lists for me :(

                    1 Reply Last reply Reply Quote 0
                    • marcellocM
                      marcelloc
                      last edited by

                      The are some file hacks to increase php memory limit, but it is not supported or recommended by core team.

                      DO IT AT YOUR OWN RISK

                      edit ini_set("memory_limit","128M"); to 500M in /etc/inc/config.inc file

                      and include ini_set('memory_limit', '490M'); on pfblocker.inc

                      Do not increase it to a value greater then 510M, suhosin.memory_limit is set to 512M

                      Treinamentos de Elite: http://sys-squad.com

                      Help a community developer! ;D

                      1 Reply Last reply Reply Quote 0
                      • T
                        taryezveb
                        last edited by

                        @marcelloc:

                        edit ini_set("memory_limit","128M"); to 500M in /etc/inc/config.inc file

                        and include ini_set('memory_limit', '490M'); on pfblocker.inc

                        Do not increase it to a value greater then 510M, suhosin.memory_limit is set to 512M

                        I take it a reboot is needed for these changes to take affect?

                        1 Reply Last reply Reply Quote 0
                        • marcellocM
                          marcelloc
                          last edited by

                          Not at all. There is no changes on php.ini

                          Treinamentos de Elite: http://sys-squad.com

                          Help a community developer! ;D

                          1 Reply Last reply Reply Quote 0
                          • T
                            taryezveb
                            last edited by

                            Hmm, I edited the following to test:

                            /etc/inc/config.inc  and changed  ini_set("memory_limit","128M"); to 500M

                            /usr/local/pkg/pfblocker.inc  and added  ini_set('memory_limit', '490M');

                            Then tried to enable the TBG list, but still get the Fatal error. Did I do something wrong?

                            1 Reply Last reply Reply Quote 0
                            • marcellocM
                              marcelloc
                              last edited by

                              Not sure, I did it on a x64 virtual machine and I do not get php limit errors.

                              All files are uploaded.

                              EDIT

                              With all these large files, my virtual machine gets 100% cpu for more then a minute and even with a very large table size, it reach virtual machine memory limit (384Mb).

                              Treinamentos de Elite: http://sys-squad.com

                              Help a community developer! ;D

                              1 Reply Last reply Reply Quote 0
                              • marcellocM
                                marcelloc
                                last edited by

                                Something interesting about table-entries limit:

                                I've set up maximum table entries to a very high value 99999999999 at gui and could see it reflected on /tmp/rules.debug

                                BUT

                                on pfctl -sa I get this:
                                LIMITS:
                                states        hard limit  1000000
                                src-nodes     hard limit  1000000
                                frags         hard limit     5000
                                tables        hard limit     1000
                                table-entries hard limit 1215752191

                                I saw my virtual machine freezing console with 99999999999 value, so I reduced it on gui to 5000000

                                EDIT

                                After 2G RAM, Firewall Maximum Table Entries set to 5000000 and a reboot

                                I get no erros with both large lists applied

                                as x64 version already has ini_set("memory_limit","256M");, I've just included ini_set('memory_limit', '250M'); on line two of /usr/local/pkg/pfblocker.inc

                                Treinamentos de Elite: http://sys-squad.com

                                Help a community developer! ;D

                                1 Reply Last reply Reply Quote 0
                                • T
                                  taryezveb
                                  last edited by

                                  @marcelloc:

                                  Something interesting about table-entries limit:

                                  I've set up maximum table entries to a very high value 99999999999 at gui and could see it reflected on /tmp/rules.debug

                                  BUT

                                  on pfctl -sa I get this:
                                  LIMITS:
                                  states        hard limit  1000000
                                  src-nodes     hard limit  1000000
                                  frags         hard limit     5000
                                  tables        hard limit     1000
                                  table-entries hard limit 1215752191

                                  I saw my virtual machine freezing console with 99999999999 value, so I reduced it on gui to 5000000

                                  EDIT

                                  After 2G RAM, Firewall Maximum Table Entries set to 5000000 and a reboot

                                  I get no erros with both large lists applied

                                  as x64 version already has ini_set("memory_limit","256M");, I've just included ini_set('memory_limit', '250M'); on line two of /usr/local/pkg/pfblocker.inc

                                  Interesting, and  yes when I edited /etc/inc/config.inc I saw it already had  ini_set("memory_limit","256M"); . I have a x64 install. Nevertheless when I get a chance I will try your last settings again and reboot. Thanks for the info.

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    taryezveb
                                    last edited by

                                    @marcelloc:

                                    EDIT

                                    After 2G RAM, Firewall Maximum Table Entries set to 5000000 and a reboot

                                    I get no erros with both large lists applied

                                    as x64 version already has ini_set("memory_limit","256M");, I've just included ini_set('memory_limit', '250M'); on line two of /usr/local/pkg/pfblocker.inc

                                    Yes! :) I tested the above and no more errors, sort of; I'll explain below.

                                    I did the following:

                                    Edited /usr/local/pkg/pfblocker.inc adding the following on the first line where the comments end[from the top]. Right after this line:
                                    */                                         <- this is line number 31
                                    ini_set("memory_limit", "250M");  <- this would be line number 32

                                    Then set the Firewall Maximum Table Entries: 5000000 and did a reboot. But received these errors:
                                    Dec 31 18:13:44 php: : There was an error while parsing the package filter rules for /usr/local/pkg/squid.inc.

                                    Dec 31 18:13:44 php: : The command '/sbin/pfctl -nf /tmp/rules.test.packages' returned exit code '1', the output was 'no IP address found for 550.164.3.166/32 /tmp/rules.test.packages:17: file "/var/db/aliastables/pfBlockerBluetack.txt" contains bad data'

                                    After some testing those errors are due to using the following list:
                                    http://list.iblocklist.com/?list=bt_proxy&fileformat=p2p&archiveformat=gz

                                    NOTE: I tried several times removing and adding that list, with the same result.

                                    Once I removed that list I got this:
                                    Dec 31 18:46:28 php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'no IP address found for 75.35.185. /tmp/rules.debug:19: file "/var/db/aliastables/pfBlockerTBG.txt" contains bad data no IP address found for ÿ0.0.0.0/8 /tmp/rules.debug:95: file "/etc/bogons" contains bad data pfctl: Syntax error in config file: pf rules not loaded'

                                    More testing and the error above was due to this list:
                                    http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz

                                    But this time after removing the list above and re-adding the error went away. I guess there are some issues with those lists. Will try to contact the author to see if that can be fixed.

                                    So in the end, no more errors and Squid works as a transparent proxy using pfBlocker as well now :)

                                    For reference these are the lists that I'm currently using error free:

                                    <pfblockerlists><config><aliasname>Bluetack</aliasname>

                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_level2&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_spyware&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_templist&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_hijacked&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_dshield&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=bt_bogon&fileformat=p2p&archiveformat=gz</url></row>
                                    <action>Deny_Inbound</action>
                                    <cron>EveryDay</cron></config>
                                    <config><aliasname>TBG</aliasname>

                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=ijfqtofzixtwayqovmxn&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=ecqbsykllnadihkdirsh&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=tbnuqfclfkemqivekikv&fileformat=p2p&archiveformat=gz</url></row>
                                    <row><format>gz</format>
                                    <url>http://list.iblocklist.com/?list=ewqglwibdgjttwttrinl&fileformat=p2p&archiveformat=gz</url></row>
                                    <action>Deny_Inbound</action>
                                    <cron>EveryDay</cron></config></pfblockerlists>

                                    I would like to Thank marcelloc for his time looking into this. A donation was made[yesterday, before marcelloc posted the fix]; will donate again when I can. Hope this change is added to the package for all to benefit, after a few more days of testing. To make sure this change does not affect anything else.

                                    EDIT: Happy New Year :)

                                    1 Reply Last reply Reply Quote 0
                                    • marcellocM
                                      marcelloc
                                      last edited by

                                      Thanks  ;D. Did you mentioned these packages developers on donation?

                                      Treinamentos de Elite: http://sys-squad.com

                                      Help a community developer! ;D

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        taryezveb
                                        last edited by

                                        @marcelloc:

                                        Did you mentioned these packages developers on donation?

                                        Yes, I broke it down on how the donation should broken up. Like so: $XX for the X package maintainer(s).

                                        1 Reply Last reply Reply Quote 0
                                        • marcellocM
                                          marcelloc
                                          last edited by

                                          I'll include this ini_set('memory_limit', '250M') patch when pfblocker detects a x64 version.

                                          There are other places to edit it, maybe a redmine for that as diagnostics-> tables also has a 128Mb limit even on x64.

                                          Treinamentos de Elite: http://sys-squad.com

                                          Help a community developer! ;D

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            taryezveb
                                            last edited by

                                            @marcelloc:

                                            I'll include this ini_set('memory_limit', '250M') patch when pfblocker detects a x64 version.

                                            There are other places to edit it, maybe a redmine for that as diagnostics-> tables also has a 128Mb limit even on x64.

                                            Great and yes good point, since Diagnostics -> Tables gives a similar: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /usr/local/www/diag_tables.php on line 89

                                            But at least that does not affect things like the previous "Fatal error: Allowed memory size…" did.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.