[SOLVED] 2.4.3 - /rc.filter_configure_sync: cannot define table bogonsv6



  • Clean install of 2.4.3 onto a custom build Intel 1620 v3 (4 core 8 thread) system we 32GB memory. Disk is a raptor 150GB.

    Receiving this quite often in the system long ->

    php-fpm 326 /rc.filter_configure_sync: New alert found: There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [18]: table <bogonsv6>persist file "/etc/bogonsv6"

    Is there a ulimit issue or something?

    EDIT:

    Solution is to increase Max firewall table entries to 400,000 from the default 200,000 and then reload filter. Will be fixed in the next release https://redmine.pfsense.org/issues/8417.

    System > Advanced > Firewall & NAT > Firewall Maximum Table Entries > 400000
    Status > Filter Reload > Reload Filter


    </bogonsv6>


  • Netgate

    That's a big list. You are running out of memory processing it.



  • But I haven't done any special configuration. This occurred even on first boot after a clean install with the default out of the box pfsense configuration on a system that's more than adequate.

    I assume then that this is an issue then with a default pfsense memory space setting. Any recommendations things to look at/try?



  • A post in German which gets me closer.

    https://www.taste-of-it.de/pfsense-fehler-table-bogonsv6-allocate-memory/

    But it's not clicking to me yet why it's happening out of the box.



  • The pfsense book notes that the bogon list for IPv6 is large and that default number of table entries may need to be increased. The instructions for that were given in that German post you quoted. In my case that file is actually empty (no Ipv6 bogons). This may be because all my local networks are configured for IPv4 only. So, pfsense does require some configuration out of the box, which is why there is a 600+ page book on it. For your benefit, you may consider a Gold subscription to get access to the book.



  • @revengineer:

    The pfsense book notes that the bogon list for IPv6 is large and that default number of table entries may need to be increased. The instructions for that were given in that German post you quoted. In my case that file is actually empty (no Ipv6 bogons). This may be because all my local networks are configured for IPv4 only. So, pfsense does require some configuration out of the box, which is why there is a 600+ page book on it. For your benefit, you may consider a Gold subscription to get access to the book.

    Thanks for the response. I apologize if I wasn't more clear, it was also understood to me where to adjust the default list size from the post above.

    My question is more around "why" this is happening on a fresh install of 2.4.3. This was not the noted behavior of 2.4.2.

    I have also disabled ipv6 on WAN as well as in DHCPv6 server. So you see, I also do not use ipv6. My ISP does not offer IPV6.

    For the record, I do have a gold subscription. But my question again if not just wanting to know to increase the default value, I want to know "why" I see this error out of the box.



  • @cybrnook:

    My question is more around "why" this is happening on a fresh install of 2.4.3. This was not the noted behavior of 2.4.2.

    ah ok, the fact that it worked out of the box in 2.4.2 but not in 2.4.3 was not clear to me. In this case this is a question for the devs, I can't help here.



  • I just got the same error today on 2 independent  2.4.2-p1 boxes

    
    There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [35]: table <bogonsv6> persist file "/etc/bogonsv6"
    @ 2018-04-01 14:49:30
    
    

    It has to be the bogon file that has increased in size , as i haven't touched one of the boxes in 2 weeks

    I have tried to increase from 200K to 500K entries (both are 8G Ram machines)

    Lets see if i keep on getting the errors

    Weird … still get the errors w 500K entries

    Nope 500K entries solves it

    In  "System -> Advanced -> Firewall & NAT" I increased the below from 200K to 500K

    MIght be a good idea to do a reboot after the increase.

    /Bingo




  • I was  experiencing strange errors w. my rules.

    Could the bogon error cause , that aliases can't be allocated correctly ?
    I has something that reminded of "Non working" rules , that looked ok.

    That's en evil DOS for a pfsense box - If increasing bogons beyond "A lot" would FSCK up the aliases etc.

    Is there anywhere , where one could stop the bogon files from being loaded/updated ?

    /Bingo



  • @bingo600:

    I just got the same error today on 2 independent  2.4.2-p1 boxes

    
    There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [35]: table <bogonsv6> persist file "/etc/bogonsv6"
    @ 2018-04-01 14:49:30
    
    

    It has to be the bogon file that has increased in size , as i haven't touched one of the boxes in 2 weeks

    I have tried to increase from 200K to 500K entries (both are 8G Ram machines)

    Lets see if i keep on getting the errors

    Weird … still get the errors w 500K entries

    Nope 500K entries solves it

    In  "System -> Advanced -> Firewall & NAT" I increased the below from 200K to 500K

    MIght be a good idea to do a reboot after the increase.

    /Bingo

    Just to help other people, you cannot see the full post, with the image showing the needed details, unless you're logged into the forum with a member's creds so here's where you change it if you happened upon this thread via a search engine like I did:  System > Advanced > Firewall & NAT > Firewall Maximum Table Entries

    This bit of info helped with my issue as well.  I don't have but a handful of firewall rules so I didn't think it was anything I did.  Furthermore, I hadn't seen the errors until after the upgrade to 2.4.3.  At first, I was seeing the errors nagging about the couple rules I've created but, after a reboot, it was throwing the errors about the bogon network rule in the table and I know that's a factory rule; not one I customize.  So, watch your red herrings…

    I think when the developers upped the size of the bogon networks file, they should have at least recommended the changing of aforementioned setting.  I don't think changing it for the end user is a good idea as you never know what hardware the OS is running on but a warning would have been nice; that's just my two cents.



  • Is there a way to see how many of these Firewall Entries we are using, even if we're not getting the error?
    If I'm flying close to the sun I'd like to fix it before it's a problem.

    Thanks


  • Netgate

    pfctl -vvsT | grep Addresses will get you close.

    Note that reloading the tables requires double the space so if that total is getting close to half the defined table maximum you will want to increase it.



  • Thanks guys for posting the solution!
    In my case I just did upgrade and since I have email notificatons enabled got below email:

    
    Notifications in this message: 1
    ================================
    
    15:15:49 There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6>persist file "/etc/bogonsv6"</bogonsv6> 
    

    Increased to 500k and I'm hoping I won't be getting more errors although after getting it for the first time I rebooted and did not got that error.



  • I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.



  • @cybrnook:

    I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.

    Care to share with others the steps you took to accomplish this?



  • @rkillcrazy:

    @cybrnook:

    I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.

    Care to share with others the steps you took to accomplish this?

    Sure

    System > Advanced > Firewall & NAT > Firewall Maximum Table Entries > 400000
    Status > Filter Reload > Reload Filter



  • Fresh install of pfSense 2.4.3  introduces these errors while configuring my interfaces for the first time:

    –---------------------------------------------------------------------------------
        There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [18]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:46:06
        There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:47:12
        There were error(s) loading the rules: /tmp/rules.debug:20: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [20]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:48:04
        There were error(s) loading the rules: /tmp/rules.debug:20: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [20]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:49:17
        There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:49:36
        There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:49:37
        There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:49:40
        There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:50:22
        There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:50:23
        There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:50:26
        There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:50:55
        There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:50:56
        There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:51:00
        There were error(s) loading the rules: /tmp/rules.debug:24: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [24]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:51:37
        There were error(s) loading the rules: /tmp/rules.debug:24: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [24]: table <bogonsv6> persist file "/etc/bogonsv6"
        @ 2018-04-03 13:51:38
    –----------------------------------------------------------------------------------

    I'm guessing the final solution in this thread is to increase the Firewall Maximum Table Entries to 800000 will be enough to cure the problem problem for awhile without any ill effects.  This is a new error in this release I haven't encountered before.



  • 400000 is more than enough for today. As we are right now, the bogon file and our firewall add up to about 95000~ entries. When it reloads this table, it doubles in size before the old entries are dropped. Many of us just break that 200000 limit that is default today 95000*2 + whatever else, pushing us over the 200k limit….

    The new default will be 400000 in the next release. I am using that value, and it works fine, giving you about an additional 100000 buffer (since it's X * 2 = Y, 200000 Bogon list for example would burp to 400000 on reload. but it's only at about 95000 now all-in, so you have 100000 to go, which is a lot).

    People are also seeing it in 2.4.2+ as well, so not just a 2.4.3 thing. But pops up soon after the install/upgrade, triggering the error message.



  • @cybrnook:

    People are also seeing it in 2.4.2+ as well, so not just a 2.4.3 thing. But pops up soon after the install/upgrade, triggering the error message.

    Yes Confirmed.
    I'm Running Pfsense 2.4.2 (X64) and these errors showed up in my logs yesterday (April 2, 2018).
    I haven't seen none since today but i have upend my entries to 500000 to prevent a re occurrence.

    Regards



  • I assume what's happening is the monthly cron job that downloads and installs the latest bogons file is slowly working it's way through the community. So regardless of version, people will start popping in here one by one with the issue over the next ~25 - 30 days.

    I assume what's likely happening on new installs (and likely upgrades), is that part of the post processing setup is to download the latest file, then schedule it to download again in 30 days time. That's why I think I am seeing it on fresh installs, right after initial setup wizard.



  • April 3 2018 bogons:  100,001 rows

    ipv6 bogon prefixes 95,997
    ipv4 bogon prefixes 4004

    bogon table size: 100,001 rows

    If 2x are required to reload the table, then 200K seems slightly too small  ;)

    I now notice that my 2.4.2 systems are choking the same way as my freshly updated 2.4.3 systems if I both update bogons and reload the filter.



  • New to pfSense and after installing 2.4.3 on a new install this popped up in my alerts.
    Unfortunately until I resolved this issue none of my port forwards would work at all.

    I bumped up the number of entries to 500,000 and my port forwards started working immediately.

    This is just an FYI in case you can't get your port forwards to work to anyone else who is very new to pfSense.



  • The thing I don't understand is why it was 200K for you by default in the first place. I'm still on 2.3.4 and my default for "Firewall Maximum Table Entries" is 2M. (2 000 000). Did they reduce the default on purpose ?



  • Without going through the git revisions, I would say yes that somewhere between your release and the 2.4 releases it got changed to two hundred thousand 200,000. If you look at the link in the op for the bug ticket to get it fixed, the wording reads of changing the old default value of 200k to 400k. Letting you know that yes, 200k was the current default.

    What I am also thinking is that maybe some add on packages, like pfblocker or snort etc., change this default value to a higher number based off the nature of what new rules they will likely need. This is purely speculative.



  • @Bili_boy:

    The thing I don't understand is why it was 200K for you by default in the first place. I'm still on 2.3.4 and my default for "Firewall Maximum Table Entries" is 2M. (2 000 000). Did they reduce the default on purpose ?

    I've wondered the same thing.  I'm on 2.4.3 and the default for "Firewall Maximum Table Entries" is 2M (2,000,000) on my system.  I upgraded from 2.4.2 p1 but I don't remember what it was then and I don't remember ever changing it.  Not sure where all these people are getting that their system has 200K as default.



  • @jdeloach:

    @Bili_boy:

    The thing I don't understand is why it was 200K for you by default in the first place. I'm still on 2.3.4 and my default for "Firewall Maximum Table Entries" is 2M. (2 000 000). Did they reduce the default on purpose ?

    I've wondered the same thing.  I'm on 2.4.3 and the default for "Firewall Maximum Table Entries" is 2M (2,000,000) on my system.  I upgraded from 2.4.2 p1 but I don't remember what it was then and I don't remember ever changing it.  Not sure where all these people are getting that their system has 200K as default.

    what additional packages do you have installed?



  • @cybrnook:

    @jdeloach:

    @Bili_boy:

    The thing I don't understand is why it was 200K for you by default in the first place. I'm still on 2.3.4 and my default for "Firewall Maximum Table Entries" is 2M. (2 000 000). Did they reduce the default on purpose ?

    I've wondered the same thing.  I'm on 2.4.3 and the default for "Firewall Maximum Table Entries" is 2M (2,000,000) on my system.  I upgraded from 2.4.2 p1 but I don't remember what it was then and I don't remember ever changing it.  Not sure where all these people are getting that their system has 200K as default.

    what additional packages do you have installed?

    I only have the APC UPS Daemon package installed.  Everything else is just the default install.  I don't have any of the other packages like PfBlockerNG, Squid or Squidguard installed.



  • @jdeloach:

    @cybrnook:

    @jdeloach:

    @Bili_boy:

    The thing I don't understand is why it was 200K for you by default in the first place. I'm still on 2.3.4 and my default for "Firewall Maximum Table Entries" is 2M. (2 000 000). Did they reduce the default on purpose ?

    I've wondered the same thing.  I'm on 2.4.3 and the default for "Firewall Maximum Table Entries" is 2M (2,000,000) on my system.  I upgraded from 2.4.2 p1 but I don't remember what it was then and I don't remember ever changing it.  Not sure where all these people are getting that their system has 200K as default.

    what additional packages do you have installed?

    I only have the APC UPS Daemon package installed.  Everything else is just the default install.  I don't have any of the other packages like PfBlockerNG, Squid or Squidguard installed.

    interesting. My install was just a vanilla 2.4.3. as soon as the config wizard was done, the error was already there.



  • Am I losing my mind?

    I just updated another 2.4.2 system to 2.4.3, but noticed the new default is 400,000 entries whereas this thread started because the default was 200,000 entries just yesterday.  "Ah, they did a minor point-release and updated the default" I reasoned.

    However, when I went to the machines that I'd manually overridden from 200,000 to 400,000 I noticed that their defaults had also changed, even though they have not been updated (i.e. via point-release).  Huh?  Aren't the defaults hard-coded into the release?

    What I've seen here would be more consistent with the defaults being periodically fetched from somewhere.    Is that true?


  • Netgate

    I haven't looked at the code but I think there is a logic problem in that "the system default is X." I think it just says whatever the field is set to instead of actually computing what the default would actually be.

    For instance, I didn't see this overrun on bogonsv6 because mine was set to 2,000,000 by something/someone/probablyme. It said "the default on this system is 2,000,000"



  • LOL.

    By george you're right.  Looks like on my system, the "system default" looks a an awful lot like Pi, but I've overridden it to 400,000  ;)




  • As a beginner, thank you very much for your explanation!
    I have set entries to 500K




  • I'd like to thank you all as well for explaining this! Very helpful in resolving this issue.



  • I guess the big question here is why?

    Why do we need to increase the Firewall Maximum Table Entries from 200k (default) to 500k all of a sudden? I have been running pfSense a long time and have never had to make this change. So what changed all of a sudden?

    It's great there is a solution but there isn't any real explanation as to why we have to change this value?

    Thanks,


  • Netgate

    The size of the IPv6 bogons table in the April update changed and pushed some systems over the edge.

    The default has been changed to 400000 in 2.4.4

    The timing of the bogons table monthly update and the release of 2.4.3 was simply coincidental.



  • I'm confirming that after I upgraded from 2.4.2_1 to 2.4.3, I had the same type of error:

    Filter Reload
    There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6> persist file "/etc/bogonsv6"
    @ 2018-04-09 09:15:46

    Changing the Firewall Maximum Table Entries from 200000 to 500000 and rebooting solved the problem.



  • Tricky situation

    • if increase the maximum entries size from 200k to 400k, then rules modification and filters reload work without need of reboot

    • BUT, then i lose all my bandwidth, coming from 140Mb/s to 1Mb/s

    • if i use back 200k instead of 400k, then i have the bug back, but my bandwidth is back to 140mb/s !!!

    What the hell is that issue ???



  • I have the same problem, even increased to 4,000,000, it does not make the error go away.
    Tried several values, increasing from default (listed as 200,000) to current 4,000,000 with reboots.

    Only solution i have for now is to "uncheck" the allow IPv6 Traffic in the System / Advanced / Networking section.
    No more errors.
    So i guess the bogonsv6 data is not loaded now?

    Have run PFSense for years without problems, 4 physical interfaces configured with about 10 VLANS, reasonable amount of rules, aliases etc.
    Should i continue to increase the number and try?  4,000,000 already seems excessive.


  • Netgate

    Your experience does not mirror countless others.

    Are you sure you are changing maximum table entries and not maximum states? They are completely different things.



  • yes, i'm sure, did not touch the default for Max Firewall States.

    ![Screen Shot 2018-04-10 at 02.32.01.png](/public/imported_attachments/1/Screen Shot 2018-04-10 at 02.32.01.png)
    ![Screen Shot 2018-04-10 at 02.32.01.png_thumb](/public/imported_attachments/1/Screen Shot 2018-04-10 at 02.32.01.png_thumb)


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy