[SOLVED] 2.4.3 - /rc.filter_configure_sync: cannot define table bogonsv6

cybrnook · Apr 6, 2018, 7:05 PM

Clean install of 2.4.3 onto a custom build Intel 1620 v3 (4 core 8 thread) system we 32GB memory. Disk is a raptor 150GB.

Receiving this quite often in the system long ->

php-fpm 326 /rc.filter_configure_sync: New alert found: There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [18]: table <bogonsv6>persist file "/etc/bogonsv6"

Is there a ulimit issue or something?

EDIT:

Solution is to increase Max firewall table entries to 400,000 from the default 200,000 and then reload filter. Will be fixed in the next release https://redmine.pfsense.org/issues/8417.

System > Advanced > Firewall & NAT > Firewall Maximum Table Entries > 400000
Status > Filter Reload > Reload Filter

</bogonsv6>

Derelict · Mar 31, 2018, 9:30 PM

That's a big list. You are running out of memory processing it.

cybrnook · Mar 31, 2018, 9:50 PM

But I haven't done any special configuration. This occurred even on first boot after a clean install with the default out of the box pfsense configuration on a system that's more than adequate.

I assume then that this is an issue then with a default pfsense memory space setting. Any recommendations things to look at/try?

cybrnook · Apr 1, 2018, 11:19 AM

A post in German which gets me closer.

https://www.taste-of-it.de/pfsense-fehler-table-bogonsv6-allocate-memory/

But it's not clicking to me yet why it's happening out of the box.

revengineer · Apr 1, 2018, 11:50 AM

The pfsense book notes that the bogon list for IPv6 is large and that default number of table entries may need to be increased. The instructions for that were given in that German post you quoted. In my case that file is actually empty (no Ipv6 bogons). This may be because all my local networks are configured for IPv4 only. So, pfsense does require some configuration out of the box, which is why there is a 600+ page book on it. For your benefit, you may consider a Gold subscription to get access to the book.

cybrnook · Apr 1, 2018, 12:23 PM

@revengineer:

The pfsense book notes that the bogon list for IPv6 is large and that default number of table entries may need to be increased. The instructions for that were given in that German post you quoted. In my case that file is actually empty (no Ipv6 bogons). This may be because all my local networks are configured for IPv4 only. So, pfsense does require some configuration out of the box, which is why there is a 600+ page book on it. For your benefit, you may consider a Gold subscription to get access to the book.

Thanks for the response. I apologize if I wasn't more clear, it was also understood to me where to adjust the default list size from the post above.

My question is more around "why" this is happening on a fresh install of 2.4.3. This was not the noted behavior of 2.4.2.

I have also disabled ipv6 on WAN as well as in DHCPv6 server. So you see, I also do not use ipv6. My ISP does not offer IPV6.

For the record, I do have a gold subscription. But my question again if not just wanting to know to increase the default value, I want to know "why" I see this error out of the box.

revengineer · Apr 1, 2018, 12:41 PM

@cybrnook:

My question is more around "why" this is happening on a fresh install of 2.4.3. This was not the noted behavior of 2.4.2.

ah ok, the fact that it worked out of the box in 2.4.2 but not in 2.4.3 was not clear to me. In this case this is a question for the devs, I can't help here.

bingo600 · Apr 2, 2018, 7:28 AM

I just got the same error today on 2 independent 2.4.2-p1 boxes


There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [35]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-01 14:49:30

It has to be the bogon file that has increased in size , as i haven't touched one of the boxes in 2 weeks

I have tried to increase from 200K to 500K entries (both are 8G Ram machines)

Lets see if i keep on getting the errors

~~Weird … still get the errors w 500K entries~~

Nope 500K entries solves it

In "System -> Advanced -> Firewall & NAT" I increased the below from 200K to 500K

MIght be a good idea to do a reboot after the increase.

/Bingo

bingo600 · Apr 1, 2018, 3:34 PM

I was experiencing strange errors w. my rules.

Could the bogon error cause , that aliases can't be allocated correctly ?
I has something that reminded of "Non working" rules , that looked ok.

That's en evil DOS for a pfsense box - If increasing bogons beyond "A lot" would FSCK up the aliases etc.

Is there anywhere , where one could stop the bogon files from being loaded/updated ?

/Bingo

rkillcrazy · Apr 2, 2018, 3:08 PM

@bingo600:

I just got the same error today on 2 independent 2.4.2-p1 boxes
There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [35]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-01 14:49:30
It has to be the bogon file that has increased in size , as i haven't touched one of the boxes in 2 weeks

I have tried to increase from 200K to 500K entries (both are 8G Ram machines)

Lets see if i keep on getting the errors

~~Weird … still get the errors w 500K entries~~

Nope 500K entries solves it

In "System -> Advanced -> Firewall & NAT" I increased the below from 200K to 500K

MIght be a good idea to do a reboot after the increase.

/Bingo

Just to help other people, you cannot see the full post, with the image showing the needed details, unless you're logged into the forum with a member's creds so here's where you change it if you happened upon this thread via a search engine like I did: System > Advanced > Firewall & NAT > Firewall Maximum Table Entries

This bit of info helped with my issue as well. I don't have but a handful of firewall rules so I didn't think it was anything I did. Furthermore, I hadn't seen the errors until after the upgrade to 2.4.3. At first, I was seeing the errors nagging about the couple rules I've created but, after a reboot, it was throwing the errors about the bogon network rule in the table and I know that's a factory rule; not one I customize. So, watch your red herrings…

I think when the developers upped the size of the bogon networks file, they should have at least recommended the changing of aforementioned setting. I don't think changing it for the end user is a good idea as you never know what hardware the OS is running on but a warning would have been nice; that's just my two cents.

A Former User · Apr 2, 2018, 7:01 PM

Is there a way to see how many of these Firewall Entries we are using, even if we're not getting the error?
If I'm flying close to the sun I'd like to fix it before it's a problem.

Thanks

Derelict · Apr 2, 2018, 7:15 PM

pfctl -vvsT | grep Addresses will get you close.

Note that reloading the tables requires double the space so if that total is getting close to half the defined table maximum you will want to increase it.

JohnnyBeGood · Apr 2, 2018, 11:40 PM

Thanks guys for posting the solution!
In my case I just did upgrade and since I have email notificatons enabled got below email:


Notifications in this message: 1
================================

15:15:49 There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6>persist file "/etc/bogonsv6"</bogonsv6>

Increased to 500k and I'm hoping I won't be getting more errors although after getting it for the first time I rebooted and did not got that error.

cybrnook · Apr 3, 2018, 2:31 AM

I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.

rkillcrazy · Apr 3, 2018, 11:38 AM

@cybrnook:

I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.

Care to share with others the steps you took to accomplish this?

cybrnook · Apr 3, 2018, 12:41 PM

@rkillcrazy:

@cybrnook:

I did a filter reload as well after changing to 400k. Just to make sure all rules were loaded properly.

Care to share with others the steps you took to accomplish this?

Sure

System > Advanced > Firewall & NAT > Firewall Maximum Table Entries > 400000
Status > Filter Reload > Reload Filter

corvey · Apr 3, 2018, 6:03 PM

Fresh install of pfSense 2.4.3 introduces these errors while configuring my interfaces for the first time:

–---------------------------------------------------------------------------------
There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [18]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:46:06
There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:47:12
There were error(s) loading the rules: /tmp/rules.debug:20: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [20]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:48:04
There were error(s) loading the rules: /tmp/rules.debug:20: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [20]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:49:17
There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:49:36
There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:49:37
There were error(s) loading the rules: /tmp/rules.debug:21: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [21]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:49:40
There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:50:22
There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:50:23
There were error(s) loading the rules: /tmp/rules.debug:22: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [22]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:50:26
There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:50:55
There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:50:56
There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [23]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:51:00
There were error(s) loading the rules: /tmp/rules.debug:24: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [24]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:51:37
There were error(s) loading the rules: /tmp/rules.debug:24: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [24]: table <bogonsv6> persist file "/etc/bogonsv6"
@ 2018-04-03 13:51:38
–----------------------------------------------------------------------------------

I'm guessing the final solution in this thread is to increase the Firewall Maximum Table Entries to 800000 will be enough to cure the problem problem for awhile without any ill effects. This is a new error in this release I haven't encountered before.

cybrnook · Apr 3, 2018, 6:21 PM

400000 is more than enough for today. As we are right now, the bogon file and our firewall add up to about 95000~ entries. When it reloads this table, it doubles in size before the old entries are dropped. Many of us just break that 200000 limit that is default today 95000*2 + whatever else, pushing us over the 200k limit….

The new default will be 400000 in the next release. I am using that value, and it works fine, giving you about an additional 100000 buffer (since it's X * 2 = Y, 200000 Bogon list for example would burp to 400000 on reload. but it's only at about 95000 now all-in, so you have 100000 to go, which is a lot).

People are also seeing it in 2.4.2+ as well, so not just a 2.4.3 thing. But pops up soon after the install/upgrade, triggering the error message.

humps · Apr 3, 2018, 7:12 PM

@cybrnook:

People are also seeing it in 2.4.2+ as well, so not just a 2.4.3 thing. But pops up soon after the install/upgrade, triggering the error message.

Yes Confirmed.
I'm Running Pfsense 2.4.2 (X64) and these errors showed up in my logs yesterday (April 2, 2018).
I haven't seen none since today but i have upend my entries to 500000 to prevent a re occurrence.

Regards

cybrnook · Apr 3, 2018, 7:28 PM

I assume what's happening is the monthly cron job that downloads and installs the latest bogons file is slowly working it's way through the community. So regardless of version, people will start popping in here one by one with the issue over the next ~25 - 30 days.

I assume what's likely happening on new installs (and likely upgrades), is that part of the post processing setup is to download the latest file, then schedule it to download again in 30 days time. That's why I think I am seeing it on fresh installs, right after initial setup wizard.