state counters, firewall rules resetting?

johnpoz

Did something change in 23.01?

This doesn't seem right.. Before I would see GBs listed..

I will have to look into this more tonight after work.. But 23.01 has been up almost 17 days... These counters are for sure not correct..

My Lan rule should be in the GBs for sure - Just last night I download multiple gigs of linux distros ;) from my server on the internet.

I know for sure I watched some stuff off one of my roku's to my plex server..

Are they now resetting when something happens? I haven't made any firewall rule changes, latest change I show is this

02. 3/6/23 09:48:16     v22.8   admin@192.168.9.100 (Local Database)
    Firewall: NAT: Outbound - reordered outbound mappings.

If traffic flows through an existing state are they not updating?

edit: ok I just kicked off and completed a 2GB download, and that seems to be showing correctly.

I will check it later when I get home from work, etc.. hmmm what would be resetting the counters?

stephenw10

Mmm, they do seem low here on my edge box.

Doesn't seem to be resetting at filter reload...

nimrod

@stephenw10 said in state counters, firewall rules resetting?:

Mmm, they do seem low here on my edge box.

Doesn't seem to be resetting at filter reload...

I talked about this exact issue in the thread about pfBlockerNG because it affects IP counter as well but it was ignored.

To "fix" this issue just revert this patch in System Patches package:

Redmine link is here.

Once this change is reverted, no reboot is required and firewall rules will show correct number of states and A LOT more data in and out.

johnpoz

@nimrod I'm on 23.01 and I have no patch like that installed. Nor would I be able to revert it, etc.

That seems to be related to HA, which not running either.

nimrod

@johnpoz

This is tested on v2.6 CE which doesnt have this patch applied by default. Im not sure about + version.

As soon as i apply this patch via system patches firewall rules start behaving the same way as you described.

If @stephenw10 has a test environment with v2.6 CE he can confirm this.

Steps to reproduce this are simple.

Install pfSense v2.6 CE.
Install System Patches package.
Apply patch mentioned above.
Observe firewall rules.
Revert the patch.
Observe the rules again.

stephenw10

Yeah that doesn't apply in 23.01. set keepcounters is present in the pf ruleset file by default.

nimrod

@stephenw10 said in state counters, firewall rules resetting?:

Yeah that doesn't apply in 23.01. set keepcounters is present in the pf ruleset file by default.

Just to make sure we are on the same page here. If i apply that patch on CE version does that mean set keepcounters is getting set or removed?

johnpoz

@stephenw10 well they are resetting that is for sure

And I watched some roku, and it did show traffic and now this morning its back to 0/0

Hmmm?

Gertjan

@johnpoz

I'm following these counters since yesterday.

Curious to know when counters get reset.
Something like : you modified a rule, something you obviously didn't do.
Have you rules that are scheduled ?
Counter reset means to me : "new" rules so also states are reset ( ? )

johnpoz

@gertjan yeah no changes to rules since yesterday that is for sure.. And I don't have any rules on a schedule..

The only thing that is scheduled is my aliases in pfblocker - those update, but I don't have any sort of auto rules even enabled in pfblocker, only the native aliases..

If I look in changes

03. 3/6/23 09:48:16     v22.8   admin@192.168.9.100 (Local Database)
    Firewall: NAT: Outbound - reordered outbound mappings.

02. 3/6/23 15:00:16     v22.8   (system)
    pfBlockerNG: saving DNSBL changes

01. 3/7/23 15:00:16     v22.8   (system)
    pfBlockerNG: saving DNSBL changes

That is an odd comment on the pfblocker change - since I don't even have DNSBL enabled. But do have it set to update my aliases every 6 hours..

But clearly that is on a schedule..

If I look at my crons - I would assume that listing in the config changes is the bottom one

But why should it reset the counters?

edit: well I just manually kicked off a pfblocker update and it shows in the config log

01. 3/9/23 04:59:54     v22.8   (system)
    pfBlockerNG: saving DNSBL changes

But my counters didn't reset, sill show the same amount of traffic on my lan rule, as I did before running it. So its prob not that doing it... hmmm?

stephenw10

@johnpoz said in state counters, firewall rules resetting?:

@stephenw10 well they are resetting that is for sure

Indeed, they sure seem to be.... I can't trigger it to reset either. Yet.

Jarhead

@stephenw10 Just to throw my hat in, checked yesterday and had over a gig on the LAN, today 800M.

bigsy

@johnpoz Do you have the patch for redmine #14016 applied?

I had a problem on 23.01 with pfBlocker IP counters resetting overnight until I applied this. I didn't notice the firewall counters. Something in the default cron jobs must have been resetting it?

jdeloach

This post is deleted!

tman222

@bigsy said in state counters, firewall rules resetting?:

@johnpoz Do you have the patch for redmine #14016 applied?

I had a problem on 23.01 with pfBlocker IP counters resetting overnight until I applied this. I didn't notice the firewall counters. Something in the default cron jobs must have been resetting it?

Thanks @bigsy for calling out this patch. I have been running into the same issue (pfBlockerNG IP counters resetting overnight) and then noticed similar to @johnpoz that my traffic counters didn't make sense. Applied the patch and will monitor to see if that has hopefully fixed it.

Relevant thread:
https://forum.netgate.com/topic/178107/23-01-periodic-scripts-have-been-re-enabled-and-are-broken/5

johnpoz

@bigsy I think you might of found it, I do have 2 patches installed for state issues

https://github.com/pfsense/pfsense/commit/d9fa4584e3fb63d6051e9f1db7655f931cb1be19.patch

and one I manually applied

diff --git a/blah/usr/local/www/rrd_fetch_json.php b/blah/usr/local/www/rrd_fetch_json.php
index df0401f96e89..4d7574819ac9 100644
--- a/blah/usr/local/www/rrd_fetch_json.php
+++ b/blah/usr/local/www/rrd_fetch_json.php
@@ -226,7 +226,8 @@ foreach ($side as $settings) {
                                $ds = "state changes";
                                break;
                        case "pfnat":
-                               $ignore = true;
+                               $unit_acronym = "";
+                               $ds = "NAT states";
                                break;
                        case "inpass":
                                $ninetyfifth = true;

Looking into the details now.

edit: ok applied ff715efce5e6c65b3d49dc2da7e1bdc437ecbf12 lets see if resets tonight or next couple of days.

stephenw10

Same here. Let's see....

Gertjan

@johnpoz said in state counters, firewall rules resetting?:

edit: ok applied ff715efce5e6c65b3d49dc2da7e1bdc437ecbf12

I had that one already active.
My counter are still good / plausible.

johnpoz

@gertjan nope reset...

WTF???

bigsy

@johnpoz Did you reboot or trigger a rewrite of the cron config after applying the patch per this link?