Rules order randomly changes

SteveITS

@w0w And this was on 25.03?

Recently I found one of our routers had several rules moved from near the top to the bottom of the list…still in order but last. I was upgrading from 24.03 to 24.11 and I’m not sure if it happened before or after.

Two others the separator bar had moved slightly but IIRC that was a bug a while back for that if adding or maybe copying rules, so I didn’t think much of that since the order was the same.

w0w

@SteveITS said in Rules order randomly changes:

And this was on 25.03?

Yep.

louis2

@SteveITS

Not the same but perhaps related, rule numbers are not a fixed thing

And separator lines do not stay on their place. That is not new I did discover that I think two years ago.

One of the reasons I was looking into that is that I am exporting the rule list in favor of a GrayLog lookup table. I Graylog I like to see the rule description

I am still exporting that rule list now and than, but the rule numbers are unfortunately not completely stable.

One of the ways I did test this, in favor of the list but also in favor of rule groups, is that I did add a recognizable dummy rule at the to of each interface and interface group and on at the bottum. Those dummy rules (doing nothing) had an interface ID and a begin / start identifier hidden in the rule port number like "port is <90001> at the beginning of interface 9 and <90002> at the end of the rules related to inteface 9.

Perhaps this helps to track problems

w0w

@louis2
What packages you have installed? PfBlockerNG?

SteveITS

@w0w My recent/only experience was on 24.11. We do have pfBlocker but have for years. I tried to compare the rule order of different backup config files but they seem basically the same. So it may have just happened at the time which might imply a rare issue/race condition/upgrade issue. Haven't noticed it on any other routers and I've upgraded 6 or so to 24.11 recently.

w0w

After further testing, another bug was found with sorting: when a user changes the order and deletes a rule, the next rule added to the list appears in the same position where the last deleted rule was placed. Corrected code below.

function filter_rules_sort() {
    $rules = config_get_path('filter/rule', []);

    /* Step 1: Find the highest existing `seq` value */
    $max_seq = 0;
    foreach ($rules as $rule) {
        if (isset($rule['seq']) && is_numeric($rule['seq'])) {
            $max_seq = max($max_seq, $rule['seq']);
        }
    }

    /* Step 2: Ensure rules are stored in the order they appear in the GUI */
    $new_seq = 0;
    foreach ($rules as &$rule) {
        $rule['seq'] = ++$new_seq; // Assign sequential order based on GUI order
    }

    /* Step 3: Sort rules strictly by `seq` */
    usort($rules, "filter_rules_compare");

    /* ✅ Save the correct order in `config.xml` */
    config_set_path('filter/rule', $rules);
    write_config("Updated firewall rule order based on user-defined sorting.");
}

w0w

@SteveITS
I rarely change anything in the rule settings, but almost every time I do—whether it's modifying, deleting, or moving something—this mess with movements happens often.

louis2

@w0w

I noticed the described behavoir on a system without any packages which can influence the rules at all. So it is just plain pf and related pfSense behavoir.

marcosm

Thanks for the report and testing. A fix has been committed:
https://redmine.pfsense.org/issues/16076

Use the commit ID c181ebe180017116626da28f30407a1da3cba061 to apply the fix as a System Patch.

Please report back if that does not address the issue.

w0w

@marcosm said in Rules order randomly changes:

c181ebe180017116626da28f30407a1da3cba061

Applied patch and after several times adding "dummy" rule up and down on the LAN interface, occasionally deleting it, I've got it again.

marcosm

@w0w I'm not able to reproduce the issue with the referenced patch applied. Exact steps to reproduce it would be appreciated. Make sure you're testing using a single browser tab - making changes from multiple tabs (without reloading the page first each time) is not supported. Also make sure that only the referenced patch is applied and no other custom patches.

w0w

@marcosm
Will try to reproduce it and find the steps later, currently no other patches applied, this is unmodified 25.03 but I'll re-check it.

w0w

It is assumed that the Ethernet firewall rules tab already contains some rules in a predefined order, same for LAN and WAN and others.

Steps to Reproduce the Bug:

Go to the Firewall Rules LAN Tab.
Add a rule at the top of the list, then save without applying changes.
Move the rule to the middle of the list, then save without applying changes.
Delete the rule, then save without applying changes.
Switch to the Ethernet Rules Tab and check if the rule order is still intact.

marcosm

@w0w Thanks. Try this patch in addition to the previous one:

diff --git a/src/usr/local/www/firewall_rules_edit.php b/src/usr/local/www/firewall_rules_edit.php
index dc0d260ca9..c62a03e44f 100644
--- a/src/usr/local/www/firewall_rules_edit.php
+++ b/src/usr/local/www/firewall_rules_edit.php
@@ -1369,16 +1369,16 @@ if ($_POST['save']) {
 
 			$ridx = get_interface_ruleindex($tmpif, $after);
 			if (is_numeric($after) && ($tmpif == $if || (isset($pconfig['floating'])) || isset($pconfig['ethernet']))) {
-				// save the rule after the one being requested
-				array_splice($a_filter, $after+1, 0, array($filterent));
 				// shift the separators
 				$a_separators = config_get_path('filter/separator/' . strtolower($tmpif), []);
 				if ($after == -1) {
 					// rule is being placed on top
 					shift_separators($a_separators, -1);
+					array_splice($a_filter, $ridx['first'], 0, array($filterent));
 				} else {
 					// rule is being placed after another rule
 					shift_separators($a_separators, $ridx['index']);
+					array_splice($a_filter, $after+1, 0, array($filterent));
 				}
 				config_set_path('filter/separator/' . strtolower($tmpif), $a_separators);
 			} else {

w0w

@marcosm
Looks good now, tried some additional random tests, so far, so good.
Thank you.

SteveITS

FWIW I just ran into this symptom on a different/second router with 24.11. Unclear from above if this is known to be a 24.11 issue. Probably should have screen capped but it's late and I was focused on something else (boot issue on this router) before deciding I should post here. I did check and delete two rules but I think the order was wrong before I did that. However it's been on 24.11 for a couple months now so maybe had prior edits. I'll try to pay better attention next time.

Separately (only because I'm writing), I also just realized some of my pfBlocker geo rules were missing, and when I update those the separators I just moved around were in the wrong spot again because the geo rules get added towards the top of the page. (six pfB rules were missing/removed, the separators stay at "position 14" or whatever, and they are now in the wrong spot 6 lines too far down instead of also moving up 6 lines) This is probably unrelated to the rule ordering but I wanted to mention it.

marcosm

The fixes have been added to the System Patches package for 24.11.

SteveITS

@marcosm Great, thank you

louis2

@marcosm

At this moment I am using pfSense 24.11

I intend to do some rule changes in favor to test the stability of rule number.

Note that for a start:

I did apply All recommended patches for 24.11
and did define an extra patch based on your id c181ebe180017116626da28f30407a1da3cba061
logged into pfsense using ssh
created a ruledump with pfctl -vvsr > 20250306_1941_RuleDump.txt
did read that file into the db I did create in the past in order to create graylog lookups
among other things that DB creates a CRS32 as unique key for each rule

I will make some changes to the firewall rules in the coming days and after that create another dump. Read that dump in the DB and check if the same CRS32 do still correspond with the same rule numbers.

Since I have limited time, the whole process will take probably something like a week.
Note: I might have generated a ticket related to the issue a few years ago. I did not check.

SteveITS

@louis2 said in Rules order randomly changes:

I did apply All recommended patches for 24.11

The patch is in the recommended list but you need to update that package to see it. It wasn't in the list a couple days ago.