Rules order randomly changes

[w0w]

@louis2
What packages you have installed? PfBlockerNG?

[SteveITS]

@w0w My recent/only experience was on 24.11. We do have pfBlocker but have for years. I tried to compare the rule order of different backup config files but they seem basically the same. So it may have just happened at the time which might imply a rare issue/race condition/upgrade issue. Haven't noticed it on any other routers and I've upgraded 6 or so to 24.11 recently.

[w0w]

After further testing, another bug was found with sorting: when a user changes the order and deletes a rule, the next rule added to the list appears in the same position where the last deleted rule was placed. Corrected code below.

function filter_rules_sort() {
    $rules = config_get_path('filter/rule', []);

    /* Step 1: Find the highest existing `seq` value */
    $max_seq = 0;
    foreach ($rules as $rule) {
        if (isset($rule['seq']) && is_numeric($rule['seq'])) {
            $max_seq = max($max_seq, $rule['seq']);
        }
    }

    /* Step 2: Ensure rules are stored in the order they appear in the GUI */
    $new_seq = 0;
    foreach ($rules as &$rule) {
        $rule['seq'] = ++$new_seq; // Assign sequential order based on GUI order
    }

    /* Step 3: Sort rules strictly by `seq` */
    usort($rules, "filter_rules_compare");

    /* ✅ Save the correct order in `config.xml` */
    config_set_path('filter/rule', $rules);
    write_config("Updated firewall rule order based on user-defined sorting.");
}

[w0w]

@SteveITS
I rarely change anything in the rule settings, but almost every time I do—whether it's modifying, deleting, or moving something—this mess with movements happens often.

[louis2]

@w0w

I noticed the described behavoir on a system without any packages which can influence the rules at all. So it is just plain pf and related pfSense behavoir.

[marcosm]

Thanks for the report and testing. A fix has been committed:
https://redmine.pfsense.org/issues/16076

Use the commit ID c181ebe180017116626da28f30407a1da3cba061 to apply the fix as a System Patch.

Please report back if that does not address the issue.

[w0w]

@marcosm said in Rules order randomly changes:

c181ebe180017116626da28f30407a1da3cba061

Applied patch and after several times adding "dummy" rule up and down on the LAN interface, occasionally deleting it, I've got it again.

login-to-view

[marcosm]

@w0w I'm not able to reproduce the issue with the referenced patch applied. Exact steps to reproduce it would be appreciated. Make sure you're testing using a single browser tab - making changes from multiple tabs (without reloading the page first each time) is not supported. Also make sure that only the referenced patch is applied and no other custom patches.

[w0w]

@marcosm
Will try to reproduce it and find the steps later, currently no other patches applied, this is unmodified 25.03 but I'll re-check it.

[w0w]

It is assumed that the Ethernet firewall rules tab already contains some rules in a predefined order, same for LAN and WAN and others.

Steps to Reproduce the Bug:

1. Go to the Firewall Rules LAN Tab.
2. Add a rule at the top of the list, then save without applying changes.
3. Move the rule to the middle of the list, then save without applying changes.
4. Delete the rule, then save without applying changes.
5. Switch to the Ethernet Rules Tab and check if the rule order is still intact.

[marcosm]

@w0w Thanks. Try this patch in addition to the previous one:

diff --git a/src/usr/local/www/firewall_rules_edit.php b/src/usr/local/www/firewall_rules_edit.php
index dc0d260ca9..c62a03e44f 100644
--- a/src/usr/local/www/firewall_rules_edit.php
+++ b/src/usr/local/www/firewall_rules_edit.php
@@ -1369,16 +1369,16 @@ if ($_POST['save']) {
 
 			$ridx = get_interface_ruleindex($tmpif, $after);
 			if (is_numeric($after) && ($tmpif == $if || (isset($pconfig['floating'])) || isset($pconfig['ethernet']))) {
-				// save the rule after the one being requested
-				array_splice($a_filter, $after+1, 0, array($filterent));
 				// shift the separators
 				$a_separators = config_get_path('filter/separator/' . strtolower($tmpif), []);
 				if ($after == -1) {
 					// rule is being placed on top
 					shift_separators($a_separators, -1);
+					array_splice($a_filter, $ridx['first'], 0, array($filterent));
 				} else {
 					// rule is being placed after another rule
 					shift_separators($a_separators, $ridx['index']);
+					array_splice($a_filter, $after+1, 0, array($filterent));
 				}
 				config_set_path('filter/separator/' . strtolower($tmpif), $a_separators);
 			} else {

[w0w]

@marcosm
Looks good now, tried some additional random tests, so far, so good.
Thank you.

[SteveITS]

FWIW I just ran into this symptom on a different/second router with 24.11. Unclear from above if this is known to be a 24.11 issue. Probably should have screen capped but it's late and I was focused on something else (boot issue on this router) before deciding I should post here. I did check and delete two rules but I think the order was wrong before I did that. However it's been on 24.11 for a couple months now so maybe had prior edits. I'll try to pay better attention next time.

Separately (only because I'm writing), I also just realized some of my pfBlocker geo rules were missing, and when I update those the separators I just moved around were in the wrong spot again because the geo rules get added towards the top of the page. (six pfB rules were missing/removed, the separators stay at "position 14" or whatever, and they are now in the wrong spot 6 lines too far down instead of also moving up 6 lines) This is probably unrelated to the rule ordering but I wanted to mention it.

[marcosm]

The fixes have been added to the System Patches package for 24.11.

[SteveITS]

@marcosm Great, thank you

[louis2]

@marcosm

At this moment I am using pfSense 24.11

I intend to do some rule changes in favor to test the stability of rule number.

Note that for a start:

• I did apply All recommended patches for 24.11
• and did define an extra patch based on your id c181ebe180017116626da28f30407a1da3cba061
• logged into pfsense using ssh
• created a ruledump with pfctl -vvsr > 20250306_1941_RuleDump.txt
• did read that file into the db I did create in the past in order to create graylog lookups
• among other things that DB creates a CRS32 as unique key for each rule

I will make some changes to the firewall rules in the coming days and after that create another dump. Read that dump in the DB and check if the same CRS32 do still correspond with the same rule numbers.

Since I have limited time, the whole process will take probably something like a week.
Note: I might have generated a ticket related to the issue a few years ago. I did not check.

[SteveITS]

@louis2 said in Rules order randomly changes:

I did apply All recommended patches for 24.11

The patch is in the recommended list but you need to update that package to see it. It wasn't in the list a couple days ago.

[w0w]

@w0w said in Rules order randomly changes:

I don't know if I'm just lucky or what… Sorry, it happened again.

login-to-view

Steps to reproduce:

1. Copy one or two rules using the "Actions" → "Copy" button on the right side of the selected rule. I copied two rules (7.1 and 8.1), pressed Save, but did not apply the changes.
2. Go to the LAN or any other interface and copy a rule the same way, somewhere in the middle of the list, just like you did in the Ethernet tab. Press Save, but do not apply the changes.
3. Go back to the Ethernet tab – the order is messed up.

[marcosm]

@w0w Thank you. Try with this patch:

diff --git a/src/etc/inc/pfsense-utils.inc b/src/etc/inc/pfsense-utils.inc
index 5095c9ff0f..23cdbdf295 100644
--- a/src/etc/inc/pfsense-utils.inc
+++ b/src/etc/inc/pfsense-utils.inc
@@ -3088,18 +3088,33 @@ function filter_rules_sort() {
 	config_set_path('filter/rule', $rules);
 }
 function filter_rules_compare($a, $b) {
-	if ((isset($a['floating']) && isset($b['floating'])) ||
-	    (isset($b['ethernet']) && isset($b['ethernet']))) {
-		return $a['seq'] - $b['seq'];
-	} else if (isset($a['floating']) || isset($a['ethernet'])) {
-		return -1;
-	} else if (isset($b['floating']) || isset($b['ethernet'])) {
+	if (isset($a['ethernet'])) {
+		// Ethernet rules always come first.
+		if (!isset($b['ethernet'])) {
+			return -1;
+		}
+	} elseif (isset($a['floating'])) {
+		// Floating rules are placed after Ethernet rules and before Interface rules.
+		if (isset($b['ethernet'])) {
+			return 1;
+		} elseif (!isset($b['floating'])) {
+			return -1;
+		}
+	} elseif (isset($b['ethernet']) || isset($b['floating'])) {
+		// Interface rules are placed after Ethernet and Floating rules.
 		return 1;
-	} else if ($a['interface'] == $b['interface']) {
-		return $a['seq'] - $b['seq'];
-	} else {
+	} elseif ($a['interface'] != $b['interface']) {
+		// Interface rules for the same interface are grouped together.
 		return compare_interface_friendly_names($a['interface'], $b['interface']);
 	}
+
+	// Use the sequence number as a last resort.
+	if (isset($a['seq']) && isset($b['seq'])) {
+		return ($a['seq'] <=> $b['seq']);
+	}
+
+	// Nothing to compare.
+	return 0;
 }
 
 function generate_ipv6_from_mac($mac) {

If testing on 24.11, apply this one after first applying the latest recommended System patches. If testing on 25.03-BETA, first apply c181ebe180017116626da28f30407a1da3cba061 and the previous one, then this one.

[w0w]

@marcosm said in Rules order randomly changes:

At first glance, everything seems to be working, but we won't rush. I'll test it for some more time.