Rules order randomly changes

[w0w]

I don’t know exactly what triggers this bug, but the fact remains that if I start creating or deleting some firewall rules—possibly NAT, possibly regular ones—or changing their order by dragging them with the mouse, SOMETIMES, when applying the changes, the rule order gets shuffled randomly, which, of course, causes a lot of issues.

I usually notice it in the Ethernet rules tab because some change in rule order there results in a complete loss of internet connectivity. However, I’ve also observed that rules can randomly shift on other tabs as well.

As far as I remember, this started happening around pfSense 23.xx versions, but I’m not entirely sure… I’ve only been using Ethernet rules relatively recently, and I only noticed the issue because of them.
HOW SHOULD IT BE:

BROKEN:

Has anyone else encountered this?

[w0w]

Hmm...
It looks like everything revolves around two functions:
filter_rules_sort() and filter_rules_compare(), along with pfBlockerNG.

It seems that if the 'seq' number—which appears to be a temporary sequential rule number—matches one of pfBlockerNG's numbers, issues arise. I discovered this after modifying the code to save those numbers into config.xml instead of just handling them in RAM.

After trial and error, I ended up with...

function filter_rules_compare($a, $b) {
    // 1️⃣ If both rules have `seq`, use it as the primary sorting criterion
    if (isset($a['seq']) && isset($b['seq'])) {
        return $a['seq'] - $b['seq'];
    }

    // 2️⃣ Ensure Floating and Ethernet rules are placed above standard rules
    $a_is_special = isset($a['floating']) || isset($a['ethernet']);
    $b_is_special = isset($b['floating']) || isset($b['ethernet']);

    if ($a_is_special && !$b_is_special) {
        return -1; // Floating/Ethernet rule comes first
    }
    if ($b_is_special && !$a_is_special) {
        return 1;  // Standard rule comes after special rules
    }

    // 3️⃣ If both rules belong to the same interface, sort by `seq` if available
    if ($a['interface'] == $b['interface']) {
        if (isset($a['seq']) && isset($b['seq'])) {
            return $a['seq'] - $b['seq'];
        }
    }

    // 4️⃣ As a last resort, compare interface names
    return compare_interface_friendly_names($a['interface'], $b['interface']);
}

and

function filter_rules_sort() {
    $rules = config_get_path('filter/rule', []);

    /* Step 1: Collect all existing `seq` values */
    $existing_seq = [];
    foreach ($rules as $rule) {
        if (isset($rule['seq']) && is_numeric($rule['seq'])) {
            $existing_seq[] = $rule['seq'];
        }
    }

    /* Step 2: Find the next available unique `seq` */
    $max_seq = empty($existing_seq) ? 0 : max($existing_seq);
    $used_seq = array_flip($existing_seq);

    /* Step 3: Assign unique `seq` only to new rules */
    foreach ($rules as &$rule) {
        if (!isset($rule['seq']) || !is_numeric($rule['seq']) || isset($used_seq[$rule['seq']])) {
            do {
                $max_seq++;
            } while (isset($used_seq[$max_seq])); // Ensure `seq` is unique
            $rule['seq'] = $max_seq;
            $used_seq[$max_seq] = true;
        }
    }

    /* Step 4: Sort rules strictly by `seq` */
    usort($rules, "filter_rules_compare");

    /* ✅ Save `seq` in `config.xml` */
    config_set_path('filter/rule', $rules);
    write_config("Updated firewall rule order");
}

I'm not sure if that's everything or if there's still something left to do—I'm still testing it as it is for now.

Of course, I'm not a real programmer—it's just ChatGPT and my pathetic attempts.

[SteveITS]

@w0w And this was on 25.03?

Recently I found one of our routers had several rules moved from near the top to the bottom of the list…still in order but last. I was upgrading from 24.03 to 24.11 and I’m not sure if it happened before or after.

Two others the separator bar had moved slightly but IIRC that was a bug a while back for that if adding or maybe copying rules, so I didn’t think much of that since the order was the same.

[w0w]

@SteveITS said in Rules order randomly changes:

And this was on 25.03?

Yep.

[louis2]

@SteveITS

Not the same but perhaps related, rule numbers are not a fixed thing

And separator lines do not stay on their place. That is not new I did discover that I think two years ago.

One of the reasons I was looking into that is that I am exporting the rule list in favor of a GrayLog lookup table. I Graylog I like to see the rule description

I am still exporting that rule list now and than, but the rule numbers are unfortunately not completely stable.

One of the ways I did test this, in favor of the list but also in favor of rule groups, is that I did add a recognizable dummy rule at the to of each interface and interface group and on at the bottum. Those dummy rules (doing nothing) had an interface ID and a begin / start identifier hidden in the rule port number like "port is <90001> at the beginning of interface 9 and <90002> at the end of the rules related to inteface 9.

Perhaps this helps to track problems

[w0w]

@louis2
What packages you have installed? PfBlockerNG?

[SteveITS]

@w0w My recent/only experience was on 24.11. We do have pfBlocker but have for years. I tried to compare the rule order of different backup config files but they seem basically the same. So it may have just happened at the time which might imply a rare issue/race condition/upgrade issue. Haven't noticed it on any other routers and I've upgraded 6 or so to 24.11 recently.

[w0w]

After further testing, another bug was found with sorting: when a user changes the order and deletes a rule, the next rule added to the list appears in the same position where the last deleted rule was placed. Corrected code below.

function filter_rules_sort() {
    $rules = config_get_path('filter/rule', []);

    /* Step 1: Find the highest existing `seq` value */
    $max_seq = 0;
    foreach ($rules as $rule) {
        if (isset($rule['seq']) && is_numeric($rule['seq'])) {
            $max_seq = max($max_seq, $rule['seq']);
        }
    }

    /* Step 2: Ensure rules are stored in the order they appear in the GUI */
    $new_seq = 0;
    foreach ($rules as &$rule) {
        $rule['seq'] = ++$new_seq; // Assign sequential order based on GUI order
    }

    /* Step 3: Sort rules strictly by `seq` */
    usort($rules, "filter_rules_compare");

    /* ✅ Save the correct order in `config.xml` */
    config_set_path('filter/rule', $rules);
    write_config("Updated firewall rule order based on user-defined sorting.");
}

[w0w]

@SteveITS
I rarely change anything in the rule settings, but almost every time I do—whether it's modifying, deleting, or moving something—this mess with movements happens often.

[louis2]

@w0w

I noticed the described behavoir on a system without any packages which can influence the rules at all. So it is just plain pf and related pfSense behavoir.

[marcosm]

Thanks for the report and testing. A fix has been committed:
https://redmine.pfsense.org/issues/16076

Use the commit ID c181ebe180017116626da28f30407a1da3cba061 to apply the fix as a System Patch.

Please report back if that does not address the issue.

[w0w]

@marcosm said in Rules order randomly changes:

c181ebe180017116626da28f30407a1da3cba061

Applied patch and after several times adding "dummy" rule up and down on the LAN interface, occasionally deleting it, I've got it again.

[marcosm]

@w0w I'm not able to reproduce the issue with the referenced patch applied. Exact steps to reproduce it would be appreciated. Make sure you're testing using a single browser tab - making changes from multiple tabs (without reloading the page first each time) is not supported. Also make sure that only the referenced patch is applied and no other custom patches.

[w0w]

@marcosm
Will try to reproduce it and find the steps later, currently no other patches applied, this is unmodified 25.03 but I'll re-check it.

[w0w]

It is assumed that the Ethernet firewall rules tab already contains some rules in a predefined order, same for LAN and WAN and others.

Steps to Reproduce the Bug:

1. Go to the Firewall Rules LAN Tab.
2. Add a rule at the top of the list, then save without applying changes.
3. Move the rule to the middle of the list, then save without applying changes.
4. Delete the rule, then save without applying changes.
5. Switch to the Ethernet Rules Tab and check if the rule order is still intact.

[marcosm]

@w0w Thanks. Try this patch in addition to the previous one:

diff --git a/src/usr/local/www/firewall_rules_edit.php b/src/usr/local/www/firewall_rules_edit.php
index dc0d260ca9..c62a03e44f 100644
--- a/src/usr/local/www/firewall_rules_edit.php
+++ b/src/usr/local/www/firewall_rules_edit.php
@@ -1369,16 +1369,16 @@ if ($_POST['save']) {
 
 			$ridx = get_interface_ruleindex($tmpif, $after);
 			if (is_numeric($after) && ($tmpif == $if || (isset($pconfig['floating'])) || isset($pconfig['ethernet']))) {
-				// save the rule after the one being requested
-				array_splice($a_filter, $after+1, 0, array($filterent));
 				// shift the separators
 				$a_separators = config_get_path('filter/separator/' . strtolower($tmpif), []);
 				if ($after == -1) {
 					// rule is being placed on top
 					shift_separators($a_separators, -1);
+					array_splice($a_filter, $ridx['first'], 0, array($filterent));
 				} else {
 					// rule is being placed after another rule
 					shift_separators($a_separators, $ridx['index']);
+					array_splice($a_filter, $after+1, 0, array($filterent));
 				}
 				config_set_path('filter/separator/' . strtolower($tmpif), $a_separators);
 			} else {

[w0w]

@marcosm
Looks good now, tried some additional random tests, so far, so good.
Thank you.

[SteveITS]

FWIW I just ran into this symptom on a different/second router with 24.11. Unclear from above if this is known to be a 24.11 issue. Probably should have screen capped but it's late and I was focused on something else (boot issue on this router) before deciding I should post here. I did check and delete two rules but I think the order was wrong before I did that. However it's been on 24.11 for a couple months now so maybe had prior edits. I'll try to pay better attention next time.

Separately (only because I'm writing), I also just realized some of my pfBlocker geo rules were missing, and when I update those the separators I just moved around were in the wrong spot again because the geo rules get added towards the top of the page. (six pfB rules were missing/removed, the separators stay at "position 14" or whatever, and they are now in the wrong spot 6 lines too far down instead of also moving up 6 lines) This is probably unrelated to the rule ordering but I wanted to mention it.

[marcosm]

The fixes have been added to the System Patches package for 24.11.

[SteveITS]

@marcosm Great, thank you