23.01 BETA and f/w rule duplication bug

[lohphat]

The 23.01 BETA Release Notes mention bug Bug 13507 that caused duplicated tracking IDs for copied rules.

Is there a faster way to check for dupes other than open every rule and manually check its tracking ID? Is there a log or report which could be checked to find potential dupes faster?

[rcoleman-netgate]

@lohphat Download your config. Run it through a search function (like grep) and look for <tracker>

Drop that into a spreadsheet and sort. Look for duplicates.

This should be about 2-3 minutes (depending on your PC speed and access to grep)

[Cylosoft]

@lohphat I did it with the data in the states column of the rules editor. When the rules share tracking id's they'd show the exact same amount of data in/out in that column.

[lohphat]

@cylosoft That did it! Thanks!

I only had one interface out of seven with duped rules in one section.

I copied a lot of rules between i/f but didn't see any across interfaces.

Would it be possible for other unwary users with this bug to have the 23.01 upgrade process enumerate and then change tracker IDs as part of the process?

[lohphat]

@rcoleman-netgate

You can do the same with native Windows CMD:

type config.xml | findstr /c:"<tracker>" | sort

[lohphat]

@rcoleman-netgate

I may have uncovered another bug.

In the config.xml there was a section of rules for the "WireGuard" package i/f. I had tried the package a few months ago but abandoned the effort and removed the package.

So I reinstalled the package so that I could uncheck the "Keep config settings after package uninstall", since I figured that was why the config entries are still there. I did that but the config details are STILL in the config.

Here are two of the six rule entries for the now-deleted interface still stuck in the config after package uninstall:

<rule>
<id/>
<tracker>1666565958</tracker>
<type>block</type>
<interface>WireGuard</interface>
<ipprotocol>inet</ipprotocol>
<tag/>
<tagged/>
<max/>
<max-src-nodes/>
<max-src-conn/>
<max-src-states/>
<statetimeout/>
<statetype>
<![CDATA[ keep state ]]>
</statetype>
<os/>
<source>
<any/>
</source>
<destination>
<network>lan</network>
</destination>
<descr>
<![CDATA[ NO access to LAN vlan ]]>
</descr>
<created>
<time>1620692436</time>
<username>
<![CDATA[ [REDACTED] (Local Database) ]]>
</username>
</created>
<updated>
<time>1620703184</time>
<username>
<![CDATA[ [REDACTED] (Local Database) ]]>
</username>
</updated>
</rule>
<rule>
<id/>
<tracker>1666565958</tracker>
<type>block</type>
<interface>WireGuard</interface>
<ipprotocol>inet</ipprotocol>
<tag/>
<tagged/>
<max/>
<max-src-nodes/>
<max-src-conn/>
<max-src-states/>
<statetimeout/>
<statetype>
<![CDATA[ keep state ]]>
</statetype>
<os/>
<source>
<any/>
</source>
<destination>
<network>opt3</network>
</destination>
<descr>
<![CDATA[ NO access to WIFI vlan ]]>
</descr>
<created>
<time>1620692468</time>
<username>
<![CDATA[ [REDACTED] (Local Database) ]]>
</username>
</created>
<updated>
<time>1620703170</time>
<username>
<![CDATA[ [REDACTED] (Local Database) ]]>
</username>
</updated>
</rule>

[Bob.Dig]

@lohphat said in 23.01 BETA and f/w rule duplication bug:

type config.xml | findstr /c:"<tracker>"

Thank you! I found 3 duplicates on one WG Interface, but I am on 22.05-RELEASE, so no wonder. "Fixed" it now.

[rcoleman-netgate]

@lohphat friends don't let friends run Windows. :)

[rcoleman-netgate]

@lohphat said in 23.01 BETA and f/w rule duplication bug:

I may have uncovered another bug.

Check the redmines. Open one if they don't exist yet.

https://redmine.pfsense.org/

At this point I'm moving this thread into the DEVELOPMENT channel.

[lohphat]

@rcoleman-netgate said in 23.01 BETA and f/w rule duplication bug:

friends don't let friends run Windows. :)

That's why I consider myself "bi-OS versatile".

And yes, I've had all my shots.

[lohphat]

@rcoleman-netgate

Bug Created #13829 "WG not removing interface rules from config even if "Keep Configuration" is unchecked before pkg removal"

[stephenw10]

Hmm, I don't think that is a bug. At least not in that way. The config that is kept or not is the Wireguard config specifically not firewall rules on an interface that may have been assigned to the Wireguard tunnel at that time. I would not expect rules to be removed.

Steve

Edit: Pretty much what Jim said there.

[lohphat]

@stephenw10

The bug is re-opened the bug as he agreed that the removal of the package also removes the interface from the GUI which prevents rules from being deleted afterwards. At least there should be a reminder in the WG settings to remind the user to remove any rules created before removing the package, or even an additional checkbox to "delete all WG i/f rules upon removal"

But there is a larger question that in these situations, there are times where config data gets orphaned and hidden in limbo which are related to a package. In theory, all config data should be accessible for edit/deletion via the GUI -- forcing a user to muck around with an XML file to delete orphaned rules and settings is asking for trouble.

[stephenw10]

You might argue that any config data that isn't exposed doesn't matter because it doesn't do anything on the firewall. Which appears to be the situation here.
What's potentially an issue is if/when you create and assign a new interface after removing the WG interface it can inherit the old rules. That wouldn't apply to WG group rules though.

[lohphat]

@stephenw10 said in 23.01 BETA and f/w rule duplication bug:

You might argue that any config data that isn't exposed doesn't matter because it doesn't do anything on the firewall.

Well that tingles my security Spidey Sense[tm] and not in a good way.

Having unused settings -- and potentially scripts -- hidden in a config file is an invitation for an attack vector.

My personal preference is that active config files only contain known settings and comments the admin desires to remain. Hiding or orphaning settings invites config file bloat over time and is fodder for misconfigurations or unintended behaviors.

The ability for a package to leave its config intact after package removal is really handy and needed as sometimes a package removal and reinstall in necessary. But other than that, e.g. if an interface is deleted any associated settings (like f/w rules) should also disappear since the i/f is now gone from the GUI.

[stephenw10]

Yup, that's not an unreasonable argument IMO.