Upgrade 2.4.3 to 2.4.3_1 error in firewall rules
-
All is related to this bug https://redmine.pfsense.org/issues/8408
https://github.com/pfsense/pfsense/pull/3924
looks like not eveything merged to current ?
-
The commit from that PR is in master and RELENG_2_4_3, and is in 2.4.3-p1.
I could reproduce the problem before that commit but not now. What exactly does your configuration look like (config.xml entries, at least) for the affected VIPs and gateway?
I wanted to put some extra safety belts around that rule to make sure it couldn't be blank but following through the code it already appeared to be validated higher up.
-
Since I can't reproduce this still, and I don't have any config samples to work from, try this patch:
https://gist.githubusercontent.com/jim-p/f5fa7cf5fdfc8166f54394262386682f/raw/1ff237a9a52cef67c03db532c80fcc757969e711/8518.diff
That doesn't fix the root cause but it will prevent the broken rules from being placed in the ruleset.
It's still not clear how a blank entry is making into that v4 VIP array in the first place since it explicitly tests for v4 or v6 when making the array. That's why I need to see the config samples so I can get closer to the root of the problem.
-
Hi jimp,
I just send you a PM with my config snippets. I figured you might need them unredaced, so I did not post them here.
Thanks for looking into this!
-
just did the same :-)
-
ok, that did the trick.
Somehow when a PR was merged back from master to RELENG_2_3 it missed part of a commit that led to this happening. The safety belt patch above also helps, so I committed that as well.
I couldn't reproduce it initially because I was trying on 2.4.4 and the commit was OK there (master), but it was wrong on 2.4.3-p1.
This is the real fix:
https://github.com/pfsense/pfsense/commit/c9159949e06cc91f6931bf2326672df7cad706f4This is the safety belt:
https://github.com/pfsense/pfsense/commit/63b2c4c878655746f903565dec3f34b3d410153fYou can apply the first (or both) via the system patches package and that should get things back to normal.
-
Will try this tomorrow !
Thank you!
-
This is the real fix:
https://github.com/pfsense/pfsense/commit/c9159949e06cc91f6931bf2326672df7cad706f4This is the safety belt:
https://github.com/pfsense/pfsense/commit/63b2c4c878655746f903565dec3f34b3d410153fYou can apply the first (or both) via the system patches package and that should get things back to normal.
I've applied this as you described and my system is working again.
Thank you, and the other contributors to this thread, for fixing this so quickly.
Thanks
Tim -
Thanks! I applied the "real" fix only, rules loaded fine after that. I had to reboot the system to get CARP to work again without problems, though. Without a reboot the secondary still showed "Master" for some IPs (IPv4 and also IPv6, WAN and LAN). I could not find a pattern in this.
-
I confirm the real fix seems to does the trick. :D ;)
Thank you Jim.
-
What is the process for upgrading to 2.4.4 in the future? Will I need to revert the patch and then issue the upgrade or will I simply just upgrade to the next release as usual?
Do most people wait a while to upgrade usually? I'm kind of nervous now to do upgrades given this bug which basically broke NAT.
I will say though that I should have noticed the bug on the backup prior to upgrading the master, lessons learned.
-
What is the process for upgrading to 2.4.4 in the future? Will I need to revert the patch and then issue the upgrade or will I simply just upgrade to the next release as usual?
Do most people wait a while to upgrade usually? I'm kind of nervous now to do upgrades given this bug which basically broke NAT.
This bug was never present in 2.4.4, only 2.4.3-p1. You can upgrade as usual. The patch won't reapply itself automatically unless you went out of your way to set it that way, and since the patch won't apply on 2.4.4 anyhow it wouldn't matter if you did.
-
I have not yet upgraded and am unsure how to proceed. Is this a niche issue or is every configuration affected? Will this be addressed in a 2.4.3_2 release, or would I be waiting for 2.4.4?
-
That is unclear yet. Apply the patch with the System Patches package and you will have the fix immediately and won't have to upgrade to get it (or wait for a release)
-
Problem is not solved. The patches are just working to solve the problem with rules.debug. But I have a scenario, where OpenVPN is used and when the error occurs, the IPv4 traffic is blocked over the tunnel, before I installed the patches. After patch Installation, the error message about rules.debug disappeared, but OpenVPNs IPv4 traffic is still blocked (seems to be that ruleset isnt completely loaded).
The Problem came from an IPv6 Virtual-IP, which I added to the WAN Interface. I have tested this, with an without the patch. If I remove the IPv6 virtual IP the ruleset is completly loaded and OpenVPN works out of the box. If I add the IPv6 virtual IP again, the error occurs on the unpatched box and OpenVPNs IPv4 traffic is blocked on both boxes (no changes in rulesets and yes, routing works).
Please have a look at IPv6 virtual IP handling.
-
@ollli said in Upgrade 2.4.3 to 2.4.3_1 error in firewall rules:
Problem is not solved. The patches are just working to solve the problem with rules.debug. But I have a scenario, where OpenVPN is used and when the error occurs, the IPv4 traffic is blocked over the tunnel, before I installed the patches. After patch Installation, the error message about rules.debug disappeared, but OpenVPNs IPv4 traffic is still blocked (seems to be that ruleset isnt completely loaded).
The Problem came from an IPv6 Virtual-IP, which I added to the WAN Interface. I have tested this, with an without the patch. If I remove the IPv6 virtual IP the ruleset is completly loaded and OpenVPN works out of the box. If I add the IPv6 virtual IP again, the error occurs on the unpatched box and OpenVPNs IPv4 traffic is blocked on both boxes (no changes in rulesets and yes, routing works).
Please have a look at IPv6 virtual IP handling.
I have, and all problems that could be identified so far have been fixed. If something else is happening in your case, you have not provided nearly enough detail to speculate if it's even related to this.
Try on a 2.4.4 snapshot and see if the problem can be replicated there. If so, try to find a minimal configuration that can replicate the problem exactly so we can track it down. Just having an IPv6 VIP is not enough to trigger it.
-
Hello I have same problem. When i add IPv6 Virtual IP in CARP there is added this line to /tmp/rules.debug
pass out route-to ( em0 XX.XX.XX.XX ) from to !/ tracker 1000005913 keep state allow-opts label "let out anything from firewall host itself"This is the line causing syntax error. As you can see there is missing source and destination IP addresses.
XX.XX.XX.XX is IPv4 address of default gateway.
Recover steps are to disable IPv6 on network adapter and reenable it. Until I add IPv6 CARP VIP, everything works fine. -
@dano-pogac said in Upgrade 2.4.3 to 2.4.3_1 error in firewall rules:
Hello I have same problem. When i add IPv6 Virtual IP in CARP there is added this line to /tmp/rules.debug
The real fix is posted farther up in the thread. No need for workarounds.
-
Just wanted to vent my frustration in that I was impacted by this problem as well. Worst part was I have an HA setup, I had updated the Backup unit, and was going to upgrade the primary outside of business hours. Before I could apply the update to the primary I had a hardware failure, so when all traffic moved over to my backup unit we lost connectivity because of this bug.
At the time my fix was to change over to a development/Snapshot release. I really wish they would include an easy option to revert versions. Wasted a Saturday afternoon at the office trying to figure things out when typically I could have waited until Monday to troubleshoot the hardware failure.
-
hi, this caused me mass headaches too. Ive reverted back to 2.4.3 no p1. Didnt fancy patching things. Id have pulled the release and re-issued as I noticed that even with the issue the firewall was still passing traffic but was just completely open in some instances. :/
