2.1.5 to 2.3 Cluster upgrade load balancer (IP Alias) problem
-
I have a pair of dell (pfsense 2.1.5) systems in a cluster with a primary and backup firewall in a primary/failover cluster using Carp. I used the native Load Balancer with 7 virtual IPs (IP Aliases) on the main WAN carp VIP. After the upgrade of the secondary firewall to 2.3, IPSEC and external access worked so it appears CARP is working but the Load Balancer or IP Aliases for them did not work when I failed over to the secondary (2.3). No traffic is getting blocked according to the firewall logs so it does not appear to be rule related. No configuration changes were made so that makes sense. I still have the primary at 2.1.5 so I failed back to that and it is working fine.
The only odd thing is that when the secondary came up after the upgrade, It's WAN interface was in MASTER status and the other CARP interfaces (3 others) were in BACKUP status while the primary (2.1.5) was in MASTER for all interfaces. They should have all been in BACKUP on the secondary 2.3 firewall. I assume this is because of the differences between 2.1.5 and 2.3 since it works fine on 2.1.5 for me. Interestingly the primary firewall on 2.1.5 was still working just fine including CARP and Load Balancer and VIP IP Aliases while the secondary on 2.3 showed it's WAN interface in MASTER status. I put the primary firewall into forced CARP disabled mode and the rest of the secondary firewall's interfaces went into MASTER status on the 2.3 secondary as expected and the primary firewall CARP went to disabled as expected. The Load Balancer or IP Aliases didn't work though on the secondary firewall.
I am wondering if maybe the problems of Loadbalancer/VIP Aliases not working for the secondary is because the secondary was already incorrectly in MASTER status on it's WAN interface so it didn't take over properly or something when the secondary transitioned to MASTER since it's WAN was already in MASTER status. After I disabled CARP on the primary I left it that way (primary firewall CARP disabled and secondary firewall in MASTER status on all interfaces) for about 5 hours and the connections to the Loadbalancer / IP Aliases never worked so I failed back to the primary server on 2.1.5 which worked just fine after a few seconds.
I plan to spend more time troubleshooting today looking at the interfaces with ifconfig, etc that I didn't have time to do yesterday night. If I see the same problem where the secondary 2.3 shows MASTER for WAN interface, I wonder if the secondary load balancer would start working if I stopped carp on the secondary firewall while carp is also stopped on the primary and then restart CARP on the secondary (2.3) to force the VIP CARP / IP Alias to come up on the secondary firewall without the 2.1.5 firewall using CARP during that time. I will find out later today.
-
I think this is an upgrade bug (maybe from 2.1.x to 2.3 only?).
I looked at ifconfig output and the IP Aliases were not coming up in the ifconfig output on the secondary firewall (pfsense 2.3) when disabling carp on the old primary firewall (still running 2.1.5) even though the Carp IPs were up.
I noticed that the virtualserver (VIP IP Alias interface) was blank in the GUI under Firewall\Virtual IPs. All I did was save the IP Alias for each one and then the interface then showed up as WANIF (which is what I call the wan). The VIP IP Aliases started working after that.
I looked at the change that was made when I saved the IP Alias in the Virtual Server section and it changed the interface in the <vip>section for the IP Aliases from:
<interface>wan_vip1</interface>
to
<interface>wan</interface>I remember reading that CARP interfaces were changed for 2.2 so maybe the upgrade script does not convert those for 2.1 to 2.3 upgrades.
The IP Aliases appear to be working now.</vip>
-
Looks like there's an issue in the config upgrade code with changing the parent interface in an edge case there. I'm looking into the root cause, but just editing the IP alias virtual IPs with CARP parents, picking the CARP IP, save and apply changes will fix in the mean time.