CARP setup - Can't ping internal IP of backup firewall or access WebGUI internal
Without testing further, I am unable to confirm whether I am able to ping or access the interface when the backup becomes the master. I am concerned about pulling my primary offline as I can't lose access to the network - it's offsite.
The network is setup such that I am connected remotely via VPN to the primary firewall, the VIP for the first VLAN is 10.0.0.1, the primary firewall is at 10.0.0.2, and the secondary is at 10.0.0.3. While I am able to access everything else on the VLAN from this VPN, the secondary firewall at 10.0.0.3 is unavailable both via WebGUI and ping - even though there is a permit all rule on the VLAN pfSync'd from the first firewall.
I am able to access the WebGUI through the public IP of the secondary firewall, so the services themselves are up, and the secondary firewall knows that the master is online.
Is this normal behaviour, or am I missing something?
Yes, this is normal if the firewall is the VPN server and you sync all settings from master to backup. This way the backup box has equal VPN setup and the same tunnel network exists on both, master and backup. So if the backup replies to a request from a VPN IP it sends the packet to its own VPN interface which is down though.
To resolve this, you can use outbound NAT. Add an outbound NAT rule for the LAN interface, which translates IPs of packets coming from VPN tunnel network and have one of the LAN addresses of the boxes as destination (you may use an alias here or add a second rule for the other box) to its LAN address.
So if you connect to the backup box over VPN, the packets get the LAN address of master and replies from backup box are sent back to the masters LAN IP and the master will route the packets to the VPN client.