Problem with connectivity outside of IPSEC when member is down.
-
I'm not sure if this belongs in the IPSEC or the Multiwan section.
I have 3 internet connections. A Starlink connection that handles most outgoing traffic, a fixed wireless connection that I use for incoming traffic (mostly VOIP) and a relatively slow connection that I use exclusively for IPSEC VPN connections for Zabbix to monitor a collection of remote servers on various client networks. I have the gateways setup so that the Starlink connection is Tier1, the second connection is Tier2 and the Slow IPSEC connection is Tier3. The gateway group is set to failover when the member is down.
Since Fiona rolled through here over the weekend the IPSEC (Opt1) connection has been down because of power issues. The gateway status screen shows that it is down and off-line:
For the most part - everything works as it should however, certain sites can not be reached. If I traceroute to them, I get timeouts for everything after the first hop (the PFSense box).
If I go into the Interface configuration and disable the OPT1 connection then everything works correctly and as soon as I re-enable the connection the problem returns.
I realized that all of the problems are servers that have an IPSEC connection. So for instance I have a server with a public IP address of 142.176.xxx.yyy. The IPSEC connction uses this address for its remote gateway, and I access the PFSense box at that end of the connection at 192.168.2.x There is a publicly available server at that site that uses port forwarding in PFsense to redirect https requests from the public address to the Natted address. However when the OPT1 connection is down, as long as the interface is enabled, pfsense tries to route all requests to 142.176.xxx.yyy through the OPT1 connection which is down.
I have studied the options in the IPSEC configuration and can find nothing that will redirect the traffic if the interface is down - is there something I am missing?