Issue with failover gateway group over VTI tunnels
I have this network configuration:
At Site "A" I have a pfSense router (2.4.4-RELEASE-p3) with 2 different WAN connections and a LAN connection to a Mikrotik router. This LAN connection is used as a transfer network between the 2 routers.
The pfSense has 3 IPSec VTI tunnels to Site "B": tunnel1 goes over WAN1, tunnel2 and tunnel3 goes over WAN2.
All the computers, notebooks and printers are connected to the Mikrotik router.
The goal would be to make a failover connection between site A and B over the 3 tunnels, this means: if tunnel1 goes down, then the traffic should be automatically routed over tunnel2, if tunnel2 goes down, then the traffic should be automatically routed over tunnel3. And vica-versa: if tunnel2 recovers, then he traffic should be automatically routed over tunnel2 and if tunnel1 recovers, then he traffic should be automatically routed over tunnel1.
A simple diagram of the described network:
At site B there are Cisco devices which handle the failover solution.
At site A pfSense should handle the failover solution.
So I created a Gateway group at pfSense: I set tunnel1 to Tier1, tunnel2 to Tier2 and tunnel3 to Tier3. Then I created a firewall rule under the LAN tab:
source: 172.x.x.x destination: y.y.y.y protocol: any and set the above created gateway group.
Until this point the configuration/solution works correctly, if the connection is initiated from Site A's LAN network toward Site B's network (y.y.y.y).
The problem starts, when the connection is initiated from Site B's y.y.y.y network to Site A's 172.x.x.x network: the connection is not established in this case. But if I create a static route on pfSense like "destination: y.y.y.y gateway: tunnel1", then the connection is established over tunnel1.
I think this is the problem: the above described firewall rule on the pfSense does not catch the "answer" package, which is the answer to Site B's initiated connection. I could prove this by enabling the logging on the above firewall rule: when I initiated the connection from Site A, then I found the entries in pfSense's log, but when I initiated the connection from Site B, the log didn't show any entry to that firewall rule.
I'm not a pfSense expert, I just started to use it for some weeks.
If you have any solution to this situation, please share with me.
Might be because at the moment, VTI interfaces don't support the
reply-tokeyword in pf so the return routing has to follow the OS routing table.
Thank you for your helpful answer!
Do you have any information, when will be this reply-to feature available on VTI interfaces?
It's an upstream issue in FreeBSD, so we do not know.