2 problems with routing on site2site + failover (carp)



  • Hi guys, I have some problem with routing between firewalls on my 2 sites, but routing between pc on this sites works fine.

    On each site firewalls are configured with CARP failover, that works fine, then VIPs are normally assigned on masters.
    There are a tunnel OpenVPN site2site where interfaces are WAN VIPs, remote and local subnet are the LAN subnets of this sites, preshared key, and tunnel network: 10.200.0.0/24. Rules on OpenVPN interface and LAN interfaces allow all traffic between sites.

    site A
    lan: 172.16.76.0/24
    Firewall Master (FAM) lan ip: 172.16.76.253
    Firewall Backup (FAB) lan ip: 172.16.76.254
    vip carp lan ip: 172.16.76.1
    vip carp wan ip: "WanVipA"

    site B
    lan: 10.10.10.0/24
    Firewall Master (FBM) lan ip: 10.10.10.250
    Firewall Backup (FBB) lan ip: 10.10.10.251
    vip carp lan ip: 10.10.10.1
    vip carp wan ip: "WanVipB"

    Problem #1)
    If I do a traceroute from pcA to pcB all is ok, ie:
    traceroute to 10.10.10.2 (10.10.10.2), 30 hops max, 40 byte packets
    1  172.16.76.253 (172.16.76.253)  0.297 ms  0.301 ms  0.315 ms –--------- FAM
    2  10.200.0.2 (10.200.0.2)  3.699 ms  3.780 ms  3.875 ms  ------------- TUNNEL
    3  10.10.10.2 (10.10.10.2)  4.501 ms  3.779 ms  3.692 ms ------------- pcB

    and if I traceroute from FAM to pcB that works too, then both firewall master a and pc a knows how to reach pcB.
    traceroute to 10.10.10.2 (10.10.10.2), 64 hops max, 52 byte packets
    1  10.200.0.2 (10.200.0.2)  3.467 ms  3.394 ms  3.548 ms ---------- tunnel
    2  10.10.10.2 (10.10.10.2)  4.178 ms  3.672 ms  3.666 ms ---------- pcB

    But If i try to traceroute from pcA or FAM to FBM, the from one host of site A to the master firewall of site B, something goes wrong.

    traceroute pcA to FBM
    traceroute to 10.10.10.250 (10.10.10.250), 30 hops max, 40 byte packets
    pfSenseNewNewMaster.tecnocall.eu (172.16.76.253)  0.322 ms  0.335 ms  0.343 ms
    2  * * *
    3  * * *

    traceroute FAM to FBM
    traceroute to 10.10.10.250 (10.10.10.250), 64 hops max, 52 byte packets
    1  * * *
    2  * * *
    3  * * *

    Howewer, if I ping from pcA or FAM to FBM everything works fine.

    I don't understand why i can trace to every ip on remote network, except firewalls ip!

    Problem #2)
    This is my real problem, the #1 is the minor one.

    Backup firewalls can't reach remote networks, then the backup firewall on other site is not reachable directly, I can do it using a NAT on master that points to backup ip, but this is horrible.
    The point is that the tunnel exists between master firewalls, and checking the routing table in backup firewall there are a route like "destination: remote network netif: ovpns3", but  ovpns3 exists only on master because the tunnel is up between masters.

    Why OpenVPN/pfsense don't create automagically a special route for backup firewall that doesn't have the tunnel up?
    I suppose i need something like a static route, es. on FAM i should add: src lan_a dst: lan_b gateway vip_lan_a. In this way, maybe, backup firewall will have a gateway (the master) that can route traffic in the tunnel. But I'm sure about that.

    What is the best pratice / solution to solve this problem? The problem #1 is really a problem or is normal?