How to: Multi WAN OVPN Site-to-site redundant connections with Quagga OSPFd



  • I have been searching for a very long time for a solution to have redundancy over VPN connections with no luck.
    I finally went to a configuration that seems to work even if there is something still under investigation.
    Please, accept that some of the arguments may not be complete, I'm just a final user of the system.
    I'm writing this post with the hope it may help other people like me.
    Any further comment or suggestion will be really welcome.

    Let's start from a very basic situation with site A and site B connected both with 2 WANs.

    I will call:
    Site A - WAN 1 = WAN1A
    Site A - WAN 2 = WAN2A
    Site B - WAN 1 = WAN1B
    Site B - WAN 2 = WAN2B

    Lets assume the following LANs are configured:
    Site A - 192.168.1.0/24
    Site B - 192.168.2.0/24

    I want to connect Site A with Site B using a VPN with failover capabilities if the WAN should fail.

    I'm currently using PFsense version 2.2.X. This version has failover capabilities over the WAN, but I will not use them.
    There are several reasons why the VPN may not work and not all of them may depend from the WAN not working.

    My first attempt was to use IPSec using Failover groups and dynamic DNS but at the end I have to say that it does not work properly.
    Well, it works, but it does not cover all the situations, including when the IPSec stucks…

    I started then considering another option:
    create all the tunnels for any possibile combinations of WAN, and then create a route over them.
    This way is not necessary to establish the tunnel when the WAN fails as all the possibile tunnels are already created.
    If one tunnel fails because of the WAN, then you just have to route over the others.

    This method brings another important advantage: the VPN is tested over the tunnel and not over the WAN.
    What does it means?
    The WAN is tested by PFsense using an IP address. If you use the IP address of your gateway the WAN will fail only when the gateway can not be pinged, but we know most of the times the problem is after the gateway.
    If you use a public IP address, like Google's DNS, the WAN will fail when this address cannot be reached.
    But what happens if you can ping Google and not your remote site? The WAN stays up the VPN goes down and no failover is starting.

    By testing the connection over the tunnel it does not care if the WAN is working or not with all the rest of the world.
    If the tunnel can be reached at the other end, then is working and I do not need to failover.

    To accomplish this mechanism, I need to install a package, named Quagga OSPFd.
    This package let's you manage what is called "dynamic routing".

    Quagga with PFsense does not work with IPSec, so I have to move to OVPN.

    Step 1 - Create OVPN tunnels
    Create as many VPN tunnels is possibile using your WANs connection.
    In my example we will have:

    IPv4 Tunnel Network      Local port  Metric

    WAN1A - WAN1B  ---  10.1.1.0/30                  1195        10
    WAN1A - WAN2B  ---  10.1.1.4/30                  1196        20
    WAN2A - WAN1B  ---  10.1.1.8/30                  1197        30
    WAN2A - WAN2B  ---  10.1.1.12/30                1198        40

    Configure one of the two sites as the OVPN Server and the other as the OVPN Client.
    For each tunnel uses different IPv4 Network.  You just need two addresses, so use subnet /30.

    I will not cover here how to create a OVPN tunnel (Server/Client).
    However I can shortly say that:
    OVPN Server
    General information - You need a Local port different for each tunnel, other parameters according to standard configurations.
    Cryptographic Settings - Standard configurations
    Tunnel Settings - Deafult values except for IPv4 Tunnel Network (follow the example)

    OVPN Client
    General information - Server port must match the Server Local port, other parameters according to standard configurations.
    Cryptographic Settings - Standard configurations
    Tunnel Settings - Deafult values except for IPv4 Tunnel Network (must match the Server)

    Do not forget to allow OVPN traffic on your WANs for all the ports used.

    Once the tunnels are up and running (you can check them, Status -> OpenVPN) we can go to the next step.

    Step 2 - Install Quagga OSPFd
    Go to Packages and look for Quagga OSPF.
    Quagga must be installed on both Pfsense sites.

    Once finished, look at the Service menu and open Quagga.
    The procedure is the same on both sites.

    First go to Interface Settings
    Add an interface
    In the Interface dropdown list you should find the LAN, all the WANs and the OVPN tunnels.
    Select the first one and assign a Metric of 10
    Leave Area empty
    Add a description for this interface
    SAVE

    Repeat this step for all available OVPN tunnels, assigning a different metric to each.
    Metric is the order the tunnel will be used. Lower value means first.

    Once created the interfaces, move to Global Settings
    Type a Master Password (same on both sides)
    Router ID, you must use a different ID for each site. The syntax is 0.0.0.1 for Site A, 0.0.0.2 for Site B.
    Area, you can use 0.0.0.0 for both sites
    Redistribute static, select
    Redistribute Kernel, select
    These rules take precedence over any redistribute options specified above, create a list following your tunnel's IPv4 addresses:

    Disable            Disable
    Redistribution  Acceptance    Subnet to route      Area ID

    X                                    10.1.1.0/30              1
          X                                    10.1.1.4/30              1
          X                                    10.1.1.8/30              1
          X                                    10.1.1.12/30            1
                                                192.168.1.0/24        1

    192.168.1.0/24 in my example is the LAN of Site A
    Repeat the configuration on Site and use 192.168.2.0/24 instead (LAN of Site B)

    Save the configuration.

    The VPN will start working immediately.
    When one of the tunnel fails, after 40 seconds (OSPF default value) a new one will be used until the failed one is not recovered.

    The only issue I have found occours when the OVPN tunnel goes down, for any reason, also if I shut it down, and PFSense does not delete the associated route.

    When the tunnel tries to go up again the service stops because it is not able to add the route (that already exists).

    The only way I found is to destroy the hanging interface "ifconfig ovpnc1 destroy".



  • @Arancho:

    [SNIP]

    The only issue I have found occours when the OVPN tunnel goes down, for any reason, also if I shut it down, and PFSense does not delete the associated route.

    When the tunnel tries to go up again the service stops because it is not able to add the route (that already exists).

    The only way I found is to destroy the hanging interface "ifconfig ovpnc1 destroy".

    You know… this sounds a lot like the problem me and some others are having… though I never thought of fixing the interface that way.... I usually reboot once a service restart won help... ;)



  • @[NUT:

    link=topic=105139.msg586807#msg586807 date=1452746714]
    @Arancho:

    [SNIP]

    The only issue I have found occours when the OVPN tunnel goes down, for any reason, also if I shut it down, and PFSense does not delete the associated route.

    When the tunnel tries to go up again the service stops because it is not able to add the route (that already exists).

    The only way I found is to destroy the hanging interface "ifconfig ovpnc1 destroy".

    You know… this sounds a lot like the problem me and some others are having… though I never thought of fixing the interface that way.... I usually reboot once a service restart won help... ;)

    that's because ospf distributes the tunnel networks aswell.
    site1=a&c
    site2=c&d
    a–--b
    c----d

    when "a" goes down, the tunnel network(=route) for "a-b" is still being distributed via the "c-d" connection and never gets removed from the routing-table of site1.

    the solution is to prevent the tunnel-networks to be distributed.
    see:
    -Services: Quagga OSPFd: Edit interface: Accept Filter
    -play with disable acceptance/distribution in the global settings.

    takes some experimenting to get it to work & behaves differently when you run it on an interface or just a plain openvpn connection


Log in to reply