Dual site to site OpenVPNs, if both up entire VPN Connections to core die.

BrianBonnell

We are trying to configure our site to site OpenVPNs with redundancy to our core network. But if we bring up the secondary VPN the primary dies then the secondary dies shortly there after, then neither will come up until one is disabled.

This is a rough layout of how the VPN connections are configured.

Core Network (192.168.2.0/24) –> Core Router/Firewall1 (acting as OpenVPN Server 2 -10.0.7.0/24-TCP Port 1994)---> Internet <----Site1 Router/Firewall1 (with a primary OpenVPN Connection acting as Client to OpenVPN Server 2- (10.0.7.2-TCP Port 1994)----Branch Office LAN (192.168.3.0/24)

Core Network (192.168.2.0/24 --> Core Router/Firewall2 (acting as OpenVPN Client to OpenVPN Server 2-10.0.8.2-TCP Port 1995)---> Internet <----Site1 Router/Firewall1 (acting as OpenVPN Server 2 (10.0.8.0/24-Port TCP 1995) ---> Branch Office LAN (192.168.3.0/24

Note: Core Routers 1 & 2 are physically separate pfsense boxes. Site1 Router/Firewall1 is one single pfsense box.
All 3 are V1.2.3-RC1

We have several sites that we would like to implement this configuration, we have this configuration on a second site, which also experiences the same issue. So we have disable one tunnel so we can maintain a connection to our core. We HAVE not set up a VPNs between the 2 sites, just between the two sites and the core.

When just either one VPN is up, traffic passes in either direction just fine. But is we bring up the other connection, then both VPNs go dead. Then if try to bring them up manually neither will come up, we end up having to disable one and reboot one of the two core router/firewalls then only one of the two VPNs will be up and traffic passes with no issues.

Is this configuration possible? We use different ports on the Core Routers/Firewalls (TCP 1994 on 1 and TCP 1995 on 2) these same ports are specified on the client site for the respective connections.

Both Core Routers/Firewalls have different PUBLIC IPs on the WAN connection from two different local ISP carriers.

Each Site has a single static PUBLIC IP on the WAN side provided by a local ISP carrier, which is a different carrier at each site and is not the same carrier as the core router ISPs

I'm not sure why both VPNs won't stay up.

Any suggestions? We need to have VPN redundancy to our core from each site before we can setup VPNs between sites (btw, these inter-site VPNs won't be redundant, just our Core to Site VPNs must be redundant.

We do NOT have RIP/RIP2 enabled on any router/firewall either.

I added a diagram of what we are trying to accomplish.

![pfsense OpenVPN Connections-Site to Core and Site to Site.jpg](/public/imported_attachments/1/pfsense OpenVPN Connections-Site to Core and Site to Site.jpg)
![pfsense OpenVPN Connections-Site to Core and Site to Site.jpg_thumb](/public/imported_attachments/1/pfsense OpenVPN Connections-Site to Core and Site to Site.jpg_thumb)

fastcon68

Since you have static IP's why not use IPSEC instead of OpenVPN? Just curious why? I have 7 vpn's comming in to one site and and have so real issues.

My only issue cross site connection communication.

RC

BrianBonnell

IPSEC doesn't work, the connection drops ever few seconds when bandwidth use is high within the IPSEC tunnel. OpenVPN is the only one that seems to stay up.

To prevent cross-communication I plan to setup static routes in for prefered paths.

fastcon68

What type of connects do you sites have? I run IPSEC tunnels over DSL through a bunch of different hardware solutions. Just curious, most of my customers has DSL with static and DCHP connects. I have seen issues with DHCP tunnels with high utilization. I use some bandwitch limitations on those connections to prevent issues.
RC

BrianBonnell

I have (2) T1s at each site. Each T1 is throttled at 5 mpbs-sync (5 up/ 5 down) for the time being. All are static IPs. We now have 2 more sites commig up, being added to CORE VPNs.

So there are now 6 sites. Currently none are inter-connected, they only run into the CORE.

fastcon68

I think if both the connections are dieing when they are brought up it could be a routing issue. You create a loop and it drop the connection. Think of it like this. Take a switch connection port 1 to port 1, take a cable and then connection port 24 to port 24 your traffic is going no where.

Now keep this in mind with your tunnel configuration you would need to give OpenVPN a priority of 10 on core 1, and on OpenVPN on core 2 a priority of 100. Your traffic knows to use the one with the high priority and if the core 1 link goes down then the traffic would go down the slower path.

I verified with a networking friends, alot depends on the router, switches and the priority of the routes. I think with OpenVPN you might not be able to control the priority of the routes, but maybe some one can let up know if two boxes have connection can one be slower than the other.
RC

BrianBonnell

Thanks, I wonder if the priority could be controlled VIA BGP?

Briantist

I had a routing issue, with openvpn, but perhaps for a different reason, as seen here:
http://forum.pfsense.org/index.php/topic,18191.0.html

The public IP I needed to connect to for the vpn was the same network available on one of my OPTs, but I really needed to connect to this one IP in that range through the WAN, so I just created a static route. For example:

WAN 9.9.9.9 with gateway of 9.9.9.1
OPT 1.2.3.3/24

I needed to connect to 1.2.3.2 over the WAN, so I added a static route on the WAN:
1.2.3.2/32 9.9.9.1

Works perfectly; I'm not sure if this could help in your situation, but maybe?

BrianBonnell

I got it working….had to fenaggle BGP but it is now working, and no route-flapping. WHooo Hoooo! :)