OpenVPN site-to-site dual-wan on one side with AUTOMATIC failover?

[cyboc]

I'm running pfSense 2 BETA 5 (Jan 17, 2011 build).

I'm trying to setup a site-to-site VPN where the server side has dual-WAN and client side has single WAN and both sides are running pfSense. I want to have automatic failover for the clients (by using two "remote" lines in client OpenVPN config) if the primary WAN connection goes down on the server side. Furthermore, I want computers on each end of the tunnel to be reachable from computers at the other end of the tunnel (e.g. ping a computer on client's LAN from a computer on server's LAN and vice-versa).

I have read page 326 in the pfSense book where it talks about OpenVPN and Multi-WAN. It says that this is easier if the OpenVPN server is running in TCP mode because I only need one server, whereas with UDP I need two servers (one for each WAN interface, with separate address pool and with a local x.x.x.x line in the advanced options).

I can confirm that automatic failover seems to work perfectly in TCP mode (also seems to be confirmed by this post).

The problem is, UDP mode is generally preferred because of the potential performance problems with tunnelling TCP over TCP (discussed on page 302 of the book; see also Why TCP Over TCP Is A Bad Idea).

I have tried following the steps in section 15.7.1.2 of the book for using UDP mode in a multi-WAN scenario but I can't seem to get failover to work. If I yank the network cable on the primary WAN connection, the tunnel goes down and comes back up a minute later BUT pinging from one side of the tunnel to the other stops working.

As far as I can tell, maybe the UDP solution doesn't work because of conflicting routes to the client's LAN. Assuming the client's LAN is 172.16.5.0/24 and the tunnel networks are 10.9.5.0/24 and 10.10.5.0/24 respectively for the WAN1 UDP server and the WAN2 UDP server, there would be two routes needed to the client's LAN:

destination, gateway, interface
172.16.5.0/24 10.9.5.2 ovpns1
172.16.5.0/24 10.10.5.2 ovpns2

If the server router has a packet destined for 172.16.5.0/24, how does it know which tunnel to send it to? Indeed, from looking at the routing table on the server after bringing up both OpenVPN UDP servers, it seems that only the first route is added. From checking the OpenVPN logs it looks like the attempted addition of the second route fails with "ERROR: FreeBSD route add command failed: external program exited with error status: 1".

Questions:

1. Has anyone managed to get automatic failover to work with UDP, following the "two server" technique on page 326 of the book?

2. The book says "OpenVPN servers with UDP are also multi-WAN capable but with some caveats that aren't applicable with TCP, because of the way pf's multi-WAN routing functions. Can anyone elaborate on this? How is multi-WAN routing of UDP different than TCP? Has it got something to do with pf's "reply-to" attribute (i.e. does reply-to work differently for UDP?)?

3. Should I just go back to using TCP or is there a better way to get automatic failover working for my VPN?

4. Would automatic failover work if I reversed which side is the client and which is the server? That is, if the dual-wan side was the client and the single-wan side was the server?

[jng]

I had a similar problem.  This is how I solved it –

Instead of creating an OpenVPN server on each WAN interface, create one on the LAN interface and create corresponding NAT/FW rules to "port forward" the appropriate UDP port back to itself on the LAN side.  Since NAT is able to "understand" the dual-wan connections much better, this abstracts away the whole complexity of the issue.

The advantages:

• you only need one server per site-to-site tunnel (as opposed to one per WAN connection per tunnel)

• clients can be configured with multiple "remote" options in OpenVPN client so that when one connection fails, it tries the other

• simple, and easy to understand.

Here's an example from my configuration.  I have two WAN connections called WAN1 and WAN2 respectively.  In VPN -> OpenVPN -> Servers, I create a configuration.  In my case, I decided to use UDP port 1295 for the server.  I selected "LAN" under interface.

In my Firewall -> NAT, I create two NAT rules that port forward UDP 1295 on WAN1 to UDP 1295 on LAN.  Similarly for WAN2.  Thanks to the wonderful interface of pfSense, firewall rules get automatically created for these NAT rules on the respective interfaces also.

The firewall rules look like this (in case you're interested):

Seems to work for me, though I haven't given much thought to any potential disadvantages.  Has any one else tried this before?  I see from some other posts that people are binding interfaces to tunnels then using things like OSPF.  That is another possible avenue you may wish to pursue.  For my scenario, the NATing just seemed simpler to understand.

[cyboc]

Thanks Jng,

I'll give that a try. Anyone think this approach is a bad idea for any reason?

[jimp]

That should work fine, I've done that a time or two in the past.