OpenVPN site-to-site multi-WAN
-
I have multiple offices and each office has internet access via 2 ISPs:
SiteA-WAN - public IP A1
SiteA-OPT1 - public IP A2SiteB-WAN - public IP B1
SiteB-OPT1 - public IP B2I would like an OpenVPN connection that works between the offices even if one of (A1,A2) or (B1,B2) is down. I don't particularly care about which connections are used when they are all available (e.g. B1->A1, B2->A2, B1->A2, B2->A1 - any pairing is fine). I am using Peer to Peer (Shared Key) at present.
Option 1:
I can make the OpenVPN Server end at SiteA listen on A1 and A2.
The client at SiteB would then need to be able to:
a) Use either B1 or B2 to exit from SiteB (whichever is available); and
b) Connect to either of A1 or A2, whichever is reachable.I should be able to sort out (a) with some firewall/routing rules to make sure OpenVPN at SiteB will try both WAN and OPT1 to get out.
But (b) seems difficult - how do I tell the OpenVPN client that it has 2 possible server IP addresses that it should try?
(This also seems to be an issue for automating road warriors if the company office OpenVPN server is available on multiple public IPs)Or, do I
Option 2:
(a) setup 2 servers, 1 listening on A1 the other listening on A2.
(b) have 2 clients, connecting from B1 to A1 and B2 to A2.but then if (A1 and B2 are down) or (A2 and B1 are down) there will be no VPN.
Option 3:
So maybe I need 4 servers and 4 clients:
Server1 listening on A1 for connects from B1
Server2 listening on A1 for connects from B2
Server3 listening on A2 for connects from B1
Server4 listening on A2 for connects from B2
Client1 connecting from B1 to A1
Client2 connecting from B2 to A1
Client3 connecting from B1 to A2
Client4 connecting from B2 to A2and in normal operation there would be 4 tunnels up - 4 routes available SiteA<->SiteB - and the routing software will do its thing to send packets over 1 or more of the 4 connections.
What is the possible and preferred way to achieve this?
-
You don't need four servers, you could simply list the alternate second server as an additional "remote" line in the advanced options…
So (just as an example) the client at Site B on WAN would have A1 as the server and a remote line for A2. The OPT1 client on would have A2 as the server and a remote line for A1.
The only downside there is you could end up with both WAN and OPT1 attached to the same WAN at the far end but it should be OK so long as you have it setup like so:
Bind the OpenVPN server instance to the LAN interface, one for each WAN from the other site on its own port. Then forward in, for example:
A1:1194 -> LAN:1194
A2:1194 -> LAN:1194For the second one:
A2:1195 -> LAN:1195
A1:1195 -> LAN:1195Then each client can hop to the other WAN IP and continue functioning.
Don't let OpenVPN do the routing, run OSPF (use the quagga pkg) and have it sort the routing out dynamically.
-
Has anyone tried this yet, and does it work? I'm about to be facing this exact same issue and have been totally stressing about how I'm going to pull it off. So far I've successfully set up exactly one OpenVPN site-to-site tunnel with single WAN at each end, and quite frankly I don't even know how to get started with this multi-WAN configuration.
Don't let OpenVPN do the routing, run OSPF (use the quagga pkg) and have it sort the routing out dynamically.
Should OSPF be installed on A, B, or both?
-
It works, lots of installs use it. OSPF has to be on both ends where the dynamic routing occurs.
-
you could simply list the alternate second server as an additional "remote" line in the advanced options
Do the extras I put in the "advanced configuration" section follow normal OpenVPN config syntax such as:
remote xxx.xxx.xxx.xxx
-or-
remote xxx.xxx.xxx.xxx:1194Also is the OSPF method better than this method:
http://forum.pfsense.org/index.php?topic=32429.0
and if so, why?
-
For doing failover, OSPF is preferred as it will automatically handle the switch to and from the other link as the wans come and go. If you only use multiple remote statements, then it would never switch back away from the one it's connected to.
-
I have got a single client-server OpenVPN working that goes redundantly from a dual-WAN branch office to a dual-WAN main office.
a) At both locations, in System:Advanced, Miscellaneous, turn on 'Allow default gateway switching'. This allows functions internal to the pfSense box to get out even when the default WAN gateway is down. (e.g. checking for updates, package installation, ping from the command line). It also lets the OpenVPN Client find its way out a working gateway (see later). Note: leave WAN as the only default gateway, when WAN fails, the system finds OPT1 and uses it anyway.
b) Define a gateway group to channel your LAN traffic into, sharing or prioritising the gateways, and then add a rule/s to feed LAN traffic into the gateway group - this is not necessary for the OpenVPN, but you probably want this for general internet access by users on the LAN at all sites.
c) Forward matching ranges of ports from WAN and OPT1 to LAN at the Main Office. Forward enough ports for the number of servers you need (e.g. if there are 3 branch offices, then you need 3 servers and 3 ports forwarded.)Now we can setup a server listening on the LAN. It will receive all the incoming connects arriving on WAN and OPT1.
-
d) At the Main Office define an OpenVPN Server listen on the LAN at one of the ports that was forwarded. There is redundancy, if a Main Office link goes down then the OpenVPN Server is still reachable via the other link.
e) In my case, OPT1 has a dynamic IP. So I register with a Dynamic DNS provider and setup a Dynamic DNS entry to keep the OPT! IP address up-to-date out in the internet.
f) At the Branch Office, define an OpenVPN Client on the LAN to connect to the Main Office. It will be routed out the default gateway, which will switch depending on which of WAN and OPT1 is up. So there is redundancy for link failures at the Branch Office. Tell the client about the other path to the Main Office by adding a "remote" option Advanced section. If there are other subnets reachable via the Main Office, the tell the client about them also, with the "route" ooption - see the screenshot.If there are only a few offices, then there is no need to use extra routing packages like OSPF - a couple of "route" options in each client config will tell it what subnets can be reached over the OpenVPN link.
-
Note: with the above config, if the Branch Office WAN goes down, the OpenVPN Client switches to going out OPT1. When WAN comes back again, the default gateway switching puts the default route back to WAN, so the OpenVPN Client switches back to going out WAN.
If the Main Office WAN goes down, then the Branch Office Client first tries to connect to the Main Office WAN but gets no response, so it next tries the Main Office OPT1 and connects. When the Main Office WAN comes up, the OpenVPN link does not fail back to the Main Office WAN.
For my situation, I don't really care which links the logical OpenVPN runs across - I just want it to work. -
Note that "Default Gateway Switching" still has many problems which is why we tend not to recommend it. In particular, it does not work well for PPP interfaces, and it can also end up selecting gateways that really don't lead to the Internet.
On 2.1 you can (once all the code is finished) select a gateway group for the client (and/or server's) "Interface" and it can failover that way.
-
"Default Gateway Switching" allows the internals of the pfSense box to find a fail over path to the internet - e.g. GUI functions that go to the internet to look something up (updates availability, available package list) or ordinary "ping" from the command line.
The code does check that it is not about to switch the default gateway to LAN. But I guess it has to just pick another interface that is up and hope it leads somewhere useful. It seems to work fine on simple configs, like having 1 LAN and 2 WAN.
I tried to find a way to add rules to feed this stuff into a gateway group - e.g. a floating rule feeding anything from 127.0.0.1, WAN Address into a gateway group. But no joy, I guess this local traffic never made it into the pf system to get filtered and policy routed? Or did I miss something in my setups?
Is there a way to get the internal traffic generated in the box to feed into a gateway group and be policy routed? -
Is there a way to get the internal traffic generated in the box to feed into a gateway group and be policy routed?
yes, with quick floating rules
-
Phil's way seems to be working very well or me. I've done various combinations of unplugging gateways at each end, and traffic seems to flow just fine no matter what. It also seems to automatically go back to the main gateway when it is plugged back in. I guess my simple configuration (pretty much identical to Phil's in every way) doesn't do anything to confound the limitations of the default gateway switching option.
On 2.1 you can (once all the code is finished) select a gateway group for the client (and/or server's) "Interface" and it can failover that way.
I'm very much looking forward to that. How long do you think it will be before that's available Jimp?
-
So, maybe it's a mistaken half-newbie intuition but–
Why not port forward the various OpenVPN target ports on all the variously available WAN interfaces to 'localhost', have PF's openvpn listen only on 'localhost' using the various ports for the various client types without regard to route? Be sure the option that routes traffic out the port it came in on is checked, and --- viola? Yes? No?
-
On 2.1 you can (once all the code is finished) select a gateway group for the client (and/or server's) "Interface" and it can failover that way.
So if I'm not mistaken, wouldn't that completely eliminate the need for OSPF? I'm trying really hard to avoid using beta packages in this particular production environment because so much is at stake. Not to mention I'm never quite sure if I've got OSPF configured correctly. It's very new to me.
Why not port forward the various OpenVPN target ports on all the variously available WAN interfaces to 'localhost', have PF's openvpn listen only on 'localhost' using the various ports for the various client types without regard to route? Be sure the option that routes traffic out the port it came in on is checked, and –- viola? Yes? No?
That's basically what Phil's done. Traffic on the OVPN ports was NAT'd to the LAN IP and then listening is done on the LAN interface without regard to route. Although, I'm not sure what "option that routes traffic out the port it came in on" you're referring to.
-
So if I'm not mistaken, wouldn't that completely eliminate the need for OSPF? I'm trying really hard to avoid using beta packages in this particular production environment because so much is at stake. Not to mention I'm never quite sure if I've got OSPF configured correctly. It's very new to me.
In my environment, we have a group of half a dozen main offices that have a full-mesh of OpenVPN links between them all. So from any of these offices, any other office can be reached directly across a single OpenVPN link. If your network is manageably small like that, then you can have the full-mesh of OpenVPN links. By entering local and remote network CIDR on the OpenVPN links you get a complete routing table for your private intranet - no need for OSPF.
Then I add a small branch office. It sends 99% of its data to 1 partner office, so I just make 1 OpenVPN link to that office. But so that it can also reach other offices occasionally, I have to add:
a) "route" commands in the advanced section on the branch office client, telling it that the OpenVPN link is the way to a bunch of other office subnets also.
b) at each other office, saying that the way to "branch office" is via "partner office".
Once this happens more than a couple of times, you start to think it would be nice to run something like OSPF - then the routes to parts of the private network that are not full-meshed can be learnt automatically. It all depends how complicated your private network topology is and how often it changes. -
Hi everyone,
I am just trying to follow the post and I am not too much technical in terms of all this VPN and Failover. In my case, I have the following locations and desired configuration;
Head Office- Lan = 10.60.1.0/24
- DMZ = 192.168.2.0/24
- Wan 1 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
- Wan 2 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
Branch 1 - Lan = 10.60.2.0/24
- Wan 1 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
- Wan 2 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
Branch 2 - (Newly Upcoming) - Lan = 10.60.3.0/24
- Wan 1 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
- Wan 2 = xxx.xxx.xxx.xxx/30 (Data Link without Internet)
Normally, with a Single WAN link HO->Br-1 is connected over VPN and everything is fine and working. I wanted to follow the post to configure (Automatic Failover on MultiSite with Dual Wan over OpenVPN) Automatic Failover for VPN on both end i-e Head Office and Branch plus same for the newly upcoming branches.
My questions are;
- Is that post and configuration workable for more than 1 Client?
- How do I configure the next coming Branches? and What will be the VPN Configuration of Newly coming branches?
Thanking you in advance for any coming help!!
-
Since this post, 2.1-BETA1 has added:
a) the ability to a list of networks in the Local Networks and Remote Networks fields of OpenVPN Server/Client. You do not need to put "route" or "push route" statements in the advanced box any more.
b) OpenVPN Client can use a Gateway Group instead of a particular interface - so no need to bother with the default gateway switching business, or other trickery to convince the client thta it has multiple ways out to the internet.
c) OpenVPN Server can listen on a Gateway Group - I haven't used that, I still port forward to LAN and make my server listen on LAN. But the Gateway Group listen should allow you to have the server listening just on the highest-priority interface in the Gateway Group. That way the server should fail-back when the high-priority WAN interface comes back up after a failure.As I installed more pfSense at remote offices I ended up with lots of OpenVPN servers at my main offices - call them M1 and M2 - every branch office had 2 clients, connecting to M1 and M2. After 5 or 6 of these I decided to change to Peer-to-Peer SSL/TLS - that allows me to have just 1 server at M1 and 1 server at M2. With Client-specific overrides I can tell the server what remote office subnet is at the end of which connection. The routing failover principles apply just the same to the PSK or SSL/TLS security method.
You choose if you want to make a new PSK server to listen for each branch office, or 1 SSL/TLS server to listen for all offices.
This post has more interesting discussion: http://forum.pfsense.org/index.php/topic,59656.0.html
-
Hi Phil,
One more time, need a little help. With the said Guide, I followed the exact procedure. VPN established and net-to-net + DMZ all are working fine. But when I dis-connected WAN to test Failover to OPT-1, it's not establishing connection. OpenVPN Logs on server has following error lines:
Feb 22 03:13:19
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:21
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:23
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:31
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:32
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:43
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedFeb 22 03:13:43
openvpn[18246]: Authenticate/Decrypt packet error: packet HMAC authentication failedand OpenVPN logs on Client has:
Mar 13 10:48:36 openvpn[12597]: Inactivity timeout (–ping-restart), restarting
Mar 13 10:48:36 openvpn[12597]: SIGUSR1[soft,ping-restart] received, process restarting
Mar 13 10:48:38 openvpn[12597]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Mar 13 10:48:38 openvpn[12597]: Re-using pre-shared static key
Mar 13 10:48:38 openvpn[12597]: Preserving previous TUN/TAP instance: ovpnc3
Mar 13 10:48:38 openvpn[12597]: UDPv4 link local (bound): [AF_INET]10.60.3.21
Mar 13 10:48:38 openvpn[12597]: UDPv4 link remote: [AF_INET]192.168.31.34:1194Any Idea/Help appreciated.
-
The server errors are really old and I do not think related to your current problem, so I will ignore them.
It seems that the client is simply not getting through to the server at all.Mar 13 10:48:38 openvpn[12597]: UDPv4 link local (bound): [AF_INET]10.60.3.21 Mar 13 10:48:38 openvpn[12597]: UDPv4 link remote: [AF_INET]192.168.31.34:1194From the above, the client is correctly bound to a LAN IP at Branch2.
For some reason it thinks it should connect to the server on 192.168.31.34 - which is a private IP. Unless you have setup a completely private test environment, then that is not a valid address of the server.
If the client is setup correctly, with an extra "remote" statement in the advanced box, then the client log should cycle around about every minute trying
"UDPv4 link remote: [AF_INET] n.n.n.n:1194" and
"UDPv4 link remote: [AF_INET] m.m.m.m:1194"
where n.n.n.n and m.m.m.m are the 2 WAN IPs at the server end.
Both those server WAN IPs should be port-forwarded to LAN on the server end, where the server should be listening.
What did I misunderstand about your setup?
