Site to Site Dual WAN/Dual OVPN all load balanced

xternal · Sep 17, 2012, 2:13 AM

Hi guys, I have done lots of searching and used the info in these forums to almost fully complete my setup. I just need a bit of clarification on OSPF setup.
I have 2 sites which have 2 ADSL connections each. Currently both sites have Dual WAN setup with load balancing setup and working fine.

I have also managed to create 2 working OVPN Site to Site connections with no load balance/failover. Here is a diagram of the setup

SITE A LAN –- PFSENSE --- WAN/OPVN1.A ------------------------ WAN/OVPN1.B --- PFSENSE --- LAN SITE B
|--- WAN/OPVN2.A ------------------------WAN/OVPN2.B --

How do I go about setting up OSPF to failover/loadbalance the VPN connections? I am having a few problems figuring out what the settings mean/finding a good tutorial on OSPF

I am wondering what to put in global settings for router id, area etc?
Also in another post on the forum it mentions you need to bind the OPVN connection to a new interface because it doesnt show up in the drop down in the OSPF "interface settings", however on my setup the ovpn connections are showing up (I am wondering if this was an old post and you no longer need to do this step?)

Cheers

heper · Sep 17, 2012, 3:55 PM

the use of openospf is deprecated, as in not recommanded. you'd better remove it and use quagga ospf, it has less issues.

also with quagga you can select the ovpn tunnel as interface without assigning a seperate one.

dhatz · Sep 17, 2012, 8:31 PM

@heper:

ovpn tunnel as interface without assigning a seperate one.

Is there any functional difference between a ovpn* interface (ifconfig) and a pfSense webGUI OPTx interface ?

And would it be advisable (best practice) to assign a webGUI OPTx interface to onvpn* regardless ?

jimp · Sep 17, 2012, 9:00 PM

Yes, if you assign the interface, it will get a gateway, it also can need some special handling (e.g. make sure you don't delete the VPN without first deleting the assigned interface).

If you need/want the behavior of the assigned interface, it's available. However it's not required for most cases.

In the case of OpenOSPFd it was only required because the package wasn't smart enough to list the OpenVPN interfaces unless they were assigned, it didn't do anything else special in that case.

dhatz · Sep 17, 2012, 9:35 PM

Thanks for the clarification jimp.

Btw, does pfSense OpenVPN now support OSPF load balancing across equal cost path ?

jimp · Sep 17, 2012, 9:51 PM

Not that I'm aware of. Quagga is the recommended OSPF daemon at the moment, I don't know if (a) it's capable of equal cost OSPF, and (b) if it is, if it actually works on FreeBSD, and if it does, if it works with pfSense (it might require RADIX_MPATH in the kernel)

xternal · Sep 18, 2012, 12:58 AM

Hey,
I am using Quagga, I was just after some clarification on the router id and area. I have set the area to 0.0.0.0 and the router id's to the local lan e.g. 192.168.1.1 on site A and 192.168.2.1 on site B
I am testing with just one OPVN link at the moment, and it seems to find it, but doesnt do any routing when i remove the "remote network" from the ovpn client/server setup.

I am using 10.0.10.1/30 for the tunnel network

xternal · Sep 18, 2012, 1:44 AM

Here is the OSPF status from the client machine at site B, any help appreciated

 OSPF Routing Process, Router ID: 192.168.2.1
 Supports only single TOS (TOS0) routes
 This implementation conforms to RFC2328
 RFC1583Compatibility flag is disabled
 OpaqueCapability flag is disabled
 Initial SPF scheduling delay 200 millisec(s)
 Minimum hold time between consecutive SPFs 1000 millisec(s)
 Maximum hold time between consecutive SPFs 10000 millisec(s)
 Hold time multiplier is currently 1
 SPF algorithm last executed 1m25s ago
 SPF timer is inactive
 Refresh timer 10 secs
 Number of external LSA 0\. Checksum Sum 0x00000000
 Number of opaque AS LSA 0\. Checksum Sum 0x00000000
 Number of areas attached to this router: 1

 Area ID: 0.0.0.0 (Backbone)
   Number of interfaces in this area: Total: 1, Active: 1
   Number of fully adjacent neighbors in this area: 1
   Area has no authentication
   SPF algorithm executed 2 times
   Number of LSA 2
   Number of router LSA 2\. Checksum Sum 0x0001d02b
   Number of network LSA 0\. Checksum Sum 0x00000000
   Number of summary LSA 0\. Checksum Sum 0x00000000
   Number of ASBR summary LSA 0\. Checksum Sum 0x00000000
   Number of NSSA LSA 0\. Checksum Sum 0x00000000
   Number of opaque link LSA 0\. Checksum Sum 0x00000000

eighbor ID Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
192.168.1.1       1 Full/DROther      34.078s 10.0.10.1       ovpnc1:10.0.10.2         0     0     0

       OSPF Router with ID (192.168.2.1)

                Router Link States (Area 0.0.0.0)

Link ID         ADV Router      Age  Seq#       CkSum  Link count
192.168.1.1     192.168.1.1       86 0x80000046 0xf686 2
192.168.2.1     192.168.2.1       85 0x80000043 0xd9a5 2

xternal · Sep 18, 2012, 3:28 AM

Hi Guys, I think I am getting close, I didn't have "redistribute connected subnets" at each end ticked.

To clean up my Quagga routing table I have added the ADSL gateways to the exlcusion list when passing subnets. This way OSPF is only passing the local lan subnets. Will this cause an issue?

To get it to work, I now have a Lan firewall rule which when accessing the 192.168.0.0/16 subnets, it just uses the * gateway. So hopefully it is using Quagga to load balance across the OPVN links.
Then I have all other traffic from the Lan going across the WANBalancer gateway I setup (which was working nicely before i started with the opvn connections).
Is this correct?

EDIT: ARGH! Now I can only get one VPN connection to connect at a time. If i have both VPN connections enabled on the server, the last one to enable just shows this:

VPN on WAN UDP:1194 0 See Note Below No Management Daemon 0 0

And the logs contain:

Sep 18 14:24:50 openvpn[12660]: /sbin/ifconfig ovpns1 10.0.10.1 10.0.10.2 mtu 1500 netmask 255.255.255.255 up
Sep 18 14:24:50 openvpn[12660]: FreeBSD ifconfig failed: external program exited with error status: 1
Sep 18 14:24:50 openvpn[12660]: Exiting

The VPN connections both work, the other one just has to be disabled. Any ideas :) ??? :(

Edit 2: Upgraded to the latest snapshot, seems to be working :) Now to test and make sure load balancing is working

Edit 3: Ok it isn't load balancing, do I need to setup a gateway group with the VPN interfaces and have a firewall rule which directs traffic through that?

xternal · Sep 21, 2012, 12:19 AM

Bumpy :)
Still cannot get it to load balance the vpn connections…

heper · Sep 21, 2012, 6:31 AM

afaik the freebsd kernel currently does not support loadbalancing when using ospf (don't know the details).

what quagga will do is provide failover.

dhatz · Sep 21, 2012, 1:19 PM

@heper:

afaik the freebsd kernel currently does not support loadbalancing when using ospf (don't know the details).
what quagga will do is provide failover.

Check previous posts in this thread (mine and jimp's)

Crisao23 · Sep 21, 2012, 11:51 PM

Sorry for "hijacking the thread" but are there any tutorials on site to site openvpn using ospf ?

Thanks a lot !

heper · Sep 22, 2012, 10:15 AM

don't know of any full blown tutorial, but its fairly easy. some info bout ospf can be found here: http://forum.pfsense.org/index.php/topic,37084.0.html

First thing you should do is get you tunnel(s) up and running and routing properly without ospf.
Then you install quagga-ospf on both ends
Then you configure:

set a master password
pick an available area. if this is your (company) network, anything will be available
personally i like to specify what routes to distribute and don't use "Redistribute connected subnets"
on the interface settings specify where you want to bind ospf to
hit save
repeat on other end

After both end run ospf on the correctly, check their status screens
it should have found neighbours, and routes to distribute and lots of incomprehensible garbage you shouldn't worry about :D
verify the "Quagga OSPF Routes". Are those the subnets you wish to distribute ?
If yes, remove all routes from your Openvpn configuration (ie remote network/ local network/ pushed routes)
I keep the tunnel network fields in the openvpn config, incase the ospf fails and i need to reach the other side quickly, but i don't believe this to be necessary

and finally, verify you have other ways to reach to other end tunnel in case something goes wrong. Hit Save and if needed restart the services.

Crisao23 · Sep 22, 2012, 7:41 PM

Thanks a lot heper, I'll check it out.

eytanes · Nov 23, 2012, 3:33 PM

heper,
I tried setting up ospf as you described but for some reason the Quagga UI makes some changes that I didnt specify while ignoring other that I do.

I have two openvpn tunnels. The tunnel networks are conn1: 10.1.1.12/30 and conn2: 10.1.1.16/30.
The quagga will sometimes add those two networks as networks to distribute without me specifying them. So if conn1 is up while conn2 is down ospf will add a route to 10.1.1.18 via con1 and it will prevent the openvpn connection conn2 from coming online (openvpn client fails to add the route).

Another issue i was having was being able to distribute the openvpn 'Remote Users' network via quagga. I have a 'Remote Users' openvpn server with network 10.2.2.0/24.
I tried to add the network to be distributed via ospf but it only distributes the route 10.2.2.2/32

Oddly enough both these issues dont' appear when i use openospfd.

Any help would be greatly appreciated.
-E