OpenVPN and OSPF multi wan links - spokes can talk to hub not to one another

reqlez

Hi.

Basically I have 3 routers. 192.168.7.1 is the HUB, 192.168.3.1 and 192.168.5.1 are spokes.

OSPF is running and everything working great except that spokes cannot talk to one another. Spoke to hub and hub to spoke pings are no problem. Here is some info from one of the spokes ( .3.1 ). Any ideas ?

Quagga OSPF Routes
============ OSPF network routing table ============
N 192.168.3.0/24 [10] area: 0.0.0.0
directly attached to igb1
N 192.168.5.0/24 [20] area: 0.0.0.0
via 192.168.101.3, ovpnc2
N 192.168.7.0/24 [20] area: 0.0.0.0
via 192.168.101.1, ovpnc2
N 192.168.101.0/24 [10] area: 0.0.0.0
directly attached to ovpnc2
N 192.168.102.0/24 [30] area: 0.0.0.0
via 192.168.101.1, ovpnc2
N 192.168.103.0/24 [30] area: 0.0.0.0
directly attached to ovpnc3
N 192.168.104.0/24 [50] area: 0.0.0.0
via 192.168.101.1, ovpnc2

============ OSPF router routing table =============

============ OSPF external routing table ===========

Quagga Zebra Routes
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, A - Babel,
> - selected route, * - FIB route

K>* 0.0.0.0/0 via 174.3.212.1, igb0
C>* 127.0.0.0/8 is directly connected, lo0
C>* 174.3.212.0/22 is directly connected, igb0
O 192.168.3.0/24 [110/10] is directly connected, igb1, 00:09:38
C>* 192.168.3.0/24 is directly connected, igb1
O>* 192.168.5.0/24 [110/20] via 192.168.101.3, ovpnc2, 00:09:07
O>* 192.168.7.0/24 [110/20] via 192.168.101.1, ovpnc2, 00:09:34
K>* 192.168.39.0/24 via 192.168.39.2, ovpns1
C>* 192.168.39.2/32 is directly connected, ovpns1
O 192.168.101.0/24 [110/10] is directly connected, ovpnc2 inactive, 00:09:38
C>* 192.168.101.0/24 is directly connected, ovpnc2
O>* 192.168.102.0/24 [110/30] via 192.168.101.1, ovpnc2, 00:09:34
O 192.168.103.0/24 [110/30] is directly connected, ovpnc3 inactive, 00:09:38
C>* 192.168.103.0/24 is directly connected, ovpnc3
O>* 192.168.104.0/24 [110/50] via 192.168.101.1, ovpnc2, 00:09:34

heper

need more info.
traceroute can give you an indication where it goes wrong.

could be firewall rules or something else going on at the HUB.
can a client from spoke-LAN ping hub-tunnelip-of-different-spoke ?

reqlez

okay traceroute is not giving me much. But i can confirm that i cannot ping the tunnel IPs of spokes from other spokes. I can ping tunnel IP of spokes from HUB just fine.

From one of the spokes, it says: 192.168.101.0/24 is directly connected, ovpnc2 ( thats the tunnel network, but spokes share the same tunnel network subnet since the hub assign .2 .3 etc when more spokes connect ) so i'm assuming that it should go to the HUB and the hub should figure out where to send the traffic.

heper

i'm not sure, but are you using the "automatic' redistribute-connected/default/static/kernel checkboxes ?
its generally a bad idea to advertise your tunnel-networks because it can be messy. ( try by adding the tunnelnetwork(s) to the 'subnets to route' fields and check the 'disable acceptance&disable redistribution' buttons )

personally i don't use the automatic redistribution things, i like to maintain control on what routes i advertise

if above doesn't help, try by manually inputting some 'subnets to route' in the textfields at the bottom of the global settings page. see if you can get that working. (uncheck the automatic redistribution checkboxes)
if that works, then it means quagga is working and the firewall rules are working as intended ==> then you can try to figure out why the checkboxes get the wrong outcome

reqlez

hi.

So how i have the OSPF Set-up is that i don't have anything checked under the general tab, and i have "Accept filter" checked on the OVPN links and "interface is passive" on the LAN.

And i don't really have an idea how to use the manual feature on the bottom of the general tab, since I have never used it before. Sorry but i'm a complete OSPF noob.

But i did put all the tunnel networks and put area 0.0.0.0 on the bottom of the page of the HUB and spoke routers as you instructed, and nothing really changed… maybe because I already had "Accep filter" checked on every open vpn interface under interfaces ?

heper

so you added 192.168.101.0/24 on ALL your pfsense's with disable acceptance & disable redist checked?
and that specific route still shows up in your quagga status page ? odd

reqlez

Well … like I said, all spokes share the subnets for the links... so they are directly connected, but now it says "Inactive" maybe that is the change that happened... this is from HUB router:

Just FYI, the way i'm testing is that i'm pinging from a spoke to another spoke but i'm pinging the router LAN IP since i have no computers plugged in to those routers yet ...

K>* 0.0.0.0/0 via 174.3.32.1, igb0
C>* 75.158.60.0/24 is directly connected, igb2
C>* 127.0.0.0/8 is directly connected, lo0
C>* 174.3.32.0/22 is directly connected, igb0
O>* 192.168.3.0/24 [110/20] via 192.168.101.2, ovpns2, 16:03:54
O>* 192.168.5.0/24 [110/20] via 192.168.101.3, ovpns2, 16:04:04
O 192.168.7.0/24 [110/10] is directly connected, igb1, 16:09:04
C>* 192.168.7.0/24 is directly connected, igb1
K>* 192.168.19.0/24 via 192.168.19.2, ovpns1
C>* 192.168.19.2/32 is directly connected, ovpns1
O 192.168.101.0/24 [110/10] is directly connected, ovpns2 inactive, 16:09:04
C>* 192.168.101.0/24 is directly connected, ovpns2
O 192.168.102.0/24 [110/20] is directly connected, ovpns4 inactive, 16:09:04
C>* 192.168.102.0/24 is directly connected, ovpns4
O 192.168.103.0/24 [110/30] is directly connected, ovpns3 inactive, 16:09:04
C>* 192.168.103.0/24 is directly connected, ovpns3
O 192.168.104.0/24 [110/40] is directly connected, ovpns5 inactive, 16:09:04
C>* 192.168.104.0/24 is directly connected, ovpns5

and this is from one of the spokes:

K>* 0.0.0.0/0 via 174.3.212.1, igb0
C>* 127.0.0.0/8 is directly connected, lo0
C>* 174.3.212.0/22 is directly connected, igb0
O 192.168.3.0/24 [110/10] is directly connected, igb1, 16:07:57
C>* 192.168.3.0/24 is directly connected, igb1
O>* 192.168.5.0/24 [110/20] via 192.168.101.3, ovpnc2, 16:07:53
O>* 192.168.7.0/24 [110/20] via 192.168.101.1, ovpnc2, 16:07:53
K>* 192.168.39.0/24 via 192.168.39.2, ovpns1
C>* 192.168.39.2/32 is directly connected, ovpns1
O 192.168.101.0/24 [110/10] is directly connected, ovpnc2 inactive, 16:07:57
C>* 192.168.101.0/24 is directly connected, ovpnc2
O 192.168.102.0/24 [110/30] via 192.168.101.1, ovpnc2 inactive, 16:07:53
O 192.168.103.0/24 [110/30] is directly connected, ovpnc3 inactive, 16:07:50
C>* 192.168.103.0/24 is directly connected, ovpnc3
O 192.168.104.0/24 [110/50] via 192.168.101.1, ovpnc2 inactive, 16:07:53

heper

inactive means its offline … or lost connection ...

i don't get why the tunnel networks are getting redistributed ... they certainly shouldn't and its causing you issue's.

draw up a layout of your networks and include the used subnets (lan/wan/tunnel/...).
then post the quagga config & || screenshots of your quagga config.

reqlez

will do…

reqlez

okay I got the traffic working from spoke by changing the Mode on the OpenVPN server links to "Remote Access TLS" instead of "Peer to Peer TLS" and enabling the "inter connect" option in OpenVPN. This seems like a dirty way of doing it but it works .. not sure ( somebody suggested that ).

Now the only issue is that I think the OSPF is confused as to which router is the DR … the hub should be always the DR in my opinion ?

Look at this:

HUB: 192.168.7.1

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.3.1 1 Full/DR 18.569s 192.168.101.2 ovpns2:192.168.101.1 0 0 0
192.168.5.1 1 Full/DROther 18.706s 192.168.101.3 ovpns2:192.168.101.1 0 0 0
192.168.3.1 1 Full/DROther 18.582s 192.168.103.2 ovpns3:192.168.103.1 0 0 0
192.168.5.1 1 Full/Backup 18.721s 192.168.103.3 ovpns3:192.168.103.1 0 0 0

Spoke 1: 192.168.3.1

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.7.1 1 Full/Backup 18.995s 192.168.101.1 ovpnc2:192.168.101.2 0 0 0
192.168.5.1 1 Full/DROther 15.196s 192.168.101.3 ovpnc2:192.168.101.2 0 0 0
192.168.7.1 1 Full/DR 19.009s 192.168.103.1 ovpnc3:192.168.103.2 0 0 0
192.168.5.1 1 Full/Backup 15.219s 192.168.103.3 ovpnc3:192.168.103.2 0 0 0

Spoke 2: 192.168.5.1

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.7.1 1 Full/Backup 19.738s 192.168.101.1 ovpnc2:192.168.101.3 0 0 0
192.168.3.1 1 Full/DR 15.773s 192.168.101.2 ovpnc2:192.168.101.3 0 0 0
192.168.7.1 1 Full/DR 19.752s 192.168.103.1 ovpnc3:192.168.103.3 0 0 0
192.168.3.1 1 Full/DROther 15.801s 192.168.103.2 ovpnc3:192.168.103.3 0 0 0

Is there a way to force the DR to be on HUB ? Also it seems now that the spokes can communicate with each other its trying to create some kind of mesh or something ... but its not really a mesh since once the HUB is dead all of them area dead ... maybe i need to put the entries not to distribute routers back on there...

reqlez

okay … i'm convinced that BRIDGING the spokes inside of openvpn tunnel is not the way to do it ...

How it should work is that a Spoke 1 LAN ( 192.168.3.1 ) wants to talk to Spoke 2 LAN ( 192.168.5.1 ) there should be an entry that say ... if you want to talk to 192.168.5.1 you have to go thru the HUB LAN ( 192.168.7.1 ) and there should be another entry that says if you want to reach the HUB LAN, you have to go thru this OVPN interface ( 192.168.101.1 ).

If it can't work like that because of a limitation of networking or OSPF or whatever ... i rather not try at all ... I don't need a mesh in my network thats sooner than later going to break things.

This is the main problem I think ...
O>* 192.168.5.0/24 [110/20] via 192.168.101.3, ovpnc2, 00:27:16
O>* 192.168.7.0/24 [110/20] via 192.168.101.1, ovpnc2, 00:27:24

it should say via 192.168.101.1 not 192.168.101.3