-
I have made a test set up with three pfSense virtual machines. Call them test1, test2, test3. Test1 runs an OpenVPN peer to peer SSL/TLS server for test2 to connect as a client. Similarly, test2 has a server for test3 and test3 has a server for test 1. This all works fine, all three are interconnected, each firewall can ping the others' LAN address. The VPNs use the net30 topology, but I also tried the subnet way with the same outcome.
I then install the frr package on all three firewalls. I enable FRR and OSPF on the three firewalls, and add the OpenVPN interfaces to the OSPF interfaces section as well as the local LAN, so that's three interfaces on each firewall. The OSPF status shows that each can see its neighbors. I can ping each firewall's LAN address from the other two. I then stop the VPN between test1 and test2. The OSPF notices and now shows a route from test1 to test2 via the test1 to test3 VPN. However, when I ping test2 from test1 or vice versa, I get no reply. I was hoping that the ping would be relayed from test1 to test2 via the VPNs on test3. Here are some details.test1 LAN 192.168.128.1/24 WAN 192.168.127.216
test2 LAN 192.168.129.1/24 WAN 192.168.127.218
test3 LAN 192.168.130.1/24 WAN 192.168.127.221
test2 to test1 VPN tunnel 10.127.1.0/30
test3 to test2 VPN tunnel 10.127.2.0/30
test1 to test3 VPN tunnel 10.127.3.0/30Here are more details for test1, the other two are set up in a similar fashion.
test1 FRR zebra.conf
##################### DO NOT EDIT THIS FILE! ######################
###################################################################This file was created by an automatic configuration generator.
The contents of this file will be overwritten without warning!
###################################################################
password ZEBRA
log syslogNull Routes
Static Routes
ip route 192.168.128.0/24 em1
Accept Filters
ip prefix-list ACCEPTFILTER permit any
route-map ACCEPTFILTER permit 10
match ip address prefix-list ACCEPTFILTER
ip protocol ospf route-map ACCEPTFILTERtest1 FRR ospfd.conf
##################### DO NOT EDIT THIS FILE! ######################
###################################################################This file was created by an automatic configuration generator.
The contents of this file will be overwritten without warning!
###################################################################
password ZEBRA
log syslog
interface ovpnc2
ip ospf authentication-key ZEBRA
interface ovpns1
ip ospf authentication-key ZEBRA
interface em1
ip ospf authentication-key ZEBRArouter ospf
ospf router-id 192.168.127.216
redistribute connected route-map DNR
redistribute static route-map DNR
passive-interface em1
network 10.127.3.0/30 area 0.0.0.0
network 10.127.1.0/30 area 0.0.0.0
network 192.168.128.0/24 area 0.0.0.0
area 0.0.0.0 authentication
area 0.0.0.0 authentication
area 0.0.0.0 authentication
access-list dnr-list permit any
route-map DNR permit 10
match ip address dnr-listHere's the status on test1 when the test2 to test1 VPN is down.
test1 OSPF General
OSPF Routing Process, Router ID: 192.168.127.216
Supports only single TOS (TOS0) routes
This implementation conforms to RFC2328
RFC1583Compatibility flag is disabled
OpaqueCapability flag is disabled
Initial SPF scheduling delay 0 millisec(s)
Minimum hold time between consecutive SPFs 50 millisec(s)
Maximum hold time between consecutive SPFs 5000 millisec(s)
Hold time multiplier is currently 1
SPF algorithm last executed 2m18s ago
Last SPF duration 195 usecs
SPF timer is inactive
LSA minimum interval 5000 msecs
LSA minimum arrival 1000 msecs
Write Multiplier set to 20
Refresh timer 10 secs
This router is an ASBR (injecting external routing information)
Number of external LSA 8. Checksum Sum 0x0003a1e3
Number of opaque AS LSA 0. Checksum Sum 0x00000000
Number of areas attached to this router: 1
Area ID: 0.0.0.0 (Backbone)
Number of interfaces in this area: Total: 3, Active: 3
Number of fully adjacent neighbors in this area: 1
Area has simple password authentication
SPF algorithm executed 8 times
Number of LSA 3
Number of router LSA 3. Checksum Sum 0x000174d4
Number of network LSA 0. Checksum Sum 0x00000000
Number of summary LSA 0. Checksum Sum 0x00000000
Number of ASBR summary LSA 0. Checksum Sum 0x00000000
Number of NSSA LSA 0. Checksum Sum 0x00000000
Number of opaque link LSA 0. Checksum Sum 0x00000000
Number of opaque area LSA 0. Checksum Sum 0x00000000test1 OSFP Routes
============ OSPF network routing table ============
N 192.168.128.0/24 [100] area: 0.0.0.0
directly attached to em1
N 192.168.129.0/24 [120] area: 0.0.0.0
via 10.127.3.1, ovpnc2
N 192.168.130.0/24 [110] area: 0.0.0.0
via 10.127.3.1, ovpnc2============ OSPF router routing table =============
R 192.168.127.218 [20] area: 0.0.0.0, ASBR
via 10.127.3.1, ovpnc2
R 192.168.127.221 [10] area: 0.0.0.0, ASBR
via 10.127.3.1, ovpnc2============ OSPF external routing table ===========
N E2 10.127.2.1/32 [10/20] tag: 0
via 10.127.3.1, ovpnc2
N E2 10.127.2.2/32 [20/20] tag: 0
via 10.127.3.1, ovpnc2
N E2 10.127.3.2/32 [10/20] tag: 0
via 10.127.3.1, ovpnc2
N E2 192.168.127.0/24 [10/20] tag: 0
via 10.127.3.1, ovpnc2In this last section, it's clear that the OSPF thinks it can get to both 192.168.130.0/24 and 192.168.129.0/24 via ovpnc2 which is the VPN to test3.
I'm a newbie to OSFP, so I expect I've done something daft, any pointers gratefully received.
Thanks for reading. -
Update. I think I fixed it. But I don't really understand it properly. I'd appreciate it if anyone can explain what's happening!
I had to add some rules to Outbound NAT. On each pfSense I added a rule for all OpenVPN tunnel IP addresses (10.127.0.0/16 in my case) sources on the WAN to translate to the WAN interface address. This got the ping working via the third pfSenses during VPN outages. I then also added a rule for all IP addresses sourced from the LAN on the OpenVPN interface to translate to the OpenVPN interface address. In my example above, this meant NATing 192.168.128.0/24 on test1, 192.168.129.0/24 on test 2 etc. Now it works. If I set a ping going from one pfSense's lan to another, and I stop the VPN between the two, the pings get re-routed via the third pfSense. A few pings get lost while it's swapping, but this is what I wanted! Back of the net!
Anyway, as I said, if anyone can explain what's happening here, that would be great. I won't mark this [SOLVED] just yet until I'm sure I've done this correctly.
p.s. Don't you love it when things start to work just before pub o'clock!
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.