FRR Package restarts with Openvpn
-
Hi All,
I'm running a pair of HA Netgate 7100 appliances with openvpn server running on the WAN. Also running openbgpd.
Version: 2.4.5_p1
In order to upgrade them I needed to change from openbgpd to FRR and so I removed the package and succesfully installed and run FRR on the secondary, establishing BGP sessions to upstream peers and routing correctly.
After I put the primary CARPS in maintenance mode, all the traffic is succesfully running through the secondary ( now running FRR ) but as we know, the primary is still the master when it comes to configuration with any changes on the secondary being overwritten by changes on primary.
So I added a firewall rule on the primary ( part of the work to ensure uninterrupted flow as I removed openbgpd and installed FRR ) that would allow me to work on the primary of the HA pair and this restarted FRR on the secondary, causing a service interruption.
Here are the relevant log snippets:
ug 7 08:28:04 php-fpm 42962 /rc.start_packages: Restarting/Starting all packages.
Aug 7 08:28:03 check_reload_status Starting packages
Aug 7 08:28:03 check_reload_status Reloading filter
Aug 7 08:28:03 php-fpm 42962 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.4.1 - Restarting packages.
Aug 7 08:28:03 php-fpm 42962 /rc.newwanip: rc.newwanip called with empty interface.
Aug 7 08:28:03 php-fpm 42962 /rc.newwanip: rc.newwanip: on (IP address: 192.168.4.1) (interface: []) (real interface: ovpns1).
Aug 7 08:28:03 php-fpm 42962 /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
Aug 7 08:28:02 check_reload_status rc.newwanip starting ovpns1
Aug 7 08:28:02 check_reload_status Reloading filter
Aug 7 08:28:02 php-fpm 96209 OpenVPN PID written: 46609
Aug 7 08:28:02 kernel ovpns1: link state changed to UP
Aug 7 08:28:01 php-fpm 96209 /xmlrpc.php: Resyncing OpenVPN instances.
Aug 7 08:28:01 check_reload_status Reloading filter
Aug 7 08:28:01 check_reload_status Syncing firewall
Aug 7 08:24:43 check_reload_status Reloading filterThe Cisco BGP neighbours report this for every rule change on the HA pair:
sh ip bgp neighbour x.x.x.x
Last reset 00:26:10, due to BGP Notification received of session 1, Administrative ShutdownThe pfsense logs show that this was due to OpenVPN interface on the WAN side.
This also happens when the Openvpn server is adminstratively disabled and it does not matter if the openvpn server is listening on the WAN interface or the WAN CARP.
The ONLY way to prevent this total restart of the routing services is completely remove the Openvpn Server config from the FW's which luckily I am able to do with little impact.
I am posting this as it seems like a signifigant combination that breaks the HA piece that dynamic routing, CARP and sync on pfsense is doing so well. I imagine that openvpn is one of the main reasons for using pfsense out there and so the impact could be big if FRR is getting adopted more and more?
I cannot see this posted anywhere else and I am not sure if this is a bug or a necessary reponse to changes ( based on this chat : https://forum.netgate.com/topic/145653/ffr-restart-on-configuration-changes )