BGP over IPSec resets every time when re-authentication of Phase 1 happens.
We are seeing frequent resets of BGP over IPSec with FRR module , especially when the re-auth of P1 happens after the lifetime expires.
The other end device is Cisco ASA which sees the neighbor up down.
We see below in the sh ip bgp neighbors log on FRR
Connections established 3; dropped 2
Last reset 00:30:09, No AFI/SAFI activated for peer
This PFSense 2.4.5-P1 / FRR - 7-7.3.1 in AWS cloud.
Any known reason and possible solutions ?
We see these logs on System/routing for BGPD
Jan 13 09:38:30 bgpd 74809 bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from x.x.x.121 in vrf default
Jan 13 09:38:30 bgpd 74809 %ADJCHANGE: neighbor x.x.x.121(Unknown) in vrf default Up
Jan 13 09:35:47 bgpd 74809 %ADJCHANGE: neighbor x.x.x.121(Unknown) in vrf default Down BGP Notification send
Jan 13 09:35:47 bgpd 74809 %NOTIFICATION: sent to neighbor x.x.x.121 4/0 (Hold Timer Expired) 0 bytes
Any help ??
If you are using IKEv2 you can disable reauth, which results in a rekeying only when p1 lifetime ends.
Further you can disable "Fast External Failover" in the Advanced Section of the BGP Config.
With both settings in place, you should get stable BGP Sessions over IPSec.
@artes i have a Similar Problem (as stated in my Post) and this configuration lead to BGP "Working" as Long as a Tunnel only reauthenticates. but as soon as a a Tunnel is down for good (because of whatever) all BGP Routes are being deleted and not reinstalled for some reason even though bgp is running..... i have to restart BGP manually for it to reinstall the routes which is realy not how i want to use an automatic routing Protocol.
There was a Bug in 2.4.4_p3, which causes BGP Routes to get stuck in a failure state after reauth of a peer with changed WAN IP-Address. I've implemented these two patches back then and it worked fine for me.
Apply Patches in the order, first Part1 then Part2. Part2 revert some changes from Part1.
I believe these patches are already include in 2.4.5. So if you are already on the latest Version, your issue may have other causes and you should review your configuration or rise a redmine issue.
@artes im on the latest Stable build and i have the same Problem in my Dev Installation on 2.5....
i first thought i simply configured something incorrectly and tortured my Dev VM´s for days and i didnt get rid of it.
it also does not matter what kind of change it is. simply restarting IPSEC results in all BGP routes being deleted and not reinstalled.
Sounds pretty much like the issue I mentioned before, but it should be fixed in the latest release. Without knowing details of your configuration nor having access to any logs I can only do guesses where this comes from.
You could try to decouple BGP from the IPsec VTIs by creating VIPs in a different subnets on each site and configure a static route between these VIPs through the VTIs. Then use the VIPs to establish a Multihop BGP Session. I assume you'll use eBGP so you have to configure EBGP Multi-Hop in the Neighbor configuration. In case of iBGP it doesn't matter.
@artes thats also exactly what im doing since it is not practical to use the VTI Endpoints for BGP.
the logs are not saying anything of interest, i read those over and over... and i would´ve to do realy much anonymisation before posting them.
im Using a CARP Virtual IP for BGP (its solely used for that) why is set on the WAN interface (does not realy matter, tried it on all others).
i don´t quite understand how the Problem itself comes to happen... its not the first time im setting up BGP, just the first time using FRR. i´ll try finding out what the Problem is... if i find a few free moments i´ll setup a new Testing environment in case i messed something up unknowingly and try it again.
it just looks like FRR does not recognise IPSEC Interfaces coming back up or something... its realy annoying but for the moment it works as long as i don´t get a defective VPN tunnel for some reason....
it is not practical to use the VTI Endpoints for BGP
I use them without hitting this issue ;-) But yeah, I already thought about to migrate it to some sort of loopback.
@artes ah well, it is not practical in the rest of my Setup, so it is not for this part of it. its also a bit more flexible but this could also just be my personal preferences.
@oremountain Usually it is the better choice to setup bgp with loopback adapters - I was just to lazy to configure an additional interface and static routes for each IPsec peer.
Maybe you try to reconfigure one peer bgp session directly on a VTI Interface, see if it helps.