CARP makes OpenBGPD to ignore 'set nexthop X' configured parameter.

correajl

I don't know if this is a CARP or OpenBGPD package issue. By the way here is the debugging.

I'm trying to set a scenario with 2 pfsense (master = A.A.A.1 and backup = A.A.A.2) with CARP (A.A.A.3), each of them opening a BGP session with the ISP router. At these sessions my network prefix (N.N.N.0/24) is announced with the paramenter 'set nexthop A.A.A.3'.

Some discussions about this problem are:

https://forum.netgate.com/topic/41813/openbgp-and-carp
https://forum.netgate.com/topic/51849/openbgp-with-carp-nexthop-carp-ip-carp
https://www.mail-archive.com/misc@openbsd.org/msg25294.html
https://www.reddit.com/r/networking/comments/2maskw/bgp_carp_with_pfsense/

Without any CARP configured each pfsense establishes the BGP session and announces 'N.N.N.0/24 set nexthop A.A.A.3'. So, at the ISP router we can see

B>* N.N.N.0/24 [20/0] via A.A.A.3, enp0s8, 01:59:58

When we activate the CARP address simply the parameter 'set nexthop A.A.A.3' is ignored AT THE MASTER. So, with HA established and CARP online, at the ISP router we can see:

B>* N.N.N.0/24 [20/0] via A.A.A.1, enp0s8, 01:59:58 (A.A.A.1 is the interface address of pfsense master).

Dumping the traffic at ISP router we can see that the NEXT_HOP value in the BGP UPDATE message sent by the master pfsense is wrong:

Transmission Control Protocol, Src Port: 18397, Dst Port: 179, Seq: 73, Ack: 98, Len: 74
Border Gateway Protocol - UPDATE Message
Marker: ffffffffffffffffffffffffffffffff
Length: 51
Type: UPDATE Message (2)
Withdrawn Routes Length: 0
Total Path Attribute Length: 20
Path attributes
Path Attribute - ORIGIN: IGP
Path Attribute - AS_PATH: 11097
Path Attribute - NEXT_HOP: A.A.A.1
Network Layer Reachability Information (NLRI)
N.N.N.0/24
NLRI prefix length: 24
NLRI prefix: N.N.N.0

Dumping the same UPDATE message from backup pfsense the value is correct (A.A.A.3).

This value is configured in Services -> OpenBGPD -> Settings -> Networks, with 'N.N.N.0/24 set nexthop A.A.A.3'.

Why without CARP the announce goes right, with the correct parameter set, and when the pfsense holds a CARP address (is master) the announce goes wrong, ignoring the configured parameter? Is this a bug? An issue with kernel interfaces? Or something with OpenBGPD package? How we can know where is the problem?

Thanks you!

correajl

I think I found the trick.

https://man.openbsd.org/bgpd.conf#nexthop_3

"On EBGP multihop session no-modify has to be set to force the nexthop to address".

CARP somehow resembles a multihop scenario. So, my configuration now:

Services -> OpenBGPD -> Settings -> Networks with 'N.N.N.0/24 set nexthop A.A.A.3'.

Services -> OpenBGPD -> Neighbors -> (choose the neighbor) -> Add the parameter 'set nexthop no-modify'.

From then on the announces will be generated setting nexthop with de CARP address an this address will be applyed at ISP router.

Tks!

Smith 0

Hello,
I'm experiencing the same config and difficulties. Two PFsense (2.4.5p1) with CARP and two ISPs.

Have to annonce a network, with the "set nexthop" to external CARP IP.
But is it correct with openbgp to have the same network announced twice but with a different set nexthop ?

Eg (with RFC 1918 addresses) :

10.10.10.0/24 set nexthop 192.168.1.1
10.10.10.0/24 set nexthop 172.16.1.1

(or here https://forum.netgate.com/topic/51849/openbgp-with-carp-nexthop-carp-ip-carp )

Or will openbgp drop the second network announce ?

Thanks.