OpenBGPD flaps [SOLVED]

Adde

Hi,

I'm trying to get OpenBGPd to work with my pfSense, using the pfSense package on a newly installed 2.0.2 - Release amd64.
The idea is to run BGP through GRE on top of IPsec. IPsec is fine, GRE as well (as far as I can tell), and BGP goes up when started. I recieve and advertise routes as expected, but after exactly one minute, BGP goes down.

This is what I get in the logs:


Feb 5 10:56:09	bgpd[47221]: startup
Feb 5 10:56:09	bgpd[47773]: no kernel support for PF_KEY
Feb 5 10:56:09	bgpd[47773]: no kernel support for PF_KEY
Feb 5 10:56:09	bgpd[47773]: session engine ready
Feb 5 10:56:09	bgpd[47441]: route decision engine ready
Feb 5 10:56:09	bgpd[47441]: RDE reconfigured
Feb 5 10:56:09	bgpd[47773]: listening on 10.11.3.10
Feb 5 10:56:09	bgpd[47773]: SE reconfigured
Feb 5 10:56:09	bgpd[47773]: neighbor 10.11.3.9 (Comment): state change None -> Idle, reason: None
Feb 5 10:56:09	bgpd[47773]: neighbor 10.11.3.9 (Comment): state change Idle -> Connect, reason: Start
Feb 5 10:56:09	bgpd[47773]: neighbor 10.11.3.9 (Comment): state change Connect -> OpenSent, reason: Connection opened
Feb 5 10:56:09	bgpd[47773]: neighbor 10.11.3.9 (Comment): state change OpenSent -> OpenConfirm, reason: OPEN message received
Feb 5 10:56:09	bgpd[47773]: neighbor 10.11.3.9 (Comment): state change OpenConfirm -> Established, reason: KEEPALIVE message received
Feb 5 10:56:10	bgpd[47221]: nexthop 10.11.3.9 now valid: directly connected
Feb 5 11:00:43	bgpd[47221]: nexthop 10.11.3.9 now valid: directly connected
Feb 5 11:05:43	bgpd[47221]: nexthop 10.11.3.9 now valid: directly connected

If I look at the "status" tab in the package, I see that it disconnects after exactly one minute, which also happens to be the keep-alive threshold…?

This is how it looks when connected:


BGP neighbor is 10.11.3.9, remote AS 646**
 Description: Comment
  BGP version 4, remote router-id XX.XX.XX.XX
  BGP state = Established, up for 00:00:05
  Last read 00:00:03, holdtime 180s, keepalive interval 60s
  Neighbor capabilities:
    Multiprotocol extensions: IPv4 unicast
    Route Refresh
    4-byte AS numbers

  Message statistics:
                  Sent       Received  
  Opens                    5          3
  Notifications            0          2
  Updates                  3         15
  Keepalives              5          8
  Route Refresh          0          0
  Total                     13         28

  Update statistics:
                  Sent       Received  
  Updates                  4          7
  Withdraws                0          0

  Local host:            10.11.3.10, Local port:  26055
  Remote host:            10.11.3.9, Remote port:   179

And this when it's idle (which it remains for 4 minutes before connecting again, holdtime I presume):


BGP neighbor is 10.11.3.9, remote AS 646**
 Description: Comment
  BGP version 4, remote router-id XX.XX.XX.XX
  BGP state = Idle, down for 00:00:00
  Last read 00:00:58, holdtime 240s, keepalive interval 80s

  Message statistics:
                  Sent       Received  
  Opens                    5          3
  Notifications            0          2
  Updates                  3         15
  Keepalives              6          8
  Route Refresh          0          0
  Total                     14         28

  Update statistics:
                  Sent       Received  
  Updates                  0          0
  Withdraws                0          0

  Last error: unknown error code

This is the "raw" config from the package:


# This file was created by the package manager.  Do not edit!

AS 646**
fib-update yes
holdtime 180
listen on 10.11.3.10
router-id 10.11.3.10
network 10.83.0.0/16
neighbor 10.11.3.9 {
		descr "Comment"
	remote-as 646**
}
deny from any
deny to any
allow from 10.11.3.9
allow to 10.11.3.9

What am I missing here? Do I need to open some ports in the firewall for the keep-alive to get through, add some setting to the conf, or does the package simply not work on 2.0.2 ?

Adde

I finally found something interesting, in the firewall log I can see this traffic being blocked:


Feb 5 12:03:06	gre0	   10.11.3.10:47556	   10.11.3.9:179	TCP:
Feb 5 12:03:07	gre0	   10.11.3.10:47556	   10.11.3.9:179	TCP:FA
Feb 5 12:03:11	gre0	   10.11.3.10:47556	   10.11.3.9:179	TCP:

But how do I add an allow rule on the GRE interface? It's not available in "rules"…Am I doing it wrong?

Adde

Ok, so I solved it with some assistance from the local "Senior Network Ninjas" and google!

Solution:

Assign an interface to your GRE tunnel, Interfaces -> Assign -> Add new, choose your GRE in dropdown
Enable the interface but keep type as "none"
Add a floating rule allowing anything in both directions (probably tweakeable), enable quick, and set state type to "none"

That's it!