Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    OpenBGP Does not seem to be publishing routes from neighbor properly

    Routing and Multi WAN
    1
    1
    1635
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      relfie last edited by

      Good afternoon,

      I am a BGP newbie, so please forgive me if this is the wrong forum.

      We are using pfSense 2.1.4 and OpenBGPD package 0.9.2.  We are trying to implement Amazon AWS direct connect.  I believe I have the bgpd.conf correct as I am seeing the following in routing.log, also bgpd status shows messages being exchanged.

      === snip routing.log ===
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: startup
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: rereading config
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: route decision engine ready
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: session engine ready
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: RDE reconfigured
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: listening on 192.168.55.1
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: SE reconfigured
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change None -> Idle, reason: None
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.3 now valid: directly connected
      Aug  7 17:16:01 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Idle -> Connect, reason: Start
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Connect -> OpenSent, reason: Connection opened
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenSent -> OpenConfirm, reason: OPEN message received
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenConfirm -> Established, reason: KEEPALIVE message received
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5783]: Rib Loc-RIB: neighbor 192.168.55.5 (AWS-DC MER Peer) AS9059: update 172.16.24.0/21 via 192.168.55.5
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.5 now valid: via 192.168.55.1

      However when a server on a local subnet in our AS tries to ping a server in the remote AS the traffic gets routed to the WAN interface and not over the BGP nexthop.

      Here is our BGPD config :

      === snip ===

      This file was created by the package manager.  Do not edit!

      ########

      Our AS

      ########

      AS 65458
      fib-update yes
      listen on 192.168.55.1
      log updates
      network 192.168.48.0/25 set nexthop 192.168.55.3
      network 192.168.48.128/25 set nexthop 192.168.55.3
      network 192.168.49.0/25 set nexthop 192.168.55.3

      ########

      Peer Groups

      ########
      group "AWSDC" {
      remote-as 9059
      neighbor 192.168.55.5 {
                                            descr "AWS-DC MER Peer"
                                            tcp md5sig password 8e484c715b2be0e50d576bc0bb0c29d4
                                            announce all 
                                            local-address 192.168.55.3
      }
      }
      deny from any
      deny to any
      allow from 192.168.55.5
      allow to 192.168.55.5

      ..here is the BGPD Status

      Summary:
      Neighbor                  AS    MsgRcvd    MsgSent  OutQ Up/Down  State/PrfRcvd
      AWS-DC MER Peer          9059        13        12    0 00:04:09      1

      Interfaces:
      Interface      Nexthop state  Flags          Link state
      opt6_vip249    ok            UP            CARP, master
      igb2_vlan300  ok            UP            active, 1000 MBit/s
      ovpns1        ok            UP            active
      wan_vip250    ok            UP            CARP, master
      opt4_vip251    ok            UP            CARP, master
      opt3_vip252    ok            UP            CARP, master
      opt2_vip253    ok            UP            CARP, master
      opt1_vip254    ok            UP            CARP, master
      wan_vip255    ok            UP            CARP, master
      lagg0_vlan50  ok            UP            active, 10 MBit/s
      lagg0_vlan30  ok            UP            active, 10 MBit/s
      lagg0_vlan20  ok            UP            active, 10 MBit/s
      lagg0_vlan10  ok            UP            active, 10 MBit/s
      lagg0          ok            UP            Ethernet, active, 1000 MBit/s
      pflog0        invalid                      invalid
      lo0            ok            UP            invalid
      pfsync0        ok            UP            invalid
      enc0          ok            UP            invalid
      igb7          ok            UP            active, 1000 MBit/s
      igb6          ok            UP            Ethernet, active, 1000 MBit/s
      igb5          ok            UP            active, 1000 MBit/s
      igb4          invalid                      Ethernet, invalid, 10 MBit/s
      igb3          ok            UP            active, 1000 MBit/s
      igb2          ok            UP            Ethernet, active, 1000 MBit/s
      igb1          ok            UP            active, 1000 MBit/s
      igb0          ok            UP            Ethernet, active, 1000 MBit/s

      Routing:
      flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
      origin: i = IGP, e = EGP, ? = Incomplete

      flags destination          gateway          lpref  med aspath origin
      >    172.16.24.0/21      192.168.55.5      100    0 9059 i
      AI      >  192.168.48.0/25      192.168.55.3      100    0 i
      AI*>  192.168.48.128/25    192.168.55.3      100    0 i
      AI*>  192.168.49.0/25      192.168.55.3      100    0 i

      Forwarding:
      flags: * = valid, B = BGP, C = Connected, S = Static
            N = BGP Nexthop reachable via this route
            r = reject route, b = blackhole route

      flags prio destination          gateway
      *S      48 0.0.0.0/0            81.27.95.81
      *S      48 10.101.1.0/25        192.168.48.1
      *S      48 10.101.1.128/25      192.168.48.129
      *S      48 10.101.2.0/25        192.168.49.1
      *S      48 10.101.5.0/25        192.168.48.1
      *S      48 10.101.5.128/25      192.168.48.129
      *S      48 10.101.6.0/25        192.168.49.1
      *      48 81.27.95.80/28      81.27.95.84
      *C      48 81.27.95.84/32      link#11
      *C      48 81.27.95.93/32      link#23
      *C      48 81.27.95.94/32      link#18
      *C      48 84.20.199.91/32      link#1
      *C      0 127.0.0.0/8          link#0
      *C      48 127.0.0.1/32        link#11
      *B      48 172.16.24.0/21      192.168.55.1
      *S      48 192.168.44.0/23      192.168.48.1
      *S      48 192.168.46.0/24      192.168.48.1
      *C      48 192.168.48.0/25      link#14
      *C      48 192.168.48.118/32    link#11
      *C      48 192.168.48.126/32    link#19
      *C      48 192.168.48.128/25    link#15
      *C      48 192.168.48.246/32    link#11
      *C      48 192.168.48.254/32    link#20
      *C      48 192.168.49.0/25      link#16
      *C      48 192.168.49.118/32    link#11
      *C      48 192.168.49.126/32    link#21
      *C      48 192.168.49.128/25    link#17
      *C      48 192.168.49.246/32    link#11
      *C      48 192.168.49.254/32    link#22
      *S      48 192.168.50.0/24      192.168.48.1

      • N    48 192.168.55.0/29      192.168.55.1
        *C      48 192.168.55.1/32      link#11
        *CN    48 192.168.55.3/32      link#26
        *S      48 192.168.90.0/24      192.168.48.1
        *S      48 192.168.200.0/24    192.168.200.2
        *C      48 192.168.200.1/32    link#11
        *C      48 192.168.200.2/32    link#24
        *C      48 192.168.226.0/27    link#7
        *C      48 192.168.226.2/32    link#11
        *C      0 ::1/128              link#0
        *C      48 ::1/128              link#11
        *C      48 fe80:1::/64          link#1
        *C      48 fe80:1::225:90ff:feea:3074/128 link#11
        *C      48 fe80:2::/64          link#2
        *C      48 fe80:2::225:90ff:feea:3075/128 link#11
        *C      48 fe80:3::/64          link#3
        *C      48 fe80:3::225:90ff:feea:3076/128 link#11
        *C      48 fe80:4::/64          link#4
        *C      48 fe80:4::225:90ff:feea:3077/128 link#11
        *C      48 fe80:6::/64          link#6
        *C      48 fe80:6::225:90ff:fef3:8fc7/128 link#11
        *C      48 fe80:7::/64          link#7
        *C      48 fe80:7::225:90ff:fef3:8fc8/128 link#11
        *C      48 fe80:8::/64          link#8
        *C      48 fe80:8::225:90ff:fef3:8fc9/128 link#11
        *C      48 fe80🅱:/64          link#11
        *C      48 fe80🅱:1/128        link#11
        *C      48 fe80:d::/64          link#13
        *C      48 fe80:d::225:90ff:feea:3075/128 link#11
        *C      48 fe80:e::/64          link#14
        *C      48 fe80:e::225:90ff:feea:3074/128 link#11
        *C      48 fe80:f::/64          link#15
        *C      48 fe80:f::225:90ff:feea:3074/128 link#11
        *C      48 fe80:10::/64        link#16
        *C      48 fe80:10::225:90ff:feea:3074/128 link#11
        *C      48 fe80:11::/64        link#17
        *C      48 fe80:11::225:90ff:feea:3074/128 link#11
        *C      48 fe80:18::225:90ff:feea:3074/128 link#11
        *C      48 fe80:19::/64        link#25
        *C      48 fe80:19::225:90ff:feea:3074/128 link#11
        *      48 ff01:1::/32          fe80:1::225:90ff:feea:3074
        *      48 ff01:2::/32          fe80:2::225:90ff:feea:3075
        *      48 ff01:3::/32          fe80:3::225:90ff:feea:3076
        *      48 ff01:4::/32          fe80:4::225:90ff:feea:3077
        *      48 ff01:6::/32          fe80:6::225:90ff:fef3:8fc7
        *      48 ff01:7::/32          fe80:7::225:90ff:fef3:8fc8
        *      48 ff01:8::/32          fe80:8::225:90ff:fef3:8fc9
        *      48 ff01🅱:/32          ::1
        *      48 ff01:d::/32          fe80:d::225:90ff:feea:3075
        *      48 ff01:e::/32          fe80:e::225:90ff:feea:3074
        *      48 ff01:f::/32          fe80:f::225:90ff:feea:3074
        *      48 ff01:10::/32        fe80:10::225:90ff:feea:3074
        *      48 ff01:11::/32        fe80:11::225:90ff:feea:3074
        *      48 ff01:18::/32        fe80:18::225:90ff:feea:3074
        *      48 ff01:19::/32        fe80:19::225:90ff:feea:3074
        *      48 ff02:1::/32          fe80:1::225:90ff:feea:3074
        *      48 ff02:2::/32          fe80:2::225:90ff:feea:3075
        *      48 ff02:3::/32          fe80:3::225:90ff:feea:3076
        *      48 ff02:4::/32          fe80:4::225:90ff:feea:3077
        *      48 ff02:6::/32          fe80:6::225:90ff:fef3:8fc7
        *      48 ff02:7::/32          fe80:7::225:90ff:fef3:8fc8
        *      48 ff02:8::/32          fe80:8::225:90ff:fef3:8fc9
        *      48 ff02🅱:/32          ::1
        *      48 ff02:d::/32          fe80:d::225:90ff:feea:3075
        *      48 ff02:e::/32          fe80:e::225:90ff:feea:3074
        *      48 ff02:f::/32          fe80:f::225:90ff:feea:3074
        *      48 ff02:10::/32        fe80:10::225:90ff:feea:3074
        *      48 ff02:11::/32        fe80:11::225:90ff:feea:3074
        *      48 ff02:18::/32        fe80:18::225:90ff:feea:3074
        *      48 ff02:19::/32        fe80:19::225:90ff:feea:3074

      Network:
      flags: S = Static
      flags destination
      *S      0 192.168.48.0/25      192.168.55.3
      *S      0 192.168.48.128/25    192.168.55.3
      *S      0 192.168.49.0/25      192.168.55.3

      Nexthops:
      Flags: * = nexthop valid

      Nexthop        Route              Prio Gateway        Iface

      • 192.168.55.3    192.168.55.3/32      48 connected      opt6_vip249 (UP, master)
      • 192.168.55.5    192.168.55.0/29      48 192.168.55.1    igb2_vlan300 (UP, 1000 Mbps)

      IP:
      flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
      origin: i = IGP, e = EGP, ? = Incomplete

      flags destination          gateway          lpref  med aspath origin
      >    172.16.24.0/21      192.168.55.5      100    0 9059 i
      AI      >  192.168.48.0/25      192.168.55.3      100    0 i
      AI*>  192.168.48.128/25    192.168.55.3      100    0 i
      AI*>  192.168.49.0/25      192.168.55.3      100    0 i

      Neighbors:
      BGP neighbor is 192.168.55.5, remote AS 9059
      Description: AWS-DC MER Peer
        BGP version 4, remote router-id 192.168.55.5
        BGP state = Established, up for 00:04:09
        Last read 00:00:23, holdtime 90s, keepalive interval 30s
        Neighbor capabilities:
          Multiprotocol extensions: IPv4 unicast
          Route Refresh
          Graceful Restart
          4-byte AS numbers

      Message statistics:
                        Sent      Received 
        Opens                    1          1
        Notifications            0          0
        Updates                  2          2
        Keepalives              9        10
        Route Refresh            0          0
        Total                  12        13

      Update statistics:
                        Sent      Received 
        Updates                12          1
        Withdraws                0          0
        End-of-Rib              1          1

      Local host:          192.168.55.1, Local port:    179
        Remote host:        192.168.55.5, Remote port: 59288

      … and lastly here is the traceroute from the client server:
      tracert 172.16.24.7

      Tracing route to 172.16.24.7 over a maximum of 30 hops

      1    <1 ms    <1 ms    <1 ms  192.168.48.118
        2    5 ms    2 ms    1 ms  81.27.95.83
        3    1 ms    1 ms    1 ms  109.104.114.134
        4    1 ms    1 ms    1 ms  betelgeuse-hardy.c4l.co.uk [109.104.114.105]
        5    1 ms    2 ms    70 ms  hardy-wolverine.c4l.co.uk [109.104.114.6]
        6    *        *        *    Request timed out.
        7    *        *        *    Request timed out.
        8    *        *        *    Request timed out.
        9    *        *        *    Request timed out.

      1 Reply Last reply Reply Quote 0
      • First post
        Last post