4. OpenBGP Does not seem to be publishing routes from neighbor properly

OpenBGP Does not seem to be publishing routes from neighbor properly

  • R

    Good afternoon,

    I am a BGP newbie, so please forgive me if this is the wrong forum.

    We are using pfSense 2.1.4 and OpenBGPD package 0.9.2.  We are trying to implement Amazon AWS direct connect.  I believe I have the bgpd.conf correct as I am seeing the following in routing.log, also bgpd status shows messages being exchanged.

    === snip routing.log ===
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: startup
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: rereading config
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: route decision engine ready
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: session engine ready
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: RDE reconfigured
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: listening on 192.168.55.1
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: SE reconfigured
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change None -> Idle, reason: None
    Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.3 now valid: directly connected
    Aug  7 17:16:01 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Idle -> Connect, reason: Start
    Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Connect -> OpenSent, reason: Connection opened
    Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenSent -> OpenConfirm, reason: OPEN message received
    Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenConfirm -> Established, reason: KEEPALIVE message received
    Aug  7 17:16:30 4slgbmernfw01 bgpd[5783]: Rib Loc-RIB: neighbor 192.168.55.5 (AWS-DC MER Peer) AS9059: update 172.16.24.0/21 via 192.168.55.5
    Aug  7 17:16:30 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.5 now valid: via 192.168.55.1

    However when a server on a local subnet in our AS tries to ping a server in the remote AS the traffic gets routed to the WAN interface and not over the BGP nexthop.

    Here is our BGPD config :

    === snip ===

    This file was created by the package manager.  Do not edit!

    ########

    Our AS

    ########

    AS 65458
    fib-update yes
    listen on 192.168.55.1
    log updates
    network 192.168.48.0/25 set nexthop 192.168.55.3
    network 192.168.48.128/25 set nexthop 192.168.55.3
    network 192.168.49.0/25 set nexthop 192.168.55.3

    ########

    Peer Groups

    ########
    group "AWSDC" {
    remote-as 9059
    neighbor 192.168.55.5 {
                                          descr "AWS-DC MER Peer"
                                          tcp md5sig password 8e484c715b2be0e50d576bc0bb0c29d4
                                          announce all 
                                          local-address 192.168.55.3
    }
    }
    deny from any
    deny to any
    allow from 192.168.55.5
    allow to 192.168.55.5

    ..here is the BGPD Status

    Summary:
    Neighbor                  AS    MsgRcvd    MsgSent  OutQ Up/Down  State/PrfRcvd
    AWS-DC MER Peer          9059        13        12    0 00:04:09      1

    Interfaces:
    Interface      Nexthop state  Flags          Link state
    opt6_vip249    ok            UP            CARP, master
    igb2_vlan300  ok            UP            active, 1000 MBit/s
    ovpns1        ok            UP            active
    wan_vip250    ok            UP            CARP, master
    opt4_vip251    ok            UP            CARP, master
    opt3_vip252    ok            UP            CARP, master
    opt2_vip253    ok            UP            CARP, master
    opt1_vip254    ok            UP            CARP, master
    wan_vip255    ok            UP            CARP, master
    lagg0_vlan50  ok            UP            active, 10 MBit/s
    lagg0_vlan30  ok            UP            active, 10 MBit/s
    lagg0_vlan20  ok            UP            active, 10 MBit/s
    lagg0_vlan10  ok            UP            active, 10 MBit/s
    lagg0          ok            UP            Ethernet, active, 1000 MBit/s
    pflog0        invalid                      invalid
    lo0            ok            UP            invalid
    pfsync0        ok            UP            invalid
    enc0          ok            UP            invalid
    igb7          ok            UP            active, 1000 MBit/s
    igb6          ok            UP            Ethernet, active, 1000 MBit/s
    igb5          ok            UP            active, 1000 MBit/s
    igb4          invalid                      Ethernet, invalid, 10 MBit/s
    igb3          ok            UP            active, 1000 MBit/s
    igb2          ok            UP            Ethernet, active, 1000 MBit/s
    igb1          ok            UP            active, 1000 MBit/s
    igb0          ok            UP            Ethernet, active, 1000 MBit/s

    Routing:
    flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
    origin: i = IGP, e = EGP, ? = Incomplete

    flags destination          gateway          lpref  med aspath origin
    >    172.16.24.0/21      192.168.55.5      100    0 9059 i
    AI    >  192.168.48.0/25      192.168.55.3      100    0 i
    AI*>  192.168.48.128/25    192.168.55.3      100    0 i
    AI*>  192.168.49.0/25      192.168.55.3      100    0 i

    Forwarding:
    flags: * = valid, B = BGP, C = Connected, S = Static
          N = BGP Nexthop reachable via this route
          r = reject route, b = blackhole route

    flags prio destination          gateway
    *S      48 0.0.0.0/0            81.27.95.81
    *S      48 10.101.1.0/25        192.168.48.1
    *S      48 10.101.1.128/25      192.168.48.129
    *S      48 10.101.2.0/25        192.168.49.1
    *S      48 10.101.5.0/25        192.168.48.1
    *S      48 10.101.5.128/25      192.168.48.129
    *S      48 10.101.6.0/25        192.168.49.1
    *      48 81.27.95.80/28      81.27.95.84
    *C      48 81.27.95.84/32      link#11
    *C      48 81.27.95.93/32      link#23
    *C      48 81.27.95.94/32      link#18
    *C      48 84.20.199.91/32      link#1
    *C      0 127.0.0.0/8          link#0
    *C      48 127.0.0.1/32        link#11
    *B      48 172.16.24.0/21      192.168.55.1
    *S      48 192.168.44.0/23      192.168.48.1
    *S      48 192.168.46.0/24      192.168.48.1
    *C      48 192.168.48.0/25      link#14
    *C      48 192.168.48.118/32    link#11
    *C      48 192.168.48.126/32    link#19
    *C      48 192.168.48.128/25    link#15
    *C      48 192.168.48.246/32    link#11
    *C      48 192.168.48.254/32    link#20
    *C      48 192.168.49.0/25      link#16
    *C      48 192.168.49.118/32    link#11
    *C      48 192.168.49.126/32    link#21
    *C      48 192.168.49.128/25    link#17
    *C      48 192.168.49.246/32    link#11
    *C      48 192.168.49.254/32    link#22
    *S      48 192.168.50.0/24      192.168.48.1

    • N    48 192.168.55.0/29      192.168.55.1
      *C      48 192.168.55.1/32      link#11
      *CN    48 192.168.55.3/32      link#26
      *S      48 192.168.90.0/24      192.168.48.1
      *S      48 192.168.200.0/24    192.168.200.2
      *C      48 192.168.200.1/32    link#11
      *C      48 192.168.200.2/32    link#24
      *C      48 192.168.226.0/27    link#7
      *C      48 192.168.226.2/32    link#11
      *C      0 ::1/128              link#0
      *C      48 ::1/128              link#11
      *C      48 fe80:1::/64          link#1
      *C      48 fe80:1::225:90ff:feea:3074/128 link#11
      *C      48 fe80:2::/64          link#2
      *C      48 fe80:2::225:90ff:feea:3075/128 link#11
      *C      48 fe80:3::/64          link#3
      *C      48 fe80:3::225:90ff:feea:3076/128 link#11
      *C      48 fe80:4::/64          link#4
      *C      48 fe80:4::225:90ff:feea:3077/128 link#11
      *C      48 fe80:6::/64          link#6
      *C      48 fe80:6::225:90ff:fef3:8fc7/128 link#11
      *C      48 fe80:7::/64          link#7
      *C      48 fe80:7::225:90ff:fef3:8fc8/128 link#11
      *C      48 fe80:8::/64          link#8
      *C      48 fe80:8::225:90ff:fef3:8fc9/128 link#11
      *C      48 fe80🅱:/64          link#11
      *C      48 fe80🅱:1/128        link#11
      *C      48 fe80:d::/64          link#13
      *C      48 fe80:d::225:90ff:feea:3075/128 link#11
      *C      48 fe80:e::/64          link#14
      *C      48 fe80:e::225:90ff:feea:3074/128 link#11
      *C      48 fe80:f::/64          link#15
      *C      48 fe80:f::225:90ff:feea:3074/128 link#11
      *C      48 fe80:10::/64        link#16
      *C      48 fe80:10::225:90ff:feea:3074/128 link#11
      *C      48 fe80:11::/64        link#17
      *C      48 fe80:11::225:90ff:feea:3074/128 link#11
      *C      48 fe80:18::225:90ff:feea:3074/128 link#11
      *C      48 fe80:19::/64        link#25
      *C      48 fe80:19::225:90ff:feea:3074/128 link#11
      *      48 ff01:1::/32          fe80:1::225:90ff:feea:3074
      *      48 ff01:2::/32          fe80:2::225:90ff:feea:3075
      *      48 ff01:3::/32          fe80:3::225:90ff:feea:3076
      *      48 ff01:4::/32          fe80:4::225:90ff:feea:3077
      *      48 ff01:6::/32          fe80:6::225:90ff:fef3:8fc7
      *      48 ff01:7::/32          fe80:7::225:90ff:fef3:8fc8
      *      48 ff01:8::/32          fe80:8::225:90ff:fef3:8fc9
      *      48 ff01🅱:/32          ::1
      *      48 ff01:d::/32          fe80:d::225:90ff:feea:3075
      *      48 ff01:e::/32          fe80:e::225:90ff:feea:3074
      *      48 ff01:f::/32          fe80:f::225:90ff:feea:3074
      *      48 ff01:10::/32        fe80:10::225:90ff:feea:3074
      *      48 ff01:11::/32        fe80:11::225:90ff:feea:3074
      *      48 ff01:18::/32        fe80:18::225:90ff:feea:3074
      *      48 ff01:19::/32        fe80:19::225:90ff:feea:3074
      *      48 ff02:1::/32          fe80:1::225:90ff:feea:3074
      *      48 ff02:2::/32          fe80:2::225:90ff:feea:3075
      *      48 ff02:3::/32          fe80:3::225:90ff:feea:3076
      *      48 ff02:4::/32          fe80:4::225:90ff:feea:3077
      *      48 ff02:6::/32          fe80:6::225:90ff:fef3:8fc7
      *      48 ff02:7::/32          fe80:7::225:90ff:fef3:8fc8
      *      48 ff02:8::/32          fe80:8::225:90ff:fef3:8fc9
      *      48 ff02🅱:/32          ::1
      *      48 ff02:d::/32          fe80:d::225:90ff:feea:3075
      *      48 ff02:e::/32          fe80:e::225:90ff:feea:3074
      *      48 ff02:f::/32          fe80:f::225:90ff:feea:3074
      *      48 ff02:10::/32        fe80:10::225:90ff:feea:3074
      *      48 ff02:11::/32        fe80:11::225:90ff:feea:3074
      *      48 ff02:18::/32        fe80:18::225:90ff:feea:3074
      *      48 ff02:19::/32        fe80:19::225:90ff:feea:3074

    Network:
    flags: S = Static
    flags destination
    *S      0 192.168.48.0/25      192.168.55.3
    *S      0 192.168.48.128/25    192.168.55.3
    *S      0 192.168.49.0/25      192.168.55.3

    Nexthops:
    Flags: * = nexthop valid

    Nexthop        Route              Prio Gateway        Iface

    • 192.168.55.3    192.168.55.3/32      48 connected      opt6_vip249 (UP, master)
    • 192.168.55.5    192.168.55.0/29      48 192.168.55.1    igb2_vlan300 (UP, 1000 Mbps)

    IP:
    flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
    origin: i = IGP, e = EGP, ? = Incomplete

    flags destination          gateway          lpref  med aspath origin
    >    172.16.24.0/21      192.168.55.5      100    0 9059 i
    AI    >  192.168.48.0/25      192.168.55.3      100    0 i
    AI*>  192.168.48.128/25    192.168.55.3      100    0 i
    AI*>  192.168.49.0/25      192.168.55.3      100    0 i

    Neighbors:
    BGP neighbor is 192.168.55.5, remote AS 9059
    Description: AWS-DC MER Peer
      BGP version 4, remote router-id 192.168.55.5
      BGP state = Established, up for 00:04:09
      Last read 00:00:23, holdtime 90s, keepalive interval 30s
      Neighbor capabilities:
        Multiprotocol extensions: IPv4 unicast
        Route Refresh
        Graceful Restart
        4-byte AS numbers

    Message statistics:
                      Sent      Received 
      Opens                    1          1
      Notifications            0          0
      Updates                  2          2
      Keepalives              9        10
      Route Refresh            0          0
      Total                  12        13

    Update statistics:
                      Sent      Received 
      Updates                12          1
      Withdraws                0          0
      End-of-Rib              1          1

    Local host:          192.168.55.1, Local port:    179
      Remote host:        192.168.55.5, Remote port: 59288

    … and lastly here is the traceroute from the client server:
    tracert 172.16.24.7

    Tracing route to 172.16.24.7 over a maximum of 30 hops

    1    <1 ms    <1 ms    <1 ms  192.168.48.118
      2    5 ms    2 ms    1 ms  81.27.95.83
      3    1 ms    1 ms    1 ms  109.104.114.134
      4    1 ms    1 ms    1 ms  betelgeuse-hardy.c4l.co.uk [109.104.114.105]
      5    1 ms    2 ms    70 ms  hardy-wolverine.c4l.co.uk [109.104.114.6]
      6    *        *        *    Request timed out.
      7    *        *        *    Request timed out.
      8    *        *        *    Request timed out.
      9    *        *        *    Request timed out.

    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy