Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    OpenBGP Does not seem to be publishing routes from neighbor properly

    Routing and Multi WAN
    1
    1
    1587
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      relfie last edited by

      Good afternoon,

      I am a BGP newbie, so please forgive me if this is the wrong forum.

      We are using pfSense 2.1.4 and OpenBGPD package 0.9.2.  We are trying to implement Amazon AWS direct connect.  I believe I have the bgpd.conf correct as I am seeing the following in routing.log, also bgpd status shows messages being exchanged.

      === snip routing.log ===
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: startup
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: rereading config
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: route decision engine ready
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: session engine ready
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5783]: RDE reconfigured
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: listening on 192.168.55.1
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: SE reconfigured
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change None -> Idle, reason: None
      Aug  7 17:16:00 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.3 now valid: directly connected
      Aug  7 17:16:01 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Idle -> Connect, reason: Start
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change Connect -> OpenSent, reason: Connection opened
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenSent -> OpenConfirm, reason: OPEN message received
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5790]: neighbor 192.168.55.5 (AWS-DC MER Peer): state change OpenConfirm -> Established, reason: KEEPALIVE message received
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5783]: Rib Loc-RIB: neighbor 192.168.55.5 (AWS-DC MER Peer) AS9059: update 172.16.24.0/21 via 192.168.55.5
      Aug  7 17:16:30 4slgbmernfw01 bgpd[5653]: nexthop 192.168.55.5 now valid: via 192.168.55.1

      However when a server on a local subnet in our AS tries to ping a server in the remote AS the traffic gets routed to the WAN interface and not over the BGP nexthop.

      Here is our BGPD config :

      === snip ===

      This file was created by the package manager.  Do not edit!

      ########

      Our AS

      ########

      AS 65458
      fib-update yes
      listen on 192.168.55.1
      log updates
      network 192.168.48.0/25 set nexthop 192.168.55.3
      network 192.168.48.128/25 set nexthop 192.168.55.3
      network 192.168.49.0/25 set nexthop 192.168.55.3

      ########

      Peer Groups

      ########
      group "AWSDC" {
      remote-as 9059
      neighbor 192.168.55.5 {
                                            descr "AWS-DC MER Peer"
                                            tcp md5sig password 8e484c715b2be0e50d576bc0bb0c29d4
                                            announce all 
                                            local-address 192.168.55.3
      }
      }
      deny from any
      deny to any
      allow from 192.168.55.5
      allow to 192.168.55.5

      ..here is the BGPD Status

      Summary:
      Neighbor                  AS    MsgRcvd    MsgSent  OutQ Up/Down  State/PrfRcvd
      AWS-DC MER Peer          9059        13        12    0 00:04:09      1

      Interfaces:
      Interface      Nexthop state  Flags          Link state
      opt6_vip249    ok            UP            CARP, master
      igb2_vlan300  ok            UP            active, 1000 MBit/s
      ovpns1        ok            UP            active
      wan_vip250    ok            UP            CARP, master
      opt4_vip251    ok            UP            CARP, master
      opt3_vip252    ok            UP            CARP, master
      opt2_vip253    ok            UP            CARP, master
      opt1_vip254    ok            UP            CARP, master
      wan_vip255    ok            UP            CARP, master
      lagg0_vlan50  ok            UP            active, 10 MBit/s
      lagg0_vlan30  ok            UP            active, 10 MBit/s
      lagg0_vlan20  ok            UP            active, 10 MBit/s
      lagg0_vlan10  ok            UP            active, 10 MBit/s
      lagg0          ok            UP            Ethernet, active, 1000 MBit/s
      pflog0        invalid                      invalid
      lo0            ok            UP            invalid
      pfsync0        ok            UP            invalid
      enc0          ok            UP            invalid
      igb7          ok            UP            active, 1000 MBit/s
      igb6          ok            UP            Ethernet, active, 1000 MBit/s
      igb5          ok            UP            active, 1000 MBit/s
      igb4          invalid                      Ethernet, invalid, 10 MBit/s
      igb3          ok            UP            active, 1000 MBit/s
      igb2          ok            UP            Ethernet, active, 1000 MBit/s
      igb1          ok            UP            active, 1000 MBit/s
      igb0          ok            UP            Ethernet, active, 1000 MBit/s

      Routing:
      flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
      origin: i = IGP, e = EGP, ? = Incomplete

      flags destination          gateway          lpref  med aspath origin
      >    172.16.24.0/21      192.168.55.5      100    0 9059 i
      AI      >  192.168.48.0/25      192.168.55.3      100    0 i
      AI*>  192.168.48.128/25    192.168.55.3      100    0 i
      AI*>  192.168.49.0/25      192.168.55.3      100    0 i

      Forwarding:
      flags: * = valid, B = BGP, C = Connected, S = Static
            N = BGP Nexthop reachable via this route
            r = reject route, b = blackhole route

      flags prio destination          gateway
      *S      48 0.0.0.0/0            81.27.95.81
      *S      48 10.101.1.0/25        192.168.48.1
      *S      48 10.101.1.128/25      192.168.48.129
      *S      48 10.101.2.0/25        192.168.49.1
      *S      48 10.101.5.0/25        192.168.48.1
      *S      48 10.101.5.128/25      192.168.48.129
      *S      48 10.101.6.0/25        192.168.49.1
      *      48 81.27.95.80/28      81.27.95.84
      *C      48 81.27.95.84/32      link#11
      *C      48 81.27.95.93/32      link#23
      *C      48 81.27.95.94/32      link#18
      *C      48 84.20.199.91/32      link#1
      *C      0 127.0.0.0/8          link#0
      *C      48 127.0.0.1/32        link#11
      *B      48 172.16.24.0/21      192.168.55.1
      *S      48 192.168.44.0/23      192.168.48.1
      *S      48 192.168.46.0/24      192.168.48.1
      *C      48 192.168.48.0/25      link#14
      *C      48 192.168.48.118/32    link#11
      *C      48 192.168.48.126/32    link#19
      *C      48 192.168.48.128/25    link#15
      *C      48 192.168.48.246/32    link#11
      *C      48 192.168.48.254/32    link#20
      *C      48 192.168.49.0/25      link#16
      *C      48 192.168.49.118/32    link#11
      *C      48 192.168.49.126/32    link#21
      *C      48 192.168.49.128/25    link#17
      *C      48 192.168.49.246/32    link#11
      *C      48 192.168.49.254/32    link#22
      *S      48 192.168.50.0/24      192.168.48.1

      • N    48 192.168.55.0/29      192.168.55.1
        *C      48 192.168.55.1/32      link#11
        *CN    48 192.168.55.3/32      link#26
        *S      48 192.168.90.0/24      192.168.48.1
        *S      48 192.168.200.0/24    192.168.200.2
        *C      48 192.168.200.1/32    link#11
        *C      48 192.168.200.2/32    link#24
        *C      48 192.168.226.0/27    link#7
        *C      48 192.168.226.2/32    link#11
        *C      0 ::1/128              link#0
        *C      48 ::1/128              link#11
        *C      48 fe80:1::/64          link#1
        *C      48 fe80:1::225:90ff:feea:3074/128 link#11
        *C      48 fe80:2::/64          link#2
        *C      48 fe80:2::225:90ff:feea:3075/128 link#11
        *C      48 fe80:3::/64          link#3
        *C      48 fe80:3::225:90ff:feea:3076/128 link#11
        *C      48 fe80:4::/64          link#4
        *C      48 fe80:4::225:90ff:feea:3077/128 link#11
        *C      48 fe80:6::/64          link#6
        *C      48 fe80:6::225:90ff:fef3:8fc7/128 link#11
        *C      48 fe80:7::/64          link#7
        *C      48 fe80:7::225:90ff:fef3:8fc8/128 link#11
        *C      48 fe80:8::/64          link#8
        *C      48 fe80:8::225:90ff:fef3:8fc9/128 link#11
        *C      48 fe80🅱:/64          link#11
        *C      48 fe80🅱:1/128        link#11
        *C      48 fe80:d::/64          link#13
        *C      48 fe80:d::225:90ff:feea:3075/128 link#11
        *C      48 fe80:e::/64          link#14
        *C      48 fe80:e::225:90ff:feea:3074/128 link#11
        *C      48 fe80:f::/64          link#15
        *C      48 fe80:f::225:90ff:feea:3074/128 link#11
        *C      48 fe80:10::/64        link#16
        *C      48 fe80:10::225:90ff:feea:3074/128 link#11
        *C      48 fe80:11::/64        link#17
        *C      48 fe80:11::225:90ff:feea:3074/128 link#11
        *C      48 fe80:18::225:90ff:feea:3074/128 link#11
        *C      48 fe80:19::/64        link#25
        *C      48 fe80:19::225:90ff:feea:3074/128 link#11
        *      48 ff01:1::/32          fe80:1::225:90ff:feea:3074
        *      48 ff01:2::/32          fe80:2::225:90ff:feea:3075
        *      48 ff01:3::/32          fe80:3::225:90ff:feea:3076
        *      48 ff01:4::/32          fe80:4::225:90ff:feea:3077
        *      48 ff01:6::/32          fe80:6::225:90ff:fef3:8fc7
        *      48 ff01:7::/32          fe80:7::225:90ff:fef3:8fc8
        *      48 ff01:8::/32          fe80:8::225:90ff:fef3:8fc9
        *      48 ff01🅱:/32          ::1
        *      48 ff01:d::/32          fe80:d::225:90ff:feea:3075
        *      48 ff01:e::/32          fe80:e::225:90ff:feea:3074
        *      48 ff01:f::/32          fe80:f::225:90ff:feea:3074
        *      48 ff01:10::/32        fe80:10::225:90ff:feea:3074
        *      48 ff01:11::/32        fe80:11::225:90ff:feea:3074
        *      48 ff01:18::/32        fe80:18::225:90ff:feea:3074
        *      48 ff01:19::/32        fe80:19::225:90ff:feea:3074
        *      48 ff02:1::/32          fe80:1::225:90ff:feea:3074
        *      48 ff02:2::/32          fe80:2::225:90ff:feea:3075
        *      48 ff02:3::/32          fe80:3::225:90ff:feea:3076
        *      48 ff02:4::/32          fe80:4::225:90ff:feea:3077
        *      48 ff02:6::/32          fe80:6::225:90ff:fef3:8fc7
        *      48 ff02:7::/32          fe80:7::225:90ff:fef3:8fc8
        *      48 ff02:8::/32          fe80:8::225:90ff:fef3:8fc9
        *      48 ff02🅱:/32          ::1
        *      48 ff02:d::/32          fe80:d::225:90ff:feea:3075
        *      48 ff02:e::/32          fe80:e::225:90ff:feea:3074
        *      48 ff02:f::/32          fe80:f::225:90ff:feea:3074
        *      48 ff02:10::/32        fe80:10::225:90ff:feea:3074
        *      48 ff02:11::/32        fe80:11::225:90ff:feea:3074
        *      48 ff02:18::/32        fe80:18::225:90ff:feea:3074
        *      48 ff02:19::/32        fe80:19::225:90ff:feea:3074

      Network:
      flags: S = Static
      flags destination
      *S      0 192.168.48.0/25      192.168.55.3
      *S      0 192.168.48.128/25    192.168.55.3
      *S      0 192.168.49.0/25      192.168.55.3

      Nexthops:
      Flags: * = nexthop valid

      Nexthop        Route              Prio Gateway        Iface

      • 192.168.55.3    192.168.55.3/32      48 connected      opt6_vip249 (UP, master)
      • 192.168.55.5    192.168.55.0/29      48 192.168.55.1    igb2_vlan300 (UP, 1000 Mbps)

      IP:
      flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
      origin: i = IGP, e = EGP, ? = Incomplete

      flags destination          gateway          lpref  med aspath origin
      >    172.16.24.0/21      192.168.55.5      100    0 9059 i
      AI      >  192.168.48.0/25      192.168.55.3      100    0 i
      AI*>  192.168.48.128/25    192.168.55.3      100    0 i
      AI*>  192.168.49.0/25      192.168.55.3      100    0 i

      Neighbors:
      BGP neighbor is 192.168.55.5, remote AS 9059
      Description: AWS-DC MER Peer
        BGP version 4, remote router-id 192.168.55.5
        BGP state = Established, up for 00:04:09
        Last read 00:00:23, holdtime 90s, keepalive interval 30s
        Neighbor capabilities:
          Multiprotocol extensions: IPv4 unicast
          Route Refresh
          Graceful Restart
          4-byte AS numbers

      Message statistics:
                        Sent      Received 
        Opens                    1          1
        Notifications            0          0
        Updates                  2          2
        Keepalives              9        10
        Route Refresh            0          0
        Total                  12        13

      Update statistics:
                        Sent      Received 
        Updates                12          1
        Withdraws                0          0
        End-of-Rib              1          1

      Local host:          192.168.55.1, Local port:    179
        Remote host:        192.168.55.5, Remote port: 59288

      … and lastly here is the traceroute from the client server:
      tracert 172.16.24.7

      Tracing route to 172.16.24.7 over a maximum of 30 hops

      1    <1 ms    <1 ms    <1 ms  192.168.48.118
        2    5 ms    2 ms    1 ms  81.27.95.83
        3    1 ms    1 ms    1 ms  109.104.114.134
        4    1 ms    1 ms    1 ms  betelgeuse-hardy.c4l.co.uk [109.104.114.105]
        5    1 ms    2 ms    70 ms  hardy-wolverine.c4l.co.uk [109.104.114.6]
        6    *        *        *    Request timed out.
        7    *        *        *    Request timed out.
        8    *        *        *    Request timed out.
        9    *        *        *    Request timed out.

      1 Reply Last reply Reply Quote 0
      • First post
        Last post

      Products

      • Platform Overview
      • TNSR
      • pfSense
      • Appliances

      Services

      • Training
      • Professional Services

      Support

      • Subscription Plans
      • Contact Support
      • Product Lifecycle
      • Documentation

      News

      • Media Coverage
      • Press
      • Events

      Resources

      • Blog
      • FAQ
      • Find a Partner
      • Resource Library
      • Security Information

      Company

      • About Us
      • Careers
      • Partners
      • Contact Us
      • Legal
      Our Mission

      We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

      Subscribe to our Newsletter

      Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

      © 2021 Rubicon Communications, LLC | Privacy Policy