Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments

    Scheduled Pinned Locked Moved FRR
    31 Posts 4 Posters 2.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      michmoor LAYER 8 Rebel Alliance @amithb
      last edited by

      @amithb said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

      te these configurations, traffic between the SFO and LAX subnets is still preferring the default route (10.80.20.1) over the direct BGP-learned path. The BGP routing tables on both sides appear correct, and AS path prepending reflects properly. However, traceroute and packet captures show that traffic continues to take the path through the default gateway.

      Routing Table Snapshots:

      Are you doing any policy based routing? Setting a specific gateway in the firewall rules? Thats the only thing i can imagine that would bypass the route table.

      Firewall: NetGate,Palo Alto-VM,Juniper SRX
      Routing: Juniper, Arista, Cisco
      Switching: Juniper, Arista, Cisco
      Wireless: Unifi, Aruba IAP
      JNCIP,CCNP Enterprise

      A 1 Reply Last reply Reply Quote 0
      • A
        amithb @michmoor
        last edited by

        @michmoor said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

        Are you doing any policy based routing? Setting a specific gateway in the firewall rules? Thats the only thing i can imagine that would bypass the route table.

        There is no policy based routing or specific gateway in firewall rule. Only one default gateway present on both site routers. I would say it is just a basic BGP config. I just followed the example configuration from the netgate docs - https://docs.netgate.com/pfsense/en/latest/packages/frr/bgp/example.html

        K 1 Reply Last reply Reply Quote 0
        • K
          Kevin S Pare @amithb
          last edited by

          @amithb One thing i've noticed is that when I gracefully shutdown a bgp peer, incoming traffic moved over to the other peer. but if there were enteries in the state table using the shutdown peer, traffic would still go out that peer. i'm just looking into TCP Established and its default is 86400 or 24 hours. i've turned mine down to 90 seconds and will do some testing there with that.

          M A 2 Replies Last reply Reply Quote 0
          • M
            michmoor LAYER 8 Rebel Alliance @Kevin S Pare
            last edited by michmoor

            @Kevin-S-Pare

            Known issue.

            https://redmine.pfsense.org/issues/14630
            https://redmine.pfsense.org/issues/14633#note-2

            Supposedly, script support is needed in FRR to fix
            @marcosm Any fix incoming for this?

            Firewall: NetGate,Palo Alto-VM,Juniper SRX
            Routing: Juniper, Arista, Cisco
            Switching: Juniper, Arista, Cisco
            Wireless: Unifi, Aruba IAP
            JNCIP,CCNP Enterprise

            K 1 Reply Last reply Reply Quote 0
            • K
              Kevin S Pare @michmoor
              last edited by

              @michmoor said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

              @Kevin-S-Pare

              Known issue.

              https://redmine.pfsense.org/issues/14630
              https://redmine.pfsense.org/issues/14633#note-2

              Supposedly, script support is needed in FRR to fix
              @marcosm Any fix incoming for this?

              Shouldn't a low state time help fix this issue too? or are the states just kinda glued in there.

              I even tried marking the gateway as down but until I actually unplugged the cable and made it down, then it killed the states....not very graceful lol....

              im trying to not spend money on some cisco isr for my bgp, but this is just making my case unfortunately....

              M 1 Reply Last reply Reply Quote 0
              • M
                michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                last edited by

                @Kevin-S-Pare

                To be fair, i would never use a stateful device(firewall) to handle BGP routing to the internet.
                You are even considering a Cisco ASR which is correct - use a router not a firewall.

                Firewall: NetGate,Palo Alto-VM,Juniper SRX
                Routing: Juniper, Arista, Cisco
                Switching: Juniper, Arista, Cisco
                Wireless: Unifi, Aruba IAP
                JNCIP,CCNP Enterprise

                K 2 Replies Last reply Reply Quote 0
                • K
                  Kevin S Pare @michmoor
                  last edited by

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • A
                    amithb @Kevin S Pare
                    last edited by

                    @Kevin-S-Pare Thanks for checking on this issue. I haven’t found a solution yet, so I’m currently managing with static routes as a workaround. Any guidance or suggestions to try out would be greatly appreciated.

                    K 2 Replies Last reply Reply Quote 0
                    • K
                      Kevin S Pare @amithb
                      last edited by

                      @amithb said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

                      @Kevin-S-Pare Thanks for checking on this issue. I haven’t found a solution yet, so I’m currently managing with static routes as a workaround. Any guidance or suggestions to try out would be greatly appreciated.

                      I'll do some testing friday night and see how lowering the state timeout goes.

                      1 Reply Last reply Reply Quote 0
                      • K
                        Kevin S Pare @michmoor
                        last edited by

                        @michmoor said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

                        @Kevin-S-Pare

                        To be fair, i would never use a stateful device(firewall) to handle BGP routing to the internet.
                        You are even considering a Cisco ASR which is correct - use a router not a firewall.

                        I was actually considering picking up a ASR1001X-20G instead of running pfsense for my bgp peers.

                        But reading about it, i'm not 100% sure its not still a stateful firewall?

                        M 1 Reply Last reply Reply Quote 0
                        • K
                          Kevin S Pare @amithb
                          last edited by

                          @amithb we host hundreds of citrix sessions, and with the states low we are getting complaints about disconnects so we've change the settings back and will be looking to replace pfsense as our bgp router....just isn't working how we need it.

                          A 1 Reply Last reply Reply Quote 0
                          • M
                            michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                            last edited by

                            @Kevin-S-Pare said in [BGP Routing Issue: Traffic Still Preferring Default Route Despite

                            But reading about it, i'm not 100% sure its not still a stateful firewall?

                            Reply

                            Not sure what you are asking. Is an ASR1001 a firewall or router? Its a router. Routers are not stateful devices by nature. If you take advantage of the SDN side (the license is expensive) its a very robust platform.

                            If you want a cost-effective solution and still stick with netgate I know they offer TNSR. Ive been playing with it and its not bad. Granted I'm coming from an Arista/Juniper background so TNSR has some shortcomings that would prevent me from deploying in an enterprise but it does BGP. It can handle routes.

                            As an aside...I deployed Cumulus a few years ago and that turned me totally off on using OSS network gear. I made the exception with Netgate but man...I would never do that again.

                            The pfSense firewall just isn't meant to route at the edge using BGP. Minus the shortcoming you are seeing with FRR and pfsense holding onto states, I personally would not design any solution that requires tracking state and also doing bgp. Firewall behind the router.

                            Firewall: NetGate,Palo Alto-VM,Juniper SRX
                            Routing: Juniper, Arista, Cisco
                            Switching: Juniper, Arista, Cisco
                            Wireless: Unifi, Aruba IAP
                            JNCIP,CCNP Enterprise

                            K 1 Reply Last reply Reply Quote 0
                            • K
                              Kevin S Pare @michmoor
                              last edited by

                              @michmoor said in BGP Routing Issue: Traffic Still Preferring Default Route Despite Prepending and MED Adjustments:

                              @Kevin-S-Pare said in [BGP Routing Issue: Traffic Still Preferring Default Route Despite

                              But reading about it, i'm not 100% sure its not still a stateful firewall?

                              Reply

                              Not sure what you are asking. Is an ASR1001 a firewall or router? Its a router. Routers are not stateful devices by nature. If you take advantage of the SDN side (the license is expensive) its a very robust platform.

                              If you want a cost-effective solution and still stick with netgate I know they offer TNSR. Ive been playing with it and its not bad. Granted I'm coming from an Arista/Juniper background so TNSR has some shortcomings that would prevent me from deploying in an enterprise but it does BGP. It can handle routes.

                              As an aside...I deployed Cumulus a few years ago and that turned me totally off on using OSS network gear. I made the exception with Netgate but man...I would never do that again.

                              The pfSense firewall just isn't meant to route at the edge using BGP. Minus the shortcoming you are seeing with FRR and pfsense holding onto states, I personally would not design any solution that requires tracking state and also doing bgp. Firewall behind the router.

                              You are very correct. it's just not working how I want it to work. I found a pretty good deal on a asr1009-20gb i'll pick up and try out instead.

                              M 1 Reply Last reply Reply Quote 1
                              • M
                                michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                                last edited by michmoor

                                @Kevin-S-Pare

                                Another option to think of and I'm not sure how well this would work is for each BGP peer, you have gateway monitoring enabled. Monitor IP can be whatever you want just different for each BGP peer.

                                There is an option when the gateway fails to kill states

                                https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#state-killing-gateway-failure

                                559e42a9-abba-4934-a6d1-8ed929d33366-image.png

                                Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                Routing: Juniper, Arista, Cisco
                                Switching: Juniper, Arista, Cisco
                                Wireless: Unifi, Aruba IAP
                                JNCIP,CCNP Enterprise

                                K 1 Reply Last reply Reply Quote 0
                                • K
                                  Kevin S Pare @michmoor
                                  last edited by

                                  @michmoor
                                  I actually have that enabled….i forced the gateway down but it still didn’t reset the states until it was actually down…

                                  M M 2 Replies Last reply Reply Quote 0
                                  • M
                                    michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                                    last edited by

                                    @Kevin-S-Pare
                                    Ok yeah that sucks…migrate…

                                    Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                    Routing: Juniper, Arista, Cisco
                                    Switching: Juniper, Arista, Cisco
                                    Wireless: Unifi, Aruba IAP
                                    JNCIP,CCNP Enterprise

                                    K 1 Reply Last reply Reply Quote 0
                                    • K
                                      Kevin S Pare @michmoor
                                      last edited by

                                      @michmoor yup! Ordering the Cisco today

                                      M 1 Reply Last reply Reply Quote 0
                                      • M
                                        michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                                        last edited by

                                        @Kevin-S-Pare
                                        Just frustrating.
                                        This is a similar situation I ran into with Cumulus. I’m all for open source software and do want to support but there are just situations I find myself in where something basic just doesn’t work. whether it’s an IPsec bug or dynamic routing. It’s just frustrating so I understand where you are coming from.

                                        Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                        Routing: Juniper, Arista, Cisco
                                        Switching: Juniper, Arista, Cisco
                                        Wireless: Unifi, Aruba IAP
                                        JNCIP,CCNP Enterprise

                                        K 1 Reply Last reply Reply Quote 0
                                        • K
                                          Kevin S Pare @michmoor
                                          last edited by

                                          @michmoor we’ve done some amazing stuff with Netgate so I can’t complain….they are doing great things but they have their limits and their place…

                                          M 1 Reply Last reply Reply Quote 1
                                          • M
                                            michmoor LAYER 8 Rebel Alliance @Kevin S Pare
                                            last edited by

                                            @Kevin-S-Pare

                                            If you have the Netgate 8200 or 8300 I forget which one you said, I would seriously reach out to Netgate sales and ask for TNSR. It’s close to Cisco syntax and it’s better suited for your task

                                            Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                            Routing: Juniper, Arista, Cisco
                                            Switching: Juniper, Arista, Cisco
                                            Wireless: Unifi, Aruba IAP
                                            JNCIP,CCNP Enterprise

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.