Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    BGP configuration help

    Scheduled Pinned Locked Moved FRR
    1 Posts 1 Posters 187 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • asacocoA
      asacoco
      last edited by asacoco

      I'm trying to get BGP working between Metal LB on my Kubernetes cluster and pfSense (2.7.2)/FRR (2.0.2_1).
      My aim is to use the 192.168.254.0/24 network for my Kubernetes load balancer, advertised by metallb via BGP to pfsense, so I can access those addresses from my LAN. The Kubernetes cluster has nodes in the 10.10.10.0/24 network on vlan 254 and my local 192.168.1.0/24 network is on VLAN 10.

      However I've clearly done something wrong because my traffic never reaches my ingress objects. In fact, traceroute to one of the ingress addresses just seems to exhaust the TTL. This was working under VyOS, so I suspect it's a pfsense configuration issue (combined with the fact I have no idea what I'm doing)

      Can anyone spot any obvious config issues with the below? I hope this rambling post makes some sense.

      2d92e7f7-d3f3-4a41-819a-4e5d84764341-Screenshot_20250608_124627-1.png

      pfSense FRR-BGP configuration

      frr defaults traditional
      hostname ahostname
      password redacted123
      ip nht resolve-via-default
      service integrated-vtysh-config
      !
      ip router-id 10.10.10.1
      !
      ip route 192.168.254.0/24 10.10.10.10
      ip route 192.168.254.0/24 10.10.10.11
      ip route 192.168.254.0/24 10.10.10.12
      ip route 192.168.254.0/24 10.10.10.13
      !
      router bgp 64512
       bgp router-id 10.10.10.1
       bgp graceful-shutdown
       no bgp network import-check
       no bgp ebgp-requires-policy
       neighbor metallb peer-group
       neighbor metallb remote-as 64511
       neighbor metallb update-source 10.10.10.1
       neighbor 10.10.10.10 peer-group metallb
       neighbor 10.10.10.10 remote-as 64511
       neighbor 10.10.10.10 description Kube Master
       neighbor 10.10.10.11 peer-group metallb
       neighbor 10.10.10.11 remote-as 64511
       neighbor 10.10.10.12 peer-group metallb
       neighbor 10.10.10.12 remote-as 64511
       neighbor 10.10.10.13 remote-as 64511
       !
       address-family ipv4 unicast
        network 192.168.254.0/24
        neighbor 10.10.10.10 activate
        neighbor 10.10.10.11 activate
        neighbor 10.10.10.12 activate
        neighbor 10.10.10.13 activate
        no neighbor metallb send-community
        no neighbor 10.10.10.10 send-community
        no neighbor 10.10.10.11 send-community
        no neighbor 10.10.10.12 send-community
        no neighbor 10.10.10.13 send-community
       exit-address-family
       !
      !
      line vty
      !
      

      Metal LB configuration

      ---
      apiVersion: metallb.io/v1beta2
      kind: BGPPeer
      metadata:
        name: metallb-bgp-peer
        namespace: metallb-system
      spec:
        myASN: 64511
        peerASN: 64512
        peerAddress: 10.10.10.1
        enableGracefulRestart: true
      ---
      apiVersion: metallb.io/v1beta1
      kind: IPAddressPool
      metadata:
        name: address-pool-bgp
        namespace: metallb-system
      spec:
        addresses:
          - 192.168.254.0/24
        autoAssign: true
      ---
      apiVersion: metallb.io/v1beta1
      kind: BGPAdvertisement
      metadata:
        name: metal-lb-bgp-adv
        namespace: metallb-system
      spec:
        ipAddressPools:
        - address-pool-bgp
        aggregationLength: 24
        localPref: 100
      

      an example ingress object and some diagnostic output:

      NAME        CLASS     HOSTS                 ADDRESS         PORTS     AGE
      kuard       traefik   test.mydomain.com        192.168.254.1   80, 443   133d
      
      $ kubectl --namespace metallb-system get bgpadvertisements.metallb.io 
      NAME               IPADDRESSPOOLS         IPADDRESSPOOL SELECTORS   PEERS
      metal-lb-bgp-adv   ["address-pool-bgp"]                             
      $ kubectl --namespace metallb-system get bgppeers.metallb.io 
      NAME               ADDRESS      ASN     BFD PROFILE   MULTI HOPS
      metallb-bgp-peer   10.10.10.1   64512  
      

      Traceroute output:

      $ traceroute 192.168.254.1
      traceroute to 192.168.254.1 (192.168.254.1), 30 hops max, 60 byte packets
       1  _gateway (192.168.1.1)  0.170 ms  0.092 ms  0.075 ms
       2  rpi-cm4-tp2-04.domain.com (10.10.10.13)  0.311 ms  0.357 ms  0.335 ms
       3  10.10.10.1 (10.10.10.1)  0.351 ms  0.461 ms  0.440 ms
       4  rpi-cm4-tp2-04.domain.com (10.10.10.13)  0.609 ms  0.658 ms  0.706 ms
       5  10.10.10.1 (10.10.10.1)  0.607 ms  0.650 ms  0.604 ms
       6  * * *
       7  10.10.10.1 (10.10.10.1)  0.770 ms  0.764 ms  0.717 ms
       8  * * *
       9  10.10.10.1 (10.10.10.1)  1.250 ms  1.245 ms  1.235 ms
      10  * * *
      11  10.10.10.1 (10.10.10.1)  1.574 ms  1.535 ms  1.565 ms
      12  * * *
      13  10.10.10.1 (10.10.10.1)  1.240 ms  1.250 ms  0.988 ms
      14  * * *
      15  10.10.10.1 (10.10.10.1)  0.994 ms  0.890 ms  0.929 ms
      16  * * *
      17  10.10.10.1 (10.10.10.1)  0.949 ms  0.794 ms  0.970 ms
      18  * * *
      19  10.10.10.1 (10.10.10.1)  1.061 ms  2.817 ms  2.779 ms
      20  * * *
      21  10.10.10.1 (10.10.10.1)  3.028 ms  3.001 ms  2.953 ms
      22  * * *
      23  10.10.10.1 (10.10.10.1)  2.363 ms  2.352 ms  2.514 ms
      24  * * *
      25  10.10.10.1 (10.10.10.1)  2.662 ms  2.645 ms  2.631 ms
      26  * * *
      27  10.10.10.1 (10.10.10.1)  2.689 ms  2.700 ms  2.668 ms
      28  * * *
      29  10.10.10.1 (10.10.10.1)  1.747 ms  1.381 ms  1.128 ms
      30  * * *
      
      1 Reply Last reply Reply Quote 0
      • First post
        Last post
      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.