• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

BGP configuration help

Scheduled Pinned Locked Moved FRR
1 Posts 1 Posters 195 Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A
    asacoco
    last edited by asacoco Jun 8, 2025, 1:05 PM Jun 8, 2025, 1:03 PM

    I'm trying to get BGP working between Metal LB on my Kubernetes cluster and pfSense (2.7.2)/FRR (2.0.2_1).
    My aim is to use the 192.168.254.0/24 network for my Kubernetes load balancer, advertised by metallb via BGP to pfsense, so I can access those addresses from my LAN. The Kubernetes cluster has nodes in the 10.10.10.0/24 network on vlan 254 and my local 192.168.1.0/24 network is on VLAN 10.

    However I've clearly done something wrong because my traffic never reaches my ingress objects. In fact, traceroute to one of the ingress addresses just seems to exhaust the TTL. This was working under VyOS, so I suspect it's a pfsense configuration issue (combined with the fact I have no idea what I'm doing)

    Can anyone spot any obvious config issues with the below? I hope this rambling post makes some sense.

    2d92e7f7-d3f3-4a41-819a-4e5d84764341-Screenshot_20250608_124627-1.png

    pfSense FRR-BGP configuration

    frr defaults traditional
    hostname ahostname
    password redacted123
    ip nht resolve-via-default
    service integrated-vtysh-config
    !
    ip router-id 10.10.10.1
    !
    ip route 192.168.254.0/24 10.10.10.10
    ip route 192.168.254.0/24 10.10.10.11
    ip route 192.168.254.0/24 10.10.10.12
    ip route 192.168.254.0/24 10.10.10.13
    !
    router bgp 64512
     bgp router-id 10.10.10.1
     bgp graceful-shutdown
     no bgp network import-check
     no bgp ebgp-requires-policy
     neighbor metallb peer-group
     neighbor metallb remote-as 64511
     neighbor metallb update-source 10.10.10.1
     neighbor 10.10.10.10 peer-group metallb
     neighbor 10.10.10.10 remote-as 64511
     neighbor 10.10.10.10 description Kube Master
     neighbor 10.10.10.11 peer-group metallb
     neighbor 10.10.10.11 remote-as 64511
     neighbor 10.10.10.12 peer-group metallb
     neighbor 10.10.10.12 remote-as 64511
     neighbor 10.10.10.13 remote-as 64511
     !
     address-family ipv4 unicast
      network 192.168.254.0/24
      neighbor 10.10.10.10 activate
      neighbor 10.10.10.11 activate
      neighbor 10.10.10.12 activate
      neighbor 10.10.10.13 activate
      no neighbor metallb send-community
      no neighbor 10.10.10.10 send-community
      no neighbor 10.10.10.11 send-community
      no neighbor 10.10.10.12 send-community
      no neighbor 10.10.10.13 send-community
     exit-address-family
     !
    !
    line vty
    !
    

    Metal LB configuration

    ---
    apiVersion: metallb.io/v1beta2
    kind: BGPPeer
    metadata:
      name: metallb-bgp-peer
      namespace: metallb-system
    spec:
      myASN: 64511
      peerASN: 64512
      peerAddress: 10.10.10.1
      enableGracefulRestart: true
    ---
    apiVersion: metallb.io/v1beta1
    kind: IPAddressPool
    metadata:
      name: address-pool-bgp
      namespace: metallb-system
    spec:
      addresses:
        - 192.168.254.0/24
      autoAssign: true
    ---
    apiVersion: metallb.io/v1beta1
    kind: BGPAdvertisement
    metadata:
      name: metal-lb-bgp-adv
      namespace: metallb-system
    spec:
      ipAddressPools:
      - address-pool-bgp
      aggregationLength: 24
      localPref: 100
    

    an example ingress object and some diagnostic output:

    NAME        CLASS     HOSTS                 ADDRESS         PORTS     AGE
    kuard       traefik   test.mydomain.com        192.168.254.1   80, 443   133d
    
    $ kubectl --namespace metallb-system get bgpadvertisements.metallb.io 
    NAME               IPADDRESSPOOLS         IPADDRESSPOOL SELECTORS   PEERS
    metal-lb-bgp-adv   ["address-pool-bgp"]                             
    $ kubectl --namespace metallb-system get bgppeers.metallb.io 
    NAME               ADDRESS      ASN     BFD PROFILE   MULTI HOPS
    metallb-bgp-peer   10.10.10.1   64512  
    

    Traceroute output:

    $ traceroute 192.168.254.1
    traceroute to 192.168.254.1 (192.168.254.1), 30 hops max, 60 byte packets
     1  _gateway (192.168.1.1)  0.170 ms  0.092 ms  0.075 ms
     2  rpi-cm4-tp2-04.domain.com (10.10.10.13)  0.311 ms  0.357 ms  0.335 ms
     3  10.10.10.1 (10.10.10.1)  0.351 ms  0.461 ms  0.440 ms
     4  rpi-cm4-tp2-04.domain.com (10.10.10.13)  0.609 ms  0.658 ms  0.706 ms
     5  10.10.10.1 (10.10.10.1)  0.607 ms  0.650 ms  0.604 ms
     6  * * *
     7  10.10.10.1 (10.10.10.1)  0.770 ms  0.764 ms  0.717 ms
     8  * * *
     9  10.10.10.1 (10.10.10.1)  1.250 ms  1.245 ms  1.235 ms
    10  * * *
    11  10.10.10.1 (10.10.10.1)  1.574 ms  1.535 ms  1.565 ms
    12  * * *
    13  10.10.10.1 (10.10.10.1)  1.240 ms  1.250 ms  0.988 ms
    14  * * *
    15  10.10.10.1 (10.10.10.1)  0.994 ms  0.890 ms  0.929 ms
    16  * * *
    17  10.10.10.1 (10.10.10.1)  0.949 ms  0.794 ms  0.970 ms
    18  * * *
    19  10.10.10.1 (10.10.10.1)  1.061 ms  2.817 ms  2.779 ms
    20  * * *
    21  10.10.10.1 (10.10.10.1)  3.028 ms  3.001 ms  2.953 ms
    22  * * *
    23  10.10.10.1 (10.10.10.1)  2.363 ms  2.352 ms  2.514 ms
    24  * * *
    25  10.10.10.1 (10.10.10.1)  2.662 ms  2.645 ms  2.631 ms
    26  * * *
    27  10.10.10.1 (10.10.10.1)  2.689 ms  2.700 ms  2.668 ms
    28  * * *
    29  10.10.10.1 (10.10.10.1)  1.747 ms  1.381 ms  1.128 ms
    30  * * *
    
    1 Reply Last reply Reply Quote 0
    1 out of 1
    • First post
      1/1
      Last post
    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
      This community forum collects and processes your personal information.
      consent.not_received