FRR - BGP - Multi-Path/ECMP



  • HI All,

    I've got a simple setup that I seem to struggle getting to work correctly.

    I have a single PFSense box running FRR. It is connected to OSPF on the LAN side and BGP peered over 2x VPN connections.

    I can't seem to find a way to enable BGP Multi-Path/Load-Sharing over my 2 VPN links. Anyone know how to do this? Searching seems to have been fruitless. :(

    LAN <-> PFSense <-> 2x VPN Tunnels <-> 2x Remote Routers (Single AS across both Remote Routers)

    BGP via FRR only puts a single route in the routing table. No ECMP.

    Is there a way to enable ECMP?



  • @jchinyou Not sure if this is related, but in order to run BGP, you will have first to install OpenBGPD. Go to the System/Package Manager and search for this package.

    For the rest I cannot help because I am currently struggling to make OpenBGPD work.


  • Galactic Empire

    @christ said in FRR - BGP - Multi-Path/ECMP:

    @jchinyou Not sure if this is related, but in order to run BGP, you will have first to install OpenBGPD. Go to the System/Package Manager and search for this package.

    For the rest I cannot help because I am currently struggling to make OpenBGPD work.

    Nope.

    "FRR routing daemon for BGP, OSPF, and OSPF6 Conflicts with Quagga OSPF and OpenBGPD. These packages cannot be installed at the same time."



  • Hi,
    I can also confirm ecmp doesn't seem to be working. It is probably down to the compile options which I think need to explicitly set the ecmp max number of paths.
    Cheers!
    Jon.



  • Hi!

    I'm one of the maintainers of FRR and was asked about this, from the output I was given pfsense compiles FRR w/out any ecmp settings so the ecmp defaults to 1. If you want a higher value FRR needs to be recompiled with --enable-multipath=X where X is the desired max ecmp allowed.



  • To circle back around, I just spoke with a pfsense developer and was informed that pfsense does not support multipath, so this will never work and as such there is no point in changing the way FRR is compiled.


  • Netgate

    To circle back around again, this is now slated for pfsense 2.5.

    https://redmine.pfsense.org/issues/9544
    https://redmine.pfsense.org/issues/9545



  • @jchinyou I have faced same issue and tried to build from source FRR in pfsense. Finally I confirmed ECMP works well. Let me share just example procedure how to build FRR7.0-stable in pfsense2.4.
    pfSense FRR part in GUI partially doesn't work after installing new FRR. Please be careful.

    #--------------------------------------------------------------------------
    # Install FRR6 via pkg
    # To generate user, group and /usr/local/etc/rc.d/frr
    #--------------------------------------------------------------------------
    pkg install -y frr6
     
    # rc.conf
    cat >> /etc/rc.conf <<EOF
    frr_enable="YES"
    frr_flags=""
    frr_daemons="zebra bgpd"
    frr_vtysh_boot="YES"
    EOF
     
    # generage a file
    touch /usr/local/etc/frr/vtysh.conf
    touch /usr/local/etc/frr/frr.conf
     
    # restart
    service frr restart
    service frr status
    vtysh -c "show version"
    
    
    #--------------------------------------------------------------------------
    # Reference
    # http://docs.frrouting.org/projects/dev-guide/en/latest/building-frr-for-freebsd11.html
    #--------------------------------------------------------------------------
    # Enable FreeBSD repository
    cat /etc/pkg/FreeBSD.conf > /usr/local/etc/pkg/repos/FreeBSD.conf
    sed -ie "s/{ enabled: no }/{ enabled: yes }/g" /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf
    pkg update
     
    # Install package
    pkg install -y git
    pkg install -y autoconf automake libtool gmake
    pkg install -y json-c pkgconf bison flex
    pkg install -y c-ares texinfo texinfo
    pkg install -y py36-pytest py36-sphinx python36
    pkg install -y cmake llvm gcc debhelper libyang
    # gcc is needed to install build tool like "ld".
    
    #--------------------------------------------------------------------------
    # Workaround for build
    #--------------------------------------------------------------------------
    #/bin/sh: python: not found
    #gmake: *** [Makefile:9515: tests/tests.xml] Error 127
    ln -s /usr/local/bin/python3.6 /usr/local/bin/python
    python --version
    # > Python 3.6.8
    
    # To pass gmake check
    # Use clang instead of "gcc". PATH should be "cc"
    ln -s /usr/local/bin/clang /usr/bin/cc
    cc --version
    # > clang version 8.0.0 (tags/RELEASE_800/final)
     
    # Don't use gcc for compiling. use cc (clang8)
    # gmake check will be failed by compiling gcc so use cc(clang8).
    # check frr/config.log after compiling
    rm /usr/local/bin/gcc
    
    # Copy C-header files from another FreeBSD11 to pfsense
    # on FreeBSD11
    scp /usr/include/*.h root@PFSENSE:/usr/include
    scp -r /usr/include/sys/ root@PFSENSE:/usr/include/
    scp -r /usr/include/machine/ root@PFSENSE:/usr/include/
    scp -r /usr/include/x86/ root@PFSENSE:/usr/include/
    scp -r /usr/include/xlocale root@PFSENSE:/usr/include/
    scp -r /usr/include/netinet root@PFSENSE:/usr/include/
    scp -r /usr/include/netinet6 root@PFSENSE:/usr/include/
    scp -r /usr/include/net root@PFSENSE:/usr/include/
    scp -r /usr/include/arpa root@PFSENSE:/usr/include/
    
    #--------------------------------------------------------------------------
    # Build FRR
    #--------------------------------------------------------------------------
    cd /root
    git clone https://github.com/frrouting/frr.git frr
    cd frr
    git branch -a | grep 7.0
    git checkout -t origin/stable/7.0
    git branch
    ./bootstrap.sh
    setenv MAKE gmake
    setenv LDFLAGS -L/usr/local/lib
    setenv CPPFLAGS -I/usr/local/include
    ln -s /usr/local/bin/sphinx-build-3.6 /usr/local/bin/sphinx-build
    ./configure \
        --sysconfdir=/usr/local/etc/frr \
        --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
        --localstatedir=/var/run/frr \
        --prefix=/usr/local \
        --enable-multipath=32 \
        --enable-user=frr \
        --enable-group=frr \
        --enable-vty-group=frrvty \
        --enable-configfile-mask=0640 \
        --enable-logfile-mask=0640 \
        --enable-fpm \
        --with-pkg-git-version \
        --with-pkg-extra-version=-compiled
     
    # Compile
    gmake -j4
    gmake check
     
    # Install
    gmake install
     
    # restart
    service frr restart
    service frr status
    vtysh -c "show version"
    
    # Check
    vtysh -c "show ip bgp ipv4 unicast"
    #   Network          Next Hop            Metric LocPrf Weight Path
    #*= 0.0.0.0/0       xxxx
    #*>                       xxxx
    

  • Rebel Alliance Developer Netgate

    That is an awful way to go about things. Installing a compiler environment on the firewall is a massive security no-no. Spin up a FreeBSD VM and just compile it from ports without all the hoop jumping.



  • @jimp I didn't know that compiler environment has security risk. I will only use compiled binary which is copied from FreeBSD to pfsense.


Log in to reply