ipsec tunnels load balancing issue

jazzl0ver

Hi,

There're multiple ipsec tunnels in VTI mode between two pfsense routers. FRR package is used to exchange routes (BGP) and health checks (BFD). pfSense A announces 172.26.0.0/16 and sends traffic to the pfSense B using a gateway group fulfilled with all ipsec gateways set as Tier 1. pfSense B does announce nothing to pfSense A.

Hence, if I sniff traffic on pfSense B's ipsec interfaces I see incoming traffic from pfSense A on all of them. However, the returning traffic appears on ipsec1 only. Is there a way to load balance it between all ipsec tunnels?

Here are some tech details:

pfsenseB# vtysh
pfsenseB# show ip route
...
B>* 172.26.0.0/16 [20/0] via 10.6.106.2, ipsec1, weight 1, 00:06:49
  *                      via 10.6.106.6, ipsec2, weight 1, 00:06:49
  *                      via 10.6.106.10, ipsec3, weight 1, 00:06:49
...
pfsenseB# exit

pfsenseB# netstat -rn | grep 172.26
172.26.0.0/16      10.6.106.2         UG1      ipsec1
pfsenseB#

In other words, why is there a single route only in the netstat output when FRR routing table has all 3 routes with the same wight?

Thanks in advance!

jazzl0ver

bump. still need help on this

michmoor

@jazzl0ver BGP does not load balance by default. It only selects one best path based on the bgp path attributes.
You would need to use BGP Multipath. I think its using the maximum-paths command but you need to research.

jazzl0ver

@michmoor thank you for your reply! just checked and it does not seem to help.
after googling for that keyword I've found that it's not yet enabled in 2.6.0:
https://forum.netgate.com/topic/159714/frr-gui-maximum-paths

pity :(

michmoor

@jazzl0ver maybe not in the GUI but you can do it via the shell which i would recommend.
/var/etc/frr/frr.conf

The gui exposes some of the features of frr but to do the tweaks like multi-path you need to update the conf file.

These options ticked dont work for you?

edit - the redmine you linked says its resolved. If it isnt i suggest re-opening

jazzl0ver

@michmoor yeah, i tried to add maximum-paths via vtysh along with the changes you proposed in the WUI (not sure multipath relax makes sense here, b/c the AS path is the same) - no success:

pfSenseB# vtysh
...
router bgp 135
 bgp router-id x.y.169.90
 bgp bestpath as-path multipath-relax
 bgp bestpath compare-routerid
 no bgp network import-check
 neighbor 10.6.106.2 remote-as 172
 neighbor 10.6.106.2 description c1
 neighbor 10.6.106.2 bfd
 neighbor 10.6.106.2 bfd check-control-plane-failure
 neighbor 10.6.106.2 update-source 10.6.106.1
 neighbor 10.6.106.6 remote-as 172
 neighbor 10.6.106.6 description c2
 neighbor 10.6.106.6 bfd
 neighbor 10.6.106.6 bfd check-control-plane-failure
 neighbor 10.6.106.6 update-source 10.6.106.5
 neighbor 10.6.106.10 remote-as 172
 neighbor 10.6.106.10 description c3
 neighbor 10.6.106.10 bfd
 neighbor 10.6.106.10 bfd check-control-plane-failure
 neighbor 10.6.106.10 update-source 10.6.106.9
 !
 address-family ipv4 unicast
  no neighbor 10.6.106.2 send-community
  neighbor 10.6.106.2 soft-reconfiguration inbound
  neighbor 10.6.106.2 weight 100
  neighbor 10.6.106.2 prefix-list allow-all in
  neighbor 10.6.106.2 prefix-list allow-all out
  no neighbor 10.6.106.6 send-community
  neighbor 10.6.106.6 soft-reconfiguration inbound
  neighbor 10.6.106.6 weight 100
  neighbor 10.6.106.6 prefix-list allow-all in
  neighbor 10.6.106.6 prefix-list allow-all out
  no neighbor 10.6.106.10 send-community
  neighbor 10.6.106.10 soft-reconfiguration inbound
  neighbor 10.6.106.10 weight 100
  neighbor 10.6.106.10 prefix-list allow-all in
  neighbor 10.6.106.10 prefix-list allow-all out
  maximum-paths 3
 exit-address-family
!
ip prefix-list allow-all seq 10 permit any
...

# netstat -rn|grep ipsec
10.6.106.2         link#6             UH       ipsec1
10.6.106.6         link#8             UH       ipsec2
10.6.106.10        link#9             UH       ipsec3
172.26.0.0/16      10.6.106.2         UG1      ipsec1

This redmine issue does not seem to be resolved: https://redmine.pfsense.org/issues/9545

michmoor

@jazzl0ver ahhh ok not available in the kernel. That makes sense.