ipsec tunnels load balancing issue

[jazzl0ver]

Hi,

There're multiple ipsec tunnels in VTI mode between two pfsense routers. FRR package is used to exchange routes (BGP) and health checks (BFD). pfSense A announces 172.26.0.0/16 and sends traffic to the pfSense B using a gateway group fulfilled with all ipsec gateways set as Tier 1. pfSense B does announce nothing to pfSense A.

Hence, if I sniff traffic on pfSense B's ipsec interfaces I see incoming traffic from pfSense A on all of them. However, the returning traffic appears on ipsec1 only. Is there a way to load balance it between all ipsec tunnels?

Here are some tech details:

pfsenseB# vtysh
pfsenseB# show ip route
...
B>* 172.26.0.0/16 [20/0] via 10.6.106.2, ipsec1, weight 1, 00:06:49
  *                      via 10.6.106.6, ipsec2, weight 1, 00:06:49
  *                      via 10.6.106.10, ipsec3, weight 1, 00:06:49
...
pfsenseB# exit

pfsenseB# netstat -rn | grep 172.26
172.26.0.0/16      10.6.106.2         UG1      ipsec1
pfsenseB#

In other words, why is there a single route only in the netstat output when FRR routing table has all 3 routes with the same wight?

Thanks in advance!

[jazzl0ver]

bump. still need help on this

[michmoor]

@jazzl0ver BGP does not load balance by default. It only selects one best path based on the bgp path attributes.
You would need to use BGP Multipath. I think its using the maximum-paths command but you need to research.

[jazzl0ver]

@michmoor thank you for your reply! just checked and it does not seem to help.
after googling for that keyword I've found that it's not yet enabled in 2.6.0:
https://forum.netgate.com/topic/159714/frr-gui-maximum-paths

pity :(

[michmoor]

@jazzl0ver maybe not in the GUI but you can do it via the shell which i would recommend.
/var/etc/frr/frr.conf

The gui exposes some of the features of frr but to do the tweaks like multi-path you need to update the conf file.

These options ticked dont work for you?

edit - the redmine you linked says its resolved. If it isnt i suggest re-opening

[jazzl0ver]

@michmoor yeah, i tried to add maximum-paths via vtysh along with the changes you proposed in the WUI (not sure multipath relax makes sense here, b/c the AS path is the same) - no success:

pfSenseB# vtysh
...
router bgp 135
 bgp router-id x.y.169.90
 bgp bestpath as-path multipath-relax
 bgp bestpath compare-routerid
 no bgp network import-check
 neighbor 10.6.106.2 remote-as 172
 neighbor 10.6.106.2 description c1
 neighbor 10.6.106.2 bfd
 neighbor 10.6.106.2 bfd check-control-plane-failure
 neighbor 10.6.106.2 update-source 10.6.106.1
 neighbor 10.6.106.6 remote-as 172
 neighbor 10.6.106.6 description c2
 neighbor 10.6.106.6 bfd
 neighbor 10.6.106.6 bfd check-control-plane-failure
 neighbor 10.6.106.6 update-source 10.6.106.5
 neighbor 10.6.106.10 remote-as 172
 neighbor 10.6.106.10 description c3
 neighbor 10.6.106.10 bfd
 neighbor 10.6.106.10 bfd check-control-plane-failure
 neighbor 10.6.106.10 update-source 10.6.106.9
 !
 address-family ipv4 unicast
  no neighbor 10.6.106.2 send-community
  neighbor 10.6.106.2 soft-reconfiguration inbound
  neighbor 10.6.106.2 weight 100
  neighbor 10.6.106.2 prefix-list allow-all in
  neighbor 10.6.106.2 prefix-list allow-all out
  no neighbor 10.6.106.6 send-community
  neighbor 10.6.106.6 soft-reconfiguration inbound
  neighbor 10.6.106.6 weight 100
  neighbor 10.6.106.6 prefix-list allow-all in
  neighbor 10.6.106.6 prefix-list allow-all out
  no neighbor 10.6.106.10 send-community
  neighbor 10.6.106.10 soft-reconfiguration inbound
  neighbor 10.6.106.10 weight 100
  neighbor 10.6.106.10 prefix-list allow-all in
  neighbor 10.6.106.10 prefix-list allow-all out
  maximum-paths 3
 exit-address-family
!
ip prefix-list allow-all seq 10 permit any
...

# netstat -rn|grep ipsec
10.6.106.2         link#6             UH       ipsec1
10.6.106.6         link#8             UH       ipsec2
10.6.106.10        link#9             UH       ipsec3
172.26.0.0/16      10.6.106.2         UG1      ipsec1

This redmine issue does not seem to be resolved: https://redmine.pfsense.org/issues/9545

[michmoor]

@jazzl0ver ahhh ok not available in the kernel. That makes sense.