unexpected multiple routes

digininja99

I've got pfSense setup with two OpenVPN clients as follows:

Remote subnet 10.5.1.0/24 with tunnel IPs 10.8.0.5 and 10.8.0.6
Remote subnet 10.6.1.0/24 with tunnel IPs 10.3.0.21 and 10.3.0.22

This is my routing table:

Destination        Gateway            Flags         Netif Expire
default            77.97.0.1          UGS    mvneta0.4090
10.3.0.1           10.3.0.21          UGHS         ovpnc1
10.3.0.21          link#13            UH           ovpnc1
10.3.0.22          link#7             UHS             lo0
10.5.1.0/24        10.8.0.5           UGS          ovpnc2
10.6.1.0/24        10.3.0.21          UGS          ovpnc1
10.8.0.1           10.8.0.5           UGHS         ovpnc2
10.8.0.5           link#14            UH           ovpnc2
10.8.0.6           link#7             UHS             lo0
xx/20       link#12            U      mvneta0.4090
xx          link#12            UHS    mvneta0.4090
xx         link#7             UHS             lo0
127.0.0.1          link#7             UH              lo0
192.168.0.0/24     link#10            U      mvneta0.4091
192.168.0.9        link#7             UHS             lo0

The problem I'm having is when I try to access a host on the 10.5.1.0/24 subnet, half the time it tries to route me via the wrong VPN.

Traceroute is giving two options for routing:

traceroute 10.5.1.1
traceroute to 10.5.1.1 (10.5.1.1), 64 hops max, 40 byte packets
 1  10.3.0.1 (10.3.0.1)  16.771 ms
    10.8.0.1 (10.8.0.1)  22.657 ms
    10.3.0.1 (10.3.0.1)  15.739 ms
 2  xxx (10.5.1.1)  21.217 ms
    10.207.7.65 (10.207.7.65)  17.556 ms
    xxx (10.5.1.1)  22.450 ms

I've no idea what the IP 10.207.7.65 is

If I ping the IP, the first ping works, try again and it fails, third go it works again:

[24.11-RELEASE][root@firewall.]/root: ping 10.5.1.1
PING 10.5.1.1 (10.5.1.1): 56 data bytes
64 bytes from 10.5.1.1: icmp_seq=0 ttl=63 time=21.396 ms
64 bytes from 10.5.1.1: icmp_seq=1 ttl=63 time=21.201 ms
^C
--- 10.5.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 21.201/21.298/21.396/0.097 ms
[24.11-RELEASE][root@firewall.]/root: ping 10.5.1.1
PING 10.5.1.1 (10.5.1.1): 56 data bytes
^C
--- 10.5.1.1 ping statistics ---
11 packets transmitted, 0 packets received, 100.0% packet loss
[24.11-RELEASE][root@firewall.]/root: ping 10.5.1.1
PING 10.5.1.1 (10.5.1.1): 56 data bytes
64 bytes from 10.5.1.1: icmp_seq=0 ttl=63 time=22.902 ms
64 bytes from 10.5.1.1: icmp_seq=1 ttl=63 time=22.741 ms
^C
--- 10.5.1.1 ping statistics ---
3 packets transmitted, 2 packets received, 33.3% packet loss
round-trip min/avg/max/stddev = 22.741/22.822/22.902/0.081 ms

Based on the routing table I can't see why it has multiple routes to 10.5.1.0/24. It says go via 10.8.0.5 which is on link#14 on interface ovpnc2. The 10.3.0.0/24 subnet is on a different link and interface.

If I disable ovpnc1 then thing work as expected.

Can anyone suggest what I can do to debug or fix this?