IPSEC VTI tunnels lost packets
-
@pete35 I agree, this is the reason why I am looking for link stability, because if not implementing OSPF over unstable links, it might be the closest thing to hell...
-
The timemarks on your multiple sa's is showing a reinstall /rekey every 20 seconds. 3600 seconds would be fine, but it doesnt do that. Please check your SA situation again.
-
About OSPF, not only unstable links are causing interrupts, even simple adding or removing a route is causing an interrupt in the whole OSPF system, all pfsense routing devices will renew their routing table per restart ... , not only the OSPF routes will dissappear also any other routes in tne routing table. Interrupt time depends on settings but around 30 seconds is quite usual. If your applications can survive that, you are lucky.
-
@pete35 Perfect
-
@pete35 Yes, this is a issue, our intention, to minimize these situations is to increase the number of areas and play with death times and execution timeouts in order to minimize this problems.
The main objective of the project is to offer high availability between two IPSEC tunnels, what we would like to do is have one IPSEC on WAN1 and another IPSEC (with the same destination) on WAN2. There are more reasons why we would like to implement dynamic routes, but it could be a separate topic.
Do you recommend another solution?
-
If you want HA with multiwan you may look for Gatewaygroups with a usually straightforward painless implementation. But this only works with a low number of routes. How many sites do you have?
Is the packetloss gone now? If not, you may post all your detailed configurations for the two sites, to check again all the parameters.
-
@pete35 Hi Pete, we are looking for, more than a multiwan solution, a high availability IPSEC solution. Another option that we are evaluating is to put P2 VTIs and configurate gateway groups within the routes that go within the VTIs.
Schema:
To the previous scheme you have to add 2 WANs for each site and in each site there are 2 Pfsense in HA. The main problem is that they want point-to-point connection between all the sites and we want to avoid it with some intermediate jump.
On the other hand, the tunnel maintains the same level of packet loss, although we have improved regarding the stability of the SAs
P1 Configuration
P2 Configuration
-
As for my experience IPSEC/VTI/OSPF together with pfsense HA isnt a reliable solution at all. Each of these functionalities have their problems within pfsense and together it will be a mess. Maybe you look for less functionality or choose another Router.
As for the packet loss: Want kind of Hardware do you use, APUs are fine, but you need the right BIOS for them, please dont test with pfsense as VM's. Please configure exactly along the pfsense documentation: DPD is missing, you dont choose a hash in P2 ( which shouldnt be a problem, but who knows), configure MTU clamping to 1400 and the rekey/reauth functionality as shown above. Do the same on the other side and rule out any double NAT with the connection to the internet. Show your configuration for both sides in detail - it must match. Next show the routing tables, and the syslog for ipsec. Dont use VM's for pfsense, Netgate devices are prefered.
If you have configured all above correctly and if there is more packet loss, show the ping logs and some wireshark packetcaptures from WAN as we can rule out pfsense at that point.
I have several vti tunnels with OSPF up and running and if configured correctly, there is no packet loss.
-
@pete35 Hi Pete, answers:
-We work with several versions of APU, 2,3 and 4 all with 4 GB of RAM.
-We only have one pfsense as VM in the architecture and coincidentally the VTIs that go against that firewall are the ones that lose the least packets.
-DPD is set and is not set to test. No change.
-Hash in P2. Remember that we are changing from tunnel mode to VTI mode and we are using the same configuration, in tunnel mode it works perfectly (hardware included) and in VTI mode there is packet loss.
-MTU clamping set at 1400. Let's see if the behavior changes.
-Pfsense has only one NAT to go internet.
-We are going to capture traffic to see if we see where the ICMPs are lost.
-Remember that we do not have OSPF mounted (yet), packet losses always occur, even in models reinstalled with dedicated hardware, in fact, we have now bought 6 new computers to do tests in the laboratory.Thanks for your tips.
-
did you resolve this issue?
we had set up some ipsec VTI were seeing random SMB connection issues over VPN. it seems this was/is the root causeIPSec in transport mode between FreeBSD hosts blackholes TCP traffic
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242744 -
@metisit
A little late on this reply, but for anyone coming across this- that link concerns racoon and not strongswan.
