2.4.5 <-> Virtual IP on WAN CARP address == broken UDP OpenVPN ?
-
Has anyone running pfSense v2.4.5 got an OpenVPN tunnel to remain stable when using:
UDP OpenVPN (site-to-site) on top of
Virtual IP Address (IP Alias) on top of
CARP (WAN) on top of
LAGG (failover)I have been investigating an issue for a few weeks and blaming one of our ISPs. Some testing this morning as confirmed this might be a problem with pfSense.
Thanks
.mt -
Just giving this a bump and adding some more information.
I have not been able to get an OpenVPN UDP tunnel to remain stable when either the client or the server is bound to a virtual IP (VIP) which is sitting on a CARP interface. I have not tested this with LAGGed and standalone interfaces.
The tunnel will oscillate between working and not working. We typically see no traffic passing from the client to the server. Then we see
Inactivity timeout (--ping-restart), restartingrecorded in the logs (on either the client or the server). The OpenVPN process is restarted and the connection works for a short but variable period of time until once again, no traffic is passed from the client to the server.We have three different locations and I have reproduced this at each of them, on pfSense
2.4.5and2.4.5 p1.I believe I have also reproduced it in a lab (virtual) with
2.4.5,2.4.5 p1and2.5.0. The presentation seems slightly different because pinging a VIP which is running on top of a CARP interface (no VPN involved) will also result in 5-6 pings getting dropped in succession. The VPN presentation is pretty much the same.As soon as we we either:
- switch to using a CARP IP address (rather than Virtual IP)
- switch to using a Virtual IP on a non-CARP interface (only tested in the lab)
- switch to using TCP
The problem is resolved.
We still see ~
1.4-2.2% packet loss when we use TCP connection which is high enough to affect applications that depend on the VPN connection.The configuration of the OpenVPN tunnel seems to have no bearing on the problem (hardware crypto acceleration, ciphers, algorthms etc.)
I'm at this stage tempted to file a bug report, but I'm wondering if anyone else is able to test/confirm the issue before I do.
Thanks
.mt -
Finally worked out what is happening here.
If an OpenVPN client is bound to a Virtual IP which is an IP Alias for a CARP IP, the OpenVPN Client gets started even when the parent CARP interface is backup.
It seems under UDP, in a peer to peer arrangement, this client periodically steals the tunnel.
In a TCP arrangment, this clients attempts to connect to the server timeout but I figure there must be some part of the OpenVPN server process that blocks as a result causing packets to occasionally be dropped.
Reproduced on pfSense 2.4.5, 2.4.5 p1 and 2.5 on a range of disparate hardware in both lab and real world conditions.
I'll be raising a bug report once I've had a look at the code.
-
A bug for the issue has been raised.