Multiple OpenVPN Servers Listen on Localhost - Cannot Connect from CARP Interface

[mrnb]

We have a pair of High Availability devices running 23.05.1-RELEASE (amd64).

We run multiple OpenVPN servers that all listen on localhost interface.

We also have multiple WAN connections including a shared CARP WAN connection.

We can connect to the OpenVPN servers using the non-carp WAN connections (Wans 1-3) but cannot connect through the CARP WAN connection (Wan4)

I suspect this may have something to do with the Outbound NAT, but I cannot figure out what the solution is. We currently have Hybrid Outbound NAT, and I did not include the Automatic rules, only the Manual Mappings below.

I could use some guidance for how to make it work. Thank you!

[viragomann]

@mrnb
No, the outbound NAT only comes in play for outbound traffic. It hasn't any effect on inbound.

You can forward traffic destined to the WAN CARP VIP like any other WAN address. So there is nothing different generally.

Since you use the same port alias on all WAN addresses, I assume, all go to the same VPN servers. Hence the server should be well.

To investigate run a packet capture on the CARP interface and check if the OpenVPN packets even arrive there.

[mrnb]

@viragomann

I can confirm that the packets are being received by the WAN CARP IP (note that I replaced WAN IP and Source IP manually:

15:22:03.222156 08:ec:f5:22:9a:0a > 00:00:5e:00:01:13, ethertype IPv4 (0x0800), length 128: (tos 0x0, ttl 61, id 50065, offset 0, flags [DF], proto UDP (17), length 114)
    <SOURCE IP>.10534 > <WAN CARP IP>.49153: [udp sum ok] UDP, length 86
15:22:05.345351 08:ec:f5:22:9a:0a > 00:00:5e:00:01:13, ethertype IPv4 (0x0800), length 128: (tos 0x0, ttl 61, id 50309, offset 0, flags [DF], proto UDP (17), length 114)
    <SOURCE IP>.10534 > <WAN CARP IP>.49153: [udp sum ok] UDP, length 86
15:22:09.588182 08:ec:f5:22:9a:0a > 00:00:5e:00:01:13, ethertype IPv4 (0x0800), length 128: (tos 0x0, ttl 61, id 50943, offset 0, flags [DF], proto UDP (17), length 114)
    <SOURCE IP>.10534 > <WAN CARP IP>.49153: [udp sum ok] UDP, length 86
15:22:17.219149 08:ec:f5:22:9a:0a > 00:00:5e:00:01:13, ethertype IPv4 (0x0800), length 128: (tos 0x0, ttl 61, id 51867, offset 0, flags [DF], proto UDP (17), length 114)
    <SOURCE IP>.10534 > <WAN CARP IP>.49153: [udp sum ok] UDP, length 86
15:22:34.106623 08:ec:f5:22:9a:0a > 00:00:5e:00:01:13, ethertype IPv4 (0x0800), length 128: (tos 0x0, ttl 61, id 53979, offset 0, flags [DF], proto UDP (17), length 114)
    <SOURCE IP>.10534 > <WAN CARP IP>.49153: [udp sum ok] UDP, length 86

Here's the openvpn log showing that a connection is attempted but fails:

Oct 7 17:47:12	openvpn	37877	openvpn server 'ovpns1' user 'openvpnuser' address '<SOURCE IP>:21587' - disconnected
Oct 7 17:47:12	openvpn	32630	openvpnuser/<SOURCE IP>:21587 [openvpnuser] Inactivity timeout (--ping-restart), restarting
Oct 7 17:14:20	openvpn	10773	openvpn server 'ovpns1' user 'openvpnuser' address '<SOURCE IP>:21587' - connected
Oct 7 17:14:20	openvpn	8305	openvpn server 'ovpns1' user 'openvpnuser' address '<SOURCE IP>:21587' - connecting
Oct 7 17:14:20	openvpn	32630	openvpnuser/<SOURCE IP>:21587 MULTI_sva: pool returned IPv4=10.100.0.2, IPv6=(Not enabled)
Oct 7 17:14:20	openvpn	94819	user 'openvpnuser' authenticated
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 [openvpnuser] Peer Connection Initiated with [AF_INET]<SOURCE IP>:21587
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_TCPNL=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_COMP_STUBv2=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_COMP_STUB=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_LZO=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_LZ4v2=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_LZ4=1
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_CIPHERS=AES-256-GCM:AES-128-GCM
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_NCP=2
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_PROTO=6
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_PLAT=linux
Oct 7 17:14:19	openvpn	32630	<SOURCE IP>:21587 peer info: IV_VER=2.5.5

[viragomann]

@mrnb
Looks like a normal connection. The use connects and disconnects later, probably self-initiated.

However, as the user 'openvpnuser' looks very common, I'm wondering if the same user (not client) connects multiple times concurrently to a single server, possible to different WANs.
This would require that you check "Duplicate Connection" in the server settings.

[mrnb]

@viragomann
I changed the name of the user in the logs to post on the forum. It is not a duplicate connection, as I confirmed only a single client was connected to this server.

[mrnb]

@mrnb
After some more investigation I noticed that I cannot do the following:

Port forward to an internal IP using the CARP VIP, but I can do it through a non-CARP WAN address.

So there is some sort of issue that is unrelated to OpenVPN, but I cannot figure out how to fix it.

I made sure to add the following outbound NAT rule using Hybrid NAT:

[viragomann]

@mrnb
As mentioned above, the outbound NAT has not any impact on inbound traffic.

However, as for any Multi-WAN set up, you have to ensure that the inbound traffic is passed by a rule on the respective interface tab.
There must no floating pass rule and no one on an interface group match this traffic!

[mrnb]

@viragomann

Ok I see now.

Thank you, I did find a NAT rule that took over almost the whole UDP port range and was obviously interfering with other inbound traffic.

After re-configuring the service in question and reducing the Inbound UDP port range to something more reasonable, I was able to resolve the OpenVPN connection issue.