OpenVPN fails with 2.50

JKnott

Here are the relevant parts of the server and client logs:

Server:
Feb 25 10:04:24 firewall openvpn[88273]: 99.245.217.240:32854 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 25 10:04:24 firewall openvpn[88273]: 99.245.217.240:32854 TLS Error: TLS handshake failed
Feb 25 10:04:34 firewall openvpn[38758]: WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
Feb 25 10:04:34 firewall openvpn[38758]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Feb 25 10:04:34 firewall openvpn[38758]: TCP/UDP: Preserving recently used remote address: [AF_INET]172.16.0.1:1194
Feb 25 10:04:34 firewall openvpn[38758]: UDPv4 link local: (not bound)
Feb 25 10:04:34 firewall openvpn[38758]: UDPv4 link remote: [AF_INET]172.16.0.1:1194

Client - OpenSUSE 15.2 network manager
2021-02-25T10:04:23.967189-05:00 E520 NetworkManager[1467]: <warn>
[1614265463.9636]
vpn-connection[0x5649aad226e0,58407612-4bce-4c1b-9421-3dcc15e7f6e7,"firewall-UDP4-1194-E520-config",0]:
VPN connection: connect timeout exceeded.
2021-02-25T10:04:23.971490-05:00 E520 nm-openvpn-serv[5314]: Connect
timer expired, disconnecting.
2021-02-25T10:04:23.974072-05:00 E520 nm-openvpn[5317]: SIGTERM[hard,]
received, process exiting
2021-02-25T10:04:23.974457-05:00 E520 NetworkManager[1467]: <warn>
[1614265463.9729]
vpn-connection[0x5649aad226e0,58407612-4bce-4c1b-9421-3dcc15e7f6e7,"firewall-UDP4-1194-E520-config",0]:
VPN plugin: failed: connect-failed (1)
2021-02-25T10:04:23.974809-05:00 E520 NetworkManager[1467]: <info>
[1614265463.9730]
vpn-connection[0x5649aad226e0,58407612-4bce-4c1b-9421-3dcc15e7f6e7,"firewall-UDP4-1194-E520-config",0]:
VPN plugin: state changed: stopping (5)
2021-02-25T10:04:23.975133-05:00 E520 NetworkManager[1467]: <info>
[1614265463.9730]
vpn-connection[0x5649aad226e0,58407612-4bce-4c1b-9421-3dcc15e7f6e7,"firewall-UDP4-1194-E520-config",0]:
VPN plugin: state changed: stopped (6)

As I mentioned earlier, it will connect when the client is on the LAN side of pfsense, but not WAN side. The client is OpenVPN 2.5.1, but I had similar results with 2.4.3.

bennyc

@jknott
If you can connect on LAN, I would think the server config is fine?
Wondering now what did you set in the gui, did you change interface from WAN to LAN for that test or is it set to Any (and thus it listens on all if's)?
I've upgraded 5d ago to 2.5.0, and have 2 openvpn instances running without issues, where one serves a "Peer to Peer" for a pfSense still running 2.4.x. also with only certs and no user auth. No change has been done on that one, it reconnected after upgrade so at least we (I) know it can work...

Did you try rebooting?

JKnott

@bennyc said in OpenVPN fails with 2.50:

If you can connect on LAN, I would think the server config is fine?

My thought too.

Wondering now what did you set in the gui, did you change interface from WAN to LAN for that test or is it set to Any (and thus it listens on all if's)?

It's set to WAN, but I have tried any. No difference

Did you try rebooting?

Several times.

dotdash

@jknott
Just curious, but has your only test been off the same CPE? I've seen strange behavior between clients on the same ISP (Cough... Comcast)
Maybe verify from somewhere external.

JKnott

@dotdash

I have 2 IPv4 addresses from my ISP. With my notebook plugged into the same modem as my firewall, I get a different address. So, it would appear as though it was another customer with my ISP. BTW, I have used this setup for years, for testing. However, I just tried tethering with my cell phone and that seems to work. I wonder what's different with 2.5.0, as this using my 2nd IPv4 address worked with the pfsense 2.4.5, as I have done many times. My firewall rule for OpenVPN hasn't changed since Sept. 19, 2017 and the source is any. Also, why is it allowing a connection from the LAN?

jimp

Make sure your default gateway is set to your WAN gateway on System > Routing
Try setting the OpenVPN server Protocol value to "UDP IPv4 and IPv6 on all interfaces (multihome)"

JKnott

@jimp

I have only used IPv4 for the UDP packets, as I run into DNS issues, due to the IPv4 host name being an alias and IPv6 being an actual address. Regardless, this worked reliably until I updated to pfsense 2.5.0. I can ping pfsense from my notebook and both the OpenVPN logs and Packet Capture show the connection is being attempted. The gateway is correct as it has been since I started using pfsense 5 years ago. Connecting the VPN in this manner most recently worked the day before I updated to 2.5.0, when I was testing for an issue with the Windows client. I normally use the VPN with Linux.

Gertjan

Wow, detail are not missing here ....

This is what I have :

The settings :

.....

Ok, cut in half :

When exporting :

Note : I do accept the new "2.5.0 OpenVPN stuff".

When I export, I only use one of these button :

Result : using the iPhone OpenVPN Connect App (the official one ?!) it connects using IPv4 - IP ok - also received an IPv6 (correct address) but IPv6 is out.
I can live with that.

@home, I use the OpenVPN connect version 2.4.9 or 2.4.10 (have to check - it from last year) NOT the latest 2.5.1 from yesterday or so (and yes, shame or Netgate for not pushing that one out the minute it came out of the oven, I'm with you here )

It connects with any (or just one) warnings just fine !
IPv4 and IPv6 are up !

My OpenVPN servers bind to my WAN = 192.168.10.3 - I have an upstream ISP router with its NAT rule stuff for VPN on port 1194-UDP.
"Works for yers" .

Now this seems strange. I presumed that I needed to upgrade at least my Windows 10 home OpenVPN connect App, but no ... it was working and kept on working.

Btw : my routes :

I never use my VPN client facilities (I do have a xxxxVPN to play with) - they are just there so I can fool around with it so I can maybe once understand some one wants to tunnel my already TLS encrypted traffic.
I'm using he.net for my IPv6 because, well, you know why.

edit : Maybe I should raise my cyphering. Dono.
The VPN is just for me so I can 'do things at work' while not being at work.

JKnott

@gertjan

The VPN is working, just not when I use my 2nd IPv4 address. As I mentioned above, it does work when my notebook is tethered to my phone. My ISP provides 2 IPv4 addresses, which are not in the same /23 subnet and I normally use the 2nd address for testing. Testing this way has worked for years, until I updated to pfsense 2.5.0. I'd rather not have to tether to my phone for testing, but it appears I may have to.

What changed in pfsense or OpenVPN to cause this?

jimp

@jimp said in OpenVPN fails with 2.50:

"UDP IPv4 and IPv6 on all interfaces (multihome)"

Set that and try it again -- doesn't matter why you have the settings you have now or if it worked before.

JKnott

@jimp

It still fails when I use my second IPv4 address.

JKnott

@jknott

I just tried with ssh and the same thing happens. If I use my 2nd IPv4 address, Packet Capture shows the attempt, but no connection. I can connect with ssh if I tether to my phone. This shows the problem is with FreeBSD and not OpenVPN.

Here's what ifconfig shows:
inet 99.246.125.121 netmask 0xfffffe00 broadcast 255.255.255.255

This shows the appropriate /23 subnet mask. My 2nd address (99.245.223.190) is also a /23, but in a different subnet.

On the other hand, ping works in both directions. What is it that's causing this issue with OpenVPN & ssh?

JKnott

@jknott

Here's a packet capture, showing ping, OpenVPN & ssh (along with some other junk) from my notebook computer to pfsense, using my 2nd IPv4 address. Ping works, but OpenVPN & ssh fail.packetcapture (7).cap Both OpenVPN and ssh work if I tether through my phone.

jimp

If it is in that second subnet only and affects things other than OpenVPN, then it isn't specific to OpenVPN, so the thread title and category here need to be adjusted.

In that packet capture the packet that fails is the largest, so perhaps there is an interface MTU or MSS issue at play. Not sure why it would affect just that one subnet, but it's the first place I'd look.

JKnott

@jimp

Well, it works fine if I tether to my phone, instead of using the 2nd address and it worked for years with the 2nd address, until I updated to 2.5.0.

I agree it's something beyond OpenVPN that also affects ssh. Also, I don't know how it could be an MTU or MSS issue, when both interfaces are connected to the same modem, with the same ISP. I just checked the MTU of the ssh session and it shows 1280 on the notebook end (I don't know why that is), which shouldn't be a problem for pfsense, which has 1500 MTU. Also, UDP doesn't use MTU. Why is pfsense not properly responding to OpenVPN or ssh? It seems to start, then nothing. I also tried with the Windows client. With it, it appeared to connect, but there was no tunnel shown in ipconfig and it couldn't pass any data. That capture was taken with Packet Capture, which means pfsense is receiving, but for some reason failing the connection attempts, but was responding to pings.

As for the 1280 MTU, that's the minimum allowed for IPv6. For the first six years I had IPv6 I was using a tunnel, which forced 1280 MTU. That was never an issue for anything I'm aware of.

jimp

UDP doesn't use MSS but it does respect MTU.

It does respond but once the packets are sufficiently large it fails, which is a common problem associated with MTU issues.

As to why it's different for another subnet, I don't know, that's likely due to differences in your modem/ISP but it's tough to say without more info.

It may work from your phone because your phone connection probably has an MTU lower than whatever is causing the problem.

JKnott

@jimp

UDP respects MTU, but doesn't communicate it with the other end the way TCP does. In looking through the packet capture, I don't see anything longer than 1188 bytes, so even that is below the 1280 MTU.

As for tethering to my phone, the MTU that way is 1500.

I doubt my ISP changed anything when I updated pfsense. I used the VPN, through the 2nd address the day before I updated, as I had for years. I've had this particular modem for a few months, since I updated to IPTV, but have had at least two other modems in the time I've been using the 2nd address.

I never noticed that 1280 MTU before, as I never had any reason to check it. However, it's my notebook computer that gets it, not pfsense, which has 1500, so any UDP coming from it should have already been limited by it. TCP would negotiate the MTU used accordingly.

I wonder if there's anyone else here who can try this, with the 2nd address. I'm on Rogers.

johnpoz

@jknott said in OpenVPN fails with 2.50:

which are not in the same /23 subnet and I normally use the 2nd address for testing

So your having to hairpin up to your ISP to get to your pfsense, from your laptop..

You can see ssh answering there for example - sniffing on your laptop I take you never get those.. So it really has nothing to do with pfsense or your laptop. But your ISP.

JKnott

@johnpoz

As I have said several times, this worked well for several years until I upgraded to 2.5.0. In fact there have been times I mentioned to you that I was doing that. You may recall a topic I started recently about the Windows client not working. While that problem had nothing to do with this issue, at that time I was on 2.4.5 and the Linux client worked and continued to work until 2.5.0 You can see from the packet capture that the connection is started, with responses from pfsense, but suddenly stops. This tells me that the ISP is not blocking anything.

While the capture I provided was from pfsense, I see the same thing with Wireshark on my notebook. That is I am seeing the limited response from pfsense. Again, the ISP/modem is not blocking anything.

BTW, the 2.5.0 Windows client doesn't have the problem I mentioned.

JKnott

@johnpoz

BTW, WRT hairpinning, since it's different subnets, it wouldn't be happening in the modem. It would have to go back to the head end. How would this be any different than a neighbour on the same ISP doing it?