OpenVPN stops working on change?
-
I was advised to post about this from @sullrich on Twitter.
So, we're running pfSense 1.2.3 using OpenVPN for our VPN to remote sites (we don't have that many). Anyways, one of our remote sites closed down, so I did the following:
-
- System > Static Routes > killed the static route to the remote site
-
- Interfaces > (assign) > killed the the interface and saved
-
- VPN > OpenVPN > Server > killed the config for that site
I was pinging our other sites while doing all this and after killing and saving everything (I wasn't watching for when it happened), I checked the pings and they started timing out then. They were down for a few minutes before I decided to reboot pfSense, after which, connection was reestablished and pings were successful.
Thoughts?
-
-
When you save OpenVPN settings, it restarts the OpenVPN process. If it's a server, the client has to notice the connection is gone and then try to reconnect. The default client timeout in OpenVPN is 60 seconds, at which point the connection should have reestablished.
Are you adding routes for the VPN under System > Static Routes? If so, set them using OpenVPN's advanced options instead of using the system's routes for that. That should have only affected the VPN if you had a conflicting route.
EDIT: Also, do you have the OpenVPN interface assigned as an OPT interface? If so, 1.2.3 doesn't handle that nearly as well as 2.0 does. You'd probably have better luck on a 2.0 snapshot.
-
You made all of these changes simply to bring a tunnel back up?
That is way overkill and is not the right way to do so. You should have only needed to resave the openvpn tunnel that was having issues on either side.
Please post the logs from OpenVPN during the time that the outage occurred so that we can nail down the root of the disconnect.
-
When you save OpenVPN settings, it restarts the OpenVPN process. If it's a server, the client has to notice the connection is gone and then try to reconnect. The default client timeout in OpenVPN is 60 seconds, at which point the connection should have reestablished.
Are you adding routes for the VPN under System > Static Routes? If so, set them using OpenVPN's advanced options instead of using the system's routes for that. That should have only affected the VPN if you had a conflicting route.
EDIT: Also, do you have the OpenVPN interface assigned as an OPT interface? If so, 1.2.3 doesn't handle that nearly as well as 2.0 does. You'd probably have better luck on a 2.0 snapshot.
Does it make a difference if the set up is pfSense box–----Interwebs------pfSense box, and NOT pfSense box------Interwebs------Client Machine? It definitely took longer than 60 seconds, at least for one box that's in my office. I have an external line that I use for testing in my office where I'm currently prepping a Netgate box for deployment to another site. That box had an active connection, then I did all my ish, then the other sites came back up almost immediately after reboot, except for my test box, which took another minute or two I think.
Yes, that's where I've been adding the routes, as that's how the person before me set them up. Where are the OpenVPN advanced options? Just everything under the certs section of the server OpenVPN config?
-
There is an empty box at the bottom of the OpenVPN config. Just keep adding routes there, separated by semicolons
As an example, to route 192.168.10.x, 20.x, and 30.x over the OpenVPN connection, use:
route 192.168.10.0 255.255.255.0; route 192.168.20.0 255.255.255.0; route 192.168.30.0 255.255.255.0; -
There is an empty box at the bottom of the OpenVPN config. Just keep adding routes there, separated by semicolons
As an example, to route 192.168.10.x, 20.x, and 30.x over the OpenVPN connection, use:
route 192.168.10.0 255.255.255.0; route 192.168.20.0 255.255.255.0; route 192.168.30.0 255.255.255.0;Ah, the Custom Options box? That makes sense. I'll look into setting that up for the rest of the sites.
-
You made all of these changes simply to bring a tunnel back up?
That is way overkill and is not the right way to do so. You should have only needed to resave the openvpn tunnel that was having issues on either side.
Please post the logs from OpenVPN during the time that the outage occurred so that we can nail down the root of the disconnect.
No, the changes the were to remove the config for a site that is no longer operational, as that site's NetGate box is sitting on my shelf. To bring the tunnels back up, I rebooted the pfSense instance we're using solely for OpenVPN to remote sites.
Jun 8 07:27:50 openvpn[23663]: write TCPv4_CLIENT: Operation not permitted (code=1)
Jun 8 07:28:00 openvpn[23663]: write TCPv4_CLIENT: Operation not permitted (code=1)
Jun 8 07:28:11 openvpn[23663]: Inactivity timeout (–ping-restart), restarting
Jun 8 07:28:11 openvpn[23663]: SIGUSR1[soft,ping-restart] received, process restarting
Jun 8 07:28:16 openvpn[23663]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Jun 8 07:28:16 openvpn[23663]: Re-using pre-shared static key
Jun 8 07:28:38 openvpn[23663]: RESOLVE: Cannot resolve host address: DNS FOR OUR OPENVPN SERVER: [TRY_AGAIN] A temporary error occurred on an authoritative name server.
Jun 8 07:28:38 openvpn[23663]: Preserving previous TUN/TAP instance: ovpnc1
Jun 8 07:28:55 openvpn[23663]: Attempting to establish TCP connection with [AF_INET]OPENVPNSERVERIPADDRESS:119X [nonblock]
Jun 8 07:28:56 openvpn[23663]: TCP connection established with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 07:28:56 openvpn[23663]: TCPv4_CLIENT link local: [undef]
Jun 8 07:28:56 openvpn[23663]: TCPv4_CLIENT link remote: [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 07:28:56 openvpn[23663]: Peer Connection Initiated with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 07:28:57 openvpn[23663]: Initialization Sequence Completed
Jun 8 09:49:34 openvpn[23663]: Connection reset, restarting [0]
Jun 8 09:49:34 openvpn[23663]: SIGUSR1[soft,connection-reset] received, process restarting
Jun 8 09:49:39 openvpn[23663]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Jun 8 09:49:39 openvpn[23663]: Re-using pre-shared static key
Jun 8 09:49:39 openvpn[23663]: Preserving previous TUN/TAP instance: ovpnc1
Jun 8 09:49:39 openvpn[23663]: Attempting to establish TCP connection with [AF_INET]OPENVPNSERVERIPADDRESS:119X [nonblock]
Jun 8 09:49:49 openvpn[23663]: TCP: connect to [AF_INET]OPENVPNSERVERIPADDRESS:119X failed, will try again in 5 seconds: Operation timed out
Jun 8 09:50:04 openvpn[23663]: TCP: connect to [AF_INET]OPENVPNSERVERIPADDRESS:119X failed, will try again in 5 seconds: Operation timed out
Jun 8 09:50:19 openvpn[23663]: TCP: connect to [AF_INET]OPENVPNSERVERIPADDRESS:119X failed, will try again in 5 seconds: Operation timed out
Jun 8 09:50:25 openvpn[23663]: TCP connection established with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:50:25 openvpn[23663]: TCPv4_CLIENT link local: [undef]
Jun 8 09:50:25 openvpn[23663]: TCPv4_CLIENT link remote: [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:50:25 openvpn[23663]: Peer Connection Initiated with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:50:26 openvpn[23663]: Initialization Sequence Completed
Jun 8 09:51:27 openvpn[23663]: Inactivity timeout (–ping-restart), restarting
Jun 8 09:51:27 openvpn[23663]: SIGUSR1[soft,ping-restart] received, process restarting
Jun 8 09:51:32 openvpn[23663]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Jun 8 09:51:32 openvpn[23663]: Re-using pre-shared static key
Jun 8 09:51:32 openvpn[23663]: Preserving previous TUN/TAP instance: ovpnc1
Jun 8 09:51:32 openvpn[23663]: Attempting to establish TCP connection with [AF_INET]OPENVPNSERVERIPADDRESS:119X [nonblock]
Jun 8 09:51:33 openvpn[23663]: TCP connection established with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:51:33 openvpn[23663]: TCPv4_CLIENT link local: [undef]
Jun 8 09:51:33 openvpn[23663]: TCPv4_CLIENT link remote: [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:51:33 openvpn[23663]: Peer Connection Initiated with [AF_INET]OPENVPNSERVERIPADDRESS:119X
Jun 8 09:51:34 openvpn[23663]: Initialization Sequence Completed -
That log is from my test box on my desk connected to the external line–-interwebs---pfsense server in the closet next to me. The pfsense server open vpn logs got cleared when I restarted it. facepalm
