OpenVPN Status Incorrect
-
Hello,
After upgrading to 2.2.6, OpenVPN status shows all daemons as down, even though they are actually running and accepting connections.
If I SSH to the box and 'killall openvpn', then manually restart the daemons, the status shows correctly.
This even happens after a reboot. Anybody seen this before?
Adam
 -
I've seen it twice. I backed up my settings and reinstalled the box and restored the settings and problem went away…
-
I've been considering doing this, due to this and a few other oddities and random crashes that have come about after upgrading 2.2.4 –> 2.2.5 --> 2.2.6.
-
This happened to me once as well. Disabling and re-enabling the tunnel on ends worked for me. Haven't seen the behavior since. That was probably 8 months ago or so.
-
Just reinstalled and recovered from an XML backup - it's still the same! Also tried disabling and re-enabling every daemon in webconfigurator, also didn't make any difference,
Damn!
-
Why would you restore from a config that you know isn't working? Reset to factory defaults from the console and manually rebuild your config from scratch. That way if something breaks along the way you'll know exactly what it is.
-
It's not practical to manually rebuild the configuration from scratch - this isn't a home setup with a couple of interfaces, there are 17,000 lines of XML.
I'd more been thinking around the likelihood that binary, library or dependancy got goosed during the update (as there were other problems that had existed since 2.2.5 –> 2.2.6, which are now resolved).
OpenVPN still works and users can connect, but for whatever reason the status page doesn't reflect this. I'll try just removing the OpenVPN related stuff from the XML and manually re-create that bit, but I get the feeling that won't help.
Potentially one for paid support, I think!
-
Sorry ajrg, I posted that last message in the wrong thread :)
-
Potentially one for paid support, I think!
This sounds like an OpenVPN PID file bug that I haven't found a way of replicating. If you can go the paid support route, I'd be glad to work through this with you to find a resolution. We don't deduct incidents from your account for software problems.
If you can note this forum thread and my interest in the ticket, the support guys will make sure I get the ticket.
-
Sorry ajrg, I posted that last message in the wrong thread :)
No worries! :)
@cmb:
Potentially one for paid support, I think!
This sounds like an OpenVPN PID file bug that I haven't found a way of replicating. If you can go the paid support route, I'd be glad to work through this with you to find a resolution. We don't deduct incidents from your account for software problems.
If you can note this forum thread and my interest in the ticket, the support guys will make sure I get the ticket.
Okay, I'll get onto that as soon as I can. I'll work out a downtime window too, just in case the boxes need rebooting at any point!
Which timezone you in? -6? -
Okay, I'll get onto that as soon as I can. I'll work out a downtime window too, just in case the boxes need rebooting at any point!
Which timezone you in? -6?Yeah -6, I'm in Austin. If it's replicable with a backup of your config restored to anything else, I can just take that backup and fix it from there. If that's not the case for some reason, then yeah we'll need a bit of a maintenance window. Probably take adding some debug logging to the code and rebooting up to maybe a handful of times to track down the root cause.
-
Interestingly, we don't seem to be having this issue any more - no configuration changes since my last post. I'm a bit confused!
-
Success!
-
-
Aah, spoke too soon! The issue is back.
cmb: I'll be in touch via your support channel soon.
For more information, all the site-to-site OpenVPNs display status correctly, but the remote access OpenVPNs do not.
Tried deleting all the remote access configs, thenkillall openvpn, then manually recreating. Status shows fine until reboot, then it's back to the aforementioned error message. -
I'm pretty sure it's some kind of problem within OpenVPN where it fails to update its PID file for some reason, but without being able to replicate I don't know.
Definitely would like to work with you to track this one down.
-
I'm inclined to agree with you - looking at OpenVPN PID files, quite a few of them had really high PID numbers, into the billions!
I can run;
killall openvpn ; rm -f /var/run/openvpn_*Then when the services are restarted, they all work fine until the next service crash or config reload.
Also, (probably because of this issue), if I have the faulting services in Service Watchdog, I eventually end up having to reboot the routers (PID exhaustion? Is that still a thing these days?).
Anyhow, probably a week from today, I'll be able to get us a few dates that we'll be quiet enough to not suffer from having to reboot systems, etc.