OpenVPN could not be established after upgrade to 23.01 on SG-3100

stephenw10

@sphillips said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

removed my defined network from "IPv4 Tunnel Network" openvpn connects again

That sounds more like a subnet conflict. Perhaps something else changed coincidentally with the upgrade? Or possibly simply making any change and resaving the OpenVPN settings re-created the conf file. I would have expected to see errors logged in either of those cases though.

Steve

sphillips

@stephenw10 said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

That sounds more like a subnet conflict. Perhaps something else changed coincidentally with the upgrade? Or possibly simply making any change and resaving the OpenVPN settings re-created the conf file. I would have expected to see errors logged in either of those cases though.
Steve

I did not see any other errors besides the "Exiting due to fatal error", i dont recall making any changes between the upgrade and noticing the issue, i confess it took e a few days to notice the issue since i dont use vpn everyday.
i just got frusted with the problem and went back to pfsense CE will revisit pfsense plus when theres an update or time allows, thank you.

stephenw10

Ok, probably some other issue then. Unfortunately "Exiting due to fatal error" isn't particularly helpful in the default OpenVPN output. We'll probably need to turn up the logging level if nothing else is shown.

systemofapwne

@stephenw10 Thank you, this simple fix also did it for me!

JVComputers

@stephenw10 This worked for me, as well! Thanks!

rune-san

@stephenw10 Just adding for the web record that I also encountered this issue when upgrading my 3100 this morning. The manual update to the linker file followed by a reboot of the Firewall fixed it.

I obviously have no data on how pervasive this issue is, but like with the 1100 and 2100 maybe it's worth pulling the 23.01 upgrade for the time being on the 3100? Yes I fixed it by going to Netgate forums and see this is one of the top OpenVPN threads but there's lots of folks with these appliances out there that are going to be pulling their hair out over this especially when the interfaces are still in place. Just seems like a big headache for them waiting to happen. Doubly so if they're at remote sites. Either way, the fix was simple! Seems like this should be petted manually and reintroduced as a new release for the 3100 to cover this bug.

stephenw10

I'm not sure it would be possible to make that change without rebuilding and all that implies.
I'll certainly confirm that though.

Lavarocker

@peekay

Thanks for much for posting the solution - really appreciate the great advice. Confirming OpenVPN still works after reboot.

I'm quite disappointed that Netgate would release "stable" software with a major flaw! Next time I get a survey will voice that.

Rocker

mikej47

kldxref /boot/kernel and a restart of the OpenVPN service worked for me on my Sg3100. I issued the command through the web gui.

JVComputers

@mikej47 Yes, that worked for me, too. Problem comes when you're upgrading a bunch of firewalls and you connect to them via OpenVPN to administer and you can't restart the service after a reboot because you've been kicked off the VPN due to this bug.

stephenw10

@mmrk said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

kldxref

It's fixed in 23.05. https://redmine.pfsense.org/issues/13963

I would always advise having some out of band access for an upgrade though if at all possible. Especially a large step like this. A lot of other changes happened in OpenVPN.

Steve

cneep

@jvcomputers
What I do on the remote firewall is to allow access to ports 22 and 443, but only specifically from:

one public IP address within my primary location's static IP address range, and
an unused public IP address within a nearby satellite location's public IP address range

I've run into situations before where the VPN goes down, so 1) comes in handy to get the VPN back up and running without needing to travel to the remote location. I've also run into a situation before where my ISP actually reassigned the static IP address at my primary location to another client, causing me to lose that static IP address (and subsequently my VPN and the remote access to all of my satellite locations!) So 2) protects against that from happening again....at least I only need to travel to a nearby location to re-establish all the VPNs.

It's not exactly out-of-band, but it serves my needs.

JVComputers

@cneep That's an idea, but one I'd have to carefully consider. I don't like the idea of opening administrative ports directly to the internet, even if I lock them down to my IP.

stephenw10

If it's an SSH port you can also use key based auth only. The risk of that combined with a limited source IP is close to zero.

Steve

JVComputers

@stephenw10 True, providing no future vulnerabilities are found in the SSH services which is, unfortunately, not unheard of.

stephenw10

Right. But don't leave the port open after the upgrade is complete and the VPN is back up.
If you really wanted to you could put the rule on a schedule so it closed again automatically if you were still unable to access it for some other reason.

JVComputers

@stephenw10 I feel dumb, I should've thought of only opening it during an upgrade procedure. Good idea.

alvinestevez

After I upgraded my Netgate SG-3100 to pfsense 23.01-RELEASE (arm) from 22.05.1, the OpenVPN demon was unable to start.

Logs show: Cannot open TUN/TAP dev /dev/tun1: No such file or directory (errno=2)
I was unable to start the OpenVPN Deamon

Solution:

goto: Diagnostics, then click Command Prompt
execute: kldxref /boot/kernel
Reboot shortly after executing step 2

After I rebooted the my Openvpn server and and Client tunnel both were running!!!!!

This instructions on these threads are accurate!

jkalber

@mikej47 said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

kldxref /boot/kernel and a restart of the OpenVPN service worked for me on my Sg3100. I issued the command through the web gui.

Yup same here as well! Had a slight heart attack when the VPN tunnels didn't come back online though. Thankfully we have site to site IPSec tunnels in place or else I would have been making a 2hr round trip commute into the office to finish up everything onsite.

SteveITS

@jkalber There was also a kernel panic bug accessing the web GUI over IPSec and an nginx crash with OpenVPN, though I'm guessing from your message you weren't using pfSense for your remote access?

You might consider having an extra rule you can enable only during upgrades to allow access directly from your IP to WAN.