OpenVPN could not be established after upgrade to 23.01 on SG-3100

catloafsoft

@depam Ugh, I'd rather wait for them to fix the openvpn package (or the kernel, whichever is at fault). Did you have to recreate your VPN configurations?

shpokas

/dev/tap1 missing here, but otherwise exactly the same issue on the same hardware SG-3100

2023-02-16 09:03:59 us=269241 OpenVPN 2.6_beta1 armv7-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] [DCO]
2023-02-16 09:03:59 us=269522 library versions: OpenSSL 1.1.1t-freebsd  7 Feb 2023, LZO 2.10
2023-02-16 09:03:59 us=270896 MANAGEMENT: unix domain socket listening on /var/etc/openvpn/server1/sock
2023-02-16 09:03:59 us=271390 NOTE: when bridging your LAN adapter with the TAP adapter, note that the new bridge adapter will often take on its own IP address that is different from what the LAN adapter was previously set to
2023-02-16 09:03:59 us=272082 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2023-02-16 09:03:59 us=272302 Initializing OpenSSL support for engine 'devcrypto'
2023-02-16 09:03:59 us=277015 Diffie-Hellman initialized with 4096 bit key
2023-02-16 09:03:59 us=281213 WARNING: experimental option --capath /var/etc/openvpn/server1/ca
2023-02-16 09:03:59 us=282306 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2023-02-16 09:03:59 us=282830 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-02-16 09:03:59 us=283167 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2023-02-16 09:03:59 us=283459 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-02-16 09:03:59 us=283841 TLS-Auth MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
**2023-02-16 09:03:59 us=284554 Cannot open TUN/TAP dev /dev/tap1: No such file or directory (errno=2)**
2023-02-16 09:03:59 us=284994 Exiting due to fatal error

depam

@catloafsoft No it was taken from the backup. But still "downtime" :(. And my pfblocker and Snort is not functioning properly now. This happened to me few times but on 22.0x it seems the upgrade process is stable. I never learned my lesson to hop in on new release though I know it will always have issue.

innostus

Same issue here. Netgate should not release software that's not been tested for such basic functions!

routeless

Also got this issue on Netgate 3100

Dayve

Same issue here. I upgraded last night and lost all openvpn connections. Anyone know of a fix coming or know how to resolve?

fertig

Same issue here.

Cannot create tun-Interfaces manually either

[23.01-RELEASE][root@fw]/dev: openvpn --mktun --dev tun1
2023-02-16 13:27:50 Note: --mktun does not support DCO. Creating TUN interface.
2023-02-16 13:27:50 Options error: options --mktun and --rmtun are not available on your operating system.  Please check 'man tun' (or 'tap'), whether your system supports using 'ifconfig tun0 create' / 'destroy' to create/remove persistent tunnel interfaces.
2023-02-16 13:27:50 Exiting due to fatal error

[23.01-RELEASE][root@fw]/dev: ifconfig tun1 create
ifconfig: SIOCIFCREATE2 (tun1): Invalid argument

[23.01-RELEASE][root@fw]/dev: mknod /dev/tun1 c 10 200
mknod: /dev/tun1: No such file or directory

There are not many devices at all in /dev, perhaps there's more broken

shpokas

@fertig
why can't they release a new update, instead forcing me to go onsite to reflash the device?

fertig

@shpokas said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

@fertig
why can't they release a new update, instead forcing me to go onsite to reflash the device?

I think the reflash is a workaround, they'll probably fix it - the problem is about a half a day old

MaxK 0

@innostus Mission critical software development, testing, deployment, and support has been my career since 1985. First rule for applying an update: develop and test a contingency plan to ensure you can fallback to a known operational state if anything goes wrong during or after the update.

You can blame Netgate for missing a defect but we are all responsible for ensuring that we follow industry established best practices to ensure a rapid recovery from an unplanned event or disaster. If I upgraded my environment without testing or having a fallback plan and I am suffering from unexpected or degraded performance, I am to blame - not Netgate.

If I don’t have the time or resources to test or recover then I wait to upgrade and monitor forums like this to see if there are any issues that may impact my environment.

Finally, thank you very much to everyone who did discover issues and provided the valuable information for Netgate to address them.

innostus

@maxk-0 Thanks for replying. You seem to assume that I do not have a fallback. However, one of the reasons someone buys an appliance, is to have decent reliability and support. Also, the 3100 is aimed at small business and private persons, who do not always have a spare device to test with. My own mistakes: I missed the news items on the 23.01 beta, RC and release. They followed rapidly after each other. That should have tipped me off into waiting a little longer.

Have a nice day!

Arnold

johnpoz

@maxk-0 said in OpenVPN could not be established after upgrade to 23.01 on SG-3100:

contingency plan to ensure you can fallback to a known operational state

This is the way ;)

Youtube Video

Be it your in an enterprise or just a home user - this is the way! ;)

shpokas

@johnpoz
sure there's a way, I just, dammit, don't want to go onsite!

stephenw10

Opened a bug for this: https://redmine.pfsense.org/issues/13963

Updating the linker file manaully fixed it for me. Run: kldxref /boot/kernel

Steve

innostus

@stephenw10 This works for me!

fertig

@stephenw10

working fine and is reboot safe. Excellent. Thanks.
I won't reflash and wait for a possible bugfix release.

johnpoz

@shpokas I hear ya - but we always have someone on hand in the DC when doing any sort of upgrade.. Just in case.. The best laid plans can always fail..

I left some boxes at sites on old pfsense for longest time because of covid, and just pita to either go there or have someone available..

And also hear ya that damn it this box should just work.. And I should just be able to click update.. But no matter how easy it should be or how many times it has worked in the past, etc. And this is not some minor update, this is a major change to freebsd, and an upgrade to php as well, etc..

I haven't pulled the trigger yet on my 4860 here at home.. But I did get the new 23.01 image from tac already, and will burn it to usb and be ready to clean install if something goes horrible wrong.. Nothing will I hope - but just in case, I don't want to be down for any period of time. I also still have 22.05 image if need be to clean install too, etc.

Also notice that I might have an issue with my 4860 "may have issues with the ichsmb0 and/or ehci0" so have that info handy to put in the /boot/loader.conf.local

If something crashes and burns - I wouldn't be happy no.. But also know that things like this can happen.. I need to update the dsm version on my nas for example.. And had many an update go smooth in the past, but since the last update I have created a non supported configuration with nvme as storage.. People say shouldn't be a problem - but just in case I am currently backing up my plex and dockers that reside on the nvme storage - just in case that doesn't survive the update.. And I don't want to go through the hassle recreating them, nor using a backup from a few days ago.. So running a backup just before I do the update later this morning ;)

stephenw10

I would add the line to disable ichsmb on the 4860 before upgrading @johnpoz
You lose nothing by having it there. We will likely add that by default in 23.05.
If you're not on the console you won't notice anything anyway.

johnpoz

@stephenw10 thanks I was thinking of that.. Any reason why a clean install might be good - change to the zfs layout or anything. I was thinking I should be able to test the zfs rollback feature with this update as well.

So should I be able to rollback to 22.05 after an update to 23.01?

stephenw10

Yes you can roll back to 22.05 from 23.01. But not to 22.01.

There's no particular reason a clean install should be any better there. IMO at least!