Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    OpenVPN on 2.7.1 crashes on some circumstances

    Scheduled Pinned Locked Moved OpenVPN
    22 Posts 10 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • O
      ogghi @Volui
      last edited by

      @Volui-0
      Hi there, no solution from my end, but just wanting to say: Affected, too!

      We had a difficult upgrade from 2.6.0 to 2.7.0 where the SSD was not booting anymore. Installed fresh 2.7.1 over it (could not even find 2.6.0 image to download) and imported settings backup.
      All was up and working again.

      Only issue is the VPN server (UDP port 1443 here) crashing randomly. I'll now monitor the system.log.
      Any other log file to look at?

      OpenVPN 2.6.7 amd64-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [MH/RECVDA] [AEAD] [DCO]
      library versions: OpenSSL 3.0.12 24 Oct 2023, LZO 2.10
      Is what is currently installed on here

      1 Reply Last reply Reply Quote 0
      • O
        ogghi @Volui
        last edited by

        @Volui-0
        As found on a Reddit post:

        There was a patch to openvpn that you can install using the CLI.
        openvpn: 2.6.7 -> 2.6.7_1 [pfSense]

        pkg upgrade

        I did this and restarted the VPN server in question, hoping it's enough. Let's see!

        V 1 Reply Last reply Reply Quote 0
        • V
          Volui @ogghi
          last edited by

          @ogghi
          I will waiting until it crashes again with more verbose log and try to catch the bug in it with extended info about it. Then, i will post that logs there and try your solution, thanks!

          1 Reply Last reply Reply Quote 1
          • V
            Volui
            last edited by

            Ok, the server crashed again. The more detailed log didn't show much, but here it is nonetheless:

            openvpn.log:
            Nov 23 04:33:51 pf openvpn[56470]: MULTI: multi_create_instance called
            Nov 23 04:33:51 pf openvpn[56470]: Re-using SSL/TLS context
            Nov 23 04:33:51 pf openvpn[56470]: Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
            Nov 23 04:33:51 pf openvpn[56470]: Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
            Nov 23 04:33:51 pf openvpn[56470]: Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
            Nov 23 04:33:51 pf openvpn[56470]: Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
            Nov 23 04:33:51 pf openvpn[56470]: Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
            Nov 23 04:33:51 pf openvpn[56470]: Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1500 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
            Nov 23 04:33:51 pf openvpn[56470]: TCP connection established with [AF_INET]45.79.168.172:34222
            Nov 23 04:33:51 pf openvpn[56470]: TCPv4_SERVER link local: (not bound)
            Nov 23 04:33:51 pf openvpn[56470]: TCPv4_SERVER link remote: [AF_INET]45.79.168.172:34222
            Nov 23 04:33:53 pf openvpn[56470]: MULTI: multi_create_instance called
            Nov 23 04:33:53 pf openvpn[56470]: Re-using SSL/TLS context
            Nov 23 04:33:53 pf openvpn[56470]: Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
            Nov 23 04:33:53 pf openvpn[56470]: Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
            Nov 23 04:33:53 pf openvpn[56470]: Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
            Nov 23 04:33:53 pf openvpn[56470]: Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
            Nov 23 04:33:53 pf openvpn[56470]: Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
            Nov 23 04:33:53 pf openvpn[56470]: Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1500 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
            Nov 23 04:33:53 pf openvpn[56470]: TCP connection established with [AF_INET]45.79.168.172:34218
            Nov 23 04:33:53 pf openvpn[56470]: TCPv4_SERVER link local: (not bound)
            Nov 23 04:33:53 pf openvpn[56470]: TCPv4_SERVER link remote: [AF_INET]45.79.168.172:34218
            Nov 23 04:33:57 pf openvpn[56470]: 45.79.168.172:34215 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
            Nov 23 04:33:57 pf openvpn[56470]: 45.79.168.172:34215 TLS Error: TLS handshake failed

            system.log:
            Nov 23 04:33:57 kernel pid 56470 (openvpn), jid 0, uid 0: exited on signal 11 (core dumped)
            Nov 23 04:33:57 kernel ovpns1: link state changed to DOWN

            In openvpn.log there several connection attempts in a row from the same IP address, then the server crashed. Now all that remains is to try the solution suggested by ogghi above (pkg upgrade). I hope this works.

            O 1 Reply Last reply Reply Quote 0
            • O
              ogghi @Volui
              last edited by

              @Volui-0
              I think it will work.
              It hasn't crashed here anymore!

              I am wondering if there is any official statement from Netgate or so?

              V 1 Reply Last reply Reply Quote 0
              • V
                Volui @ogghi
                last edited by

                @ogghi
                Yes, I also updated the OpenVPN package via pkg upgrade and now all that remains is to monitor the stability of the server. You are right, it looks like netgate has updated the openvpn package in its repository since the release of 2.7.1, but has not said anything about it anywhere. In any case, pkg upgrade is the only thing we can do about this problem for now.

                1 Reply Last reply Reply Quote 1
                • O
                  ogghi
                  last edited by

                  It seems stable here in regards to disconnects, but people using remote desktop sometimes get some timeouts as it seems. Keeping an eye on logs today.

                  Happy Monday ppl!

                  1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate
                    last edited by

                    OpenVPN released OpenVPN 2.6.8 which addresses a segfault that some users see with 2.6.7:

                    https://github.com/OpenVPN/openvpn/issues/449

                    https://openvpn.net/community-downloads/

                    We're still discussing what the best course of action is to address it since it doesn't seem to be widespread.

                    Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    S 1 Reply Last reply Reply Quote 1
                    • S
                      slu @jimp
                      last edited by slu

                      @jimp
                      we are also affected by random openvpn crashes, this setup running before over 8 years without any issue.

                      There are many different openvpn clients and version on this server, maybe this combination trigger the issue more...

                      pfSense Gold subscription

                      1 Reply Last reply Reply Quote 0
                      • jimpJ
                        jimp Rebel Alliance Developer Netgate
                        last edited by

                        We'll be bringing in OpenVPN 2.6.8 to the next patch release (Plus 23.09.1, CE 2.7.2) which should be out here in the next week or so if all goes according to plan.

                        https://redmine.pfsense.org/issues/15049

                        Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                        Need help fast? Netgate Global Support!

                        Do not Chat/PM for help!

                        M 1 Reply Last reply Reply Quote 4
                        • M
                          michmoor LAYER 8 Rebel Alliance @jimp
                          last edited by

                          @jimp
                          Hey Jim. Any release notes on 23.09.1 ?

                          Firewall: NetGate,Palo Alto-VM,Juniper SRX
                          Routing: Juniper, Arista, Cisco
                          Switching: Juniper, Arista, Cisco
                          Wireless: Unifi, Aruba IAP
                          JNCIP,CCNP Enterprise

                          jimpJ 1 Reply Last reply Reply Quote 0
                          • jimpJ
                            jimp Rebel Alliance Developer Netgate @michmoor
                            last edited by

                            @michmoor said in OpenVPN on 2.7.1 crashes on some circumstances:

                            @jimp
                            Hey Jim. Any release notes on 23.09.1 ?

                            Nothing public yet, but it's primarily security/stability things. The ZFS corruption bug, OpenVPN needed a version bump, so did strongSwan, various fixes in PHP code. Not a long list, but significant enough to warrant a patch release.

                            Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                            Need help fast? Netgate Global Support!

                            Do not Chat/PM for help!

                            M 1 Reply Last reply Reply Quote 3
                            • M
                              michmoor LAYER 8 Rebel Alliance @jimp
                              last edited by

                              @jimp
                              sounds good to me. thanks!

                              Firewall: NetGate,Palo Alto-VM,Juniper SRX
                              Routing: Juniper, Arista, Cisco
                              Switching: Juniper, Arista, Cisco
                              Wireless: Unifi, Aruba IAP
                              JNCIP,CCNP Enterprise

                              1 Reply Last reply Reply Quote 0
                              • jimpJ
                                jimp Rebel Alliance Developer Netgate
                                last edited by

                                FYI- If you are on Plus 23.09 or CE 2.7.1 we picked back the update to the current repositories so you can also get OpenVPN 2.6.8_1 right now from the console or SSH shell prompt:

                                # pkg update
                                # pkg upgrade -y openvpn
                                

                                And then either restart each instance manually or reboot.

                                Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                                Need help fast? Netgate Global Support!

                                Do not Chat/PM for help!

                                T O 2 Replies Last reply Reply Quote 2
                                • T
                                  tedquade @jimp
                                  last edited by

                                  @jimp Done.

                                  Thanks
                                  Ted

                                  1 Reply Last reply Reply Quote 0
                                  • O
                                    OhYeah 0 @jimp
                                    last edited by

                                    @jimp said in OpenVPN on 2.7.1 crashes on some circumstances:

                                    FYI- If you are on Plus 23.09 or CE 2.7.1 we picked back the update to the current repositories so you can also get OpenVPN 2.6.8_1 right now from the console or SSH shell prompt:

                                    # pkg update
                                    # pkg upgrade -y openvpn
                                    

                                    And then either restart each instance manually or reboot.

                                    We had one virtual instance of pfsense that had the OpenVPN remote access server crash every 2-3 days. After pkg update the server has been running without problems for 5+ days.

                                    S T 2 Replies Last reply Reply Quote 1
                                    • S
                                      slu @OhYeah 0
                                      last edited by

                                      We can confirm this, no issue since 2.6.8_1, thank you @jimp

                                      Simply reinstall of the package openvpn-client-export does also the OpenVPN upgrade.

                                      pfSense Gold subscription

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        thuyetti @OhYeah 0
                                        last edited by

                                        @OhYeah-0 said in OpenVPN on 2.7.1 crashes on some circumstances:

                                        @jimp said in OpenVPN on 2.7.1 crashes on some circumstances:

                                        FYI- If you are on Plus 23.09 or CE 2.7.1 we picked back the update to the current repositories so you can also get OpenVPN 2.6.8_1 right now from the console or SSH shell prompt:

                                        # pkg update
                                        # pkg upgrade -y openvpn
                                        

                                        And then either restart each instance manually or reboot.

                                        We had one virtual instance of pfsense that had the OpenVPN remote access server crash every 2-3 days. After pkg update the server has been running without problems for 5+ days.

                                        Thank you a lot @jimp , it works for me since more than 20 days now. We have 3 OpenVPN servers.

                                        1 Reply Last reply Reply Quote 0
                                        • V
                                          Volui
                                          last edited by A Former User

                                          It's seems to be solved for now (running almost month without crashes). To solve problem do from console from root:

                                          pkg update
                                          pkg upgrade -y openvpn

                                          Or just upgrade to latest release! For me, openVPN run as rock solid after i do upgraded the package and then update system to the latest stable release 2.7.2! Thanks All Guys!

                                          1 Reply Last reply Reply Quote 0
                                          • L Luvirini referenced this topic on
                                          • N
                                            ncohafmuta
                                            last edited by

                                            I just ran into this crash, same errors as OP.
                                            This is the first time i've ever seen the openvpn server crash. Usually my prod system is a i3-4130 (for years now), but i've been testing on an n100 system for about a week now and this crash just happened. Same pfsense version and config from the i3 system.
                                            I'm running pfsense 2.7.1 w/openvpn 2.6.8_1 so the new version doesn't fix it

                                            # pkg info
                                            openvpn-2.6.8_1 Secure IP/Ethernet tunnel daemon

                                            # openvpn --version
                                            OpenVPN 2.6.8 amd64-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [MH/RECVDA] [AEAD] [DCO]
                                            library versions: OpenSSL 3.0.12 24 Oct 2023, LZO 2.10
                                            DCO version: FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_1-n255918-774957be06d: Wed Nov 15 17:41:06 UTC 2023 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_1-main/obj/amd64/GScwGwyy/var/jenkins/workspace/pfSense-CE-snapshots-2_7_1-main/sources/F

                                            GertjanG 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.