Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    OpenVPN client issue after upgrading to 2.7 (Solved)

    OpenVPN
    1
    3
    1.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      matt84
      last edited by matt84

      I upgraded my main pfSense box from 2.6 to 2.7 and upon reboot, connecting to it was unstable. SSH connections to the box would drop every few seconds or so, web browsing for all machines would stop and start the same, and my two VPN gateways were showing as offline.

      Looking at the systems logs I'm guessing due to the VPN client issue, pfSense is killing states which explains the connection drops

      Sep 5 10:51:36 	kernel 		ovpnc3: link state changed to DOWN
Sep 5 10:51:36 	check_reload_status 	422 	Reloading filter
Sep 5 10:51:12 	php-fpm 	383 	/rc.start_packages: Restarting/Starting all packages.
Sep 5 10:51:11 	php-fpm 	59913 	/rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.24.0.10 -> 10.24.0.18 - Restarting packages.
Sep 5 10:51:11 	check_reload_status 	422 	Starting packages
Sep 5 10:51:11 	check_reload_status 	422 	Reloading filter
Sep 5 10:51:11 	kernel 		config_aqm Unable to configure flowset, flowset busy!
Sep 5 10:51:11 	kernel 		config_aqm Unable to configure flowset, flowset busy!
Sep 5 10:51:10 	php-fpm 	383 	/rc.openvpn: Gateway, NONE AVAILABLE
Sep 5 10:51:10 	php-fpm 	383 	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed IP addresses. Reloading endpoints that may use AUVPN_VPNV4.
Sep 5 10:51:10 	php-fpm 	68019 	/rc.openvpn: Gateway, NONE AVAILABLE
Sep 5 10:51:10 	php-fpm 	68019 	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed IP addresses. Reloading endpoints that may use USAVPN_VPNV4.
Sep 5 10:51:09 	rc.gateway_alarm 	95382 	>>> Gateway alarm: USAVPN_VPNV4 (Addr:8.8.4.4 Alarm:1 RTT:0ms RTTsd:0ms Loss:100%)
Sep 5 10:51:09 	rc.gateway_alarm 	95161 	>>> Gateway alarm: AUVPN_VPNV4 (Addr:217.138.205.98 Alarm:1 RTT:0ms RTTsd:0ms Loss:100%)
Sep 5 10:51:09 	check_reload_status 	422 	updating dyndns AUVPN_VPNV4
Sep 5 10:51:09 	check_reload_status 	422 	Restarting IPsec tunnels
Sep 5 10:51:09 	check_reload_status 	422 	Restarting OpenVPN tunnels/interfaces
Sep 5 10:51:09 	check_reload_status 	422 	Reloading filter
Sep 5 10:51:09 	check_reload_status 	422 	updating dyndns USAVPN_VPNV4
Sep 5 10:51:09 	check_reload_status 	422 	Restarting IPsec tunnels
Sep 5 10:51:09 	check_reload_status 	422 	Restarting OpenVPN tunnels/interfaces
Sep 5 10:51:09 	check_reload_status 	422 	Reloading filter
Sep 5 10:51:09 	php-fpm 	59913 	/rc.newwanip: Gateway, NONE AVAILABLE
Sep 5 10:51:09 	php-fpm 	59913 	/rc.newwanip: IP Address has changed, killing all states (ip_change_kill_states is set).
Sep 5 10:51:09 	php-fpm 	59913 	/rc.newwanip: Creating rrd update script
Sep 5 10:51:07 	php-fpm 	59913 	/rc.newwanip: Removing static route for monitor 8.8.4.4 and adding a new route through 10.24.0.82
Sep 5 10:51:07 	php-fpm 	59913 	/rc.newwanip: Removing static route for monitor 217.138.205.98 and adding a new route through 10.24.0.17
Sep 5 10:51:06 	php-fpm 	59913 	/rc.newwanip: rc.newwanip: Info: starting on ovpnc3.
Sep 5 10:51:06 	php-fpm 	59913 	/rc.newwanip: rc.newwanip: on (IP address: 10.24.0.18) (interface: AUVPN[opt3]) (real interface: ovpnc3).
Sep 5 10:51:05 	kernel 		ovpnc3: link state changed to UP
Sep 5 10:51:05 	check_reload_status 	422 	rc.newwanip starting ovpnc3
Sep 5 10:51:04 	kernel 		ovpnc3: link state changed to DOWN
Sep 5 10:51:04 	check_reload_status 	422 	Reloading filter
Sep 5 10:50:50 	php-fpm 	383 	/index.php: Successful login for user 'admin' from: 192.168.1.61 (Local Database)
Sep 5 10:50:50 	nginx 		2023/09/05 10:50:50 [error] 3417#100268: send() failed (54: Connection reset by peer) while logging to syslog, server: unix:/var/run/log
Sep 5 10:50:40 	php-fpm 	384 	/rc.start_packages: Restarting/Starting all packages.
Sep 5 10:50:39 	php-fpm 	84533 	/rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.24.0.18 -> 10.24.0.10 - Restarting packages.
Sep 5 10:50:39 	check_reload_status 	422 	Starting packages
Sep 5 10:50:39 	check_reload_status 	422 	Reloading filter
Sep 5 10:50:38 	php-fpm 	9647 	/rc.openvpn: Gateway, NONE AVAILABLE
Sep 5 10:50:38 	php-fpm 	9647 	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed IP addresses. Reloading endpoints that may use USAVPN_VPNV4.
Sep 5 10:50:38 	php-fpm 	9647 	/rc.openvpn: Gateway, NONE AVAILABLE
Sep 5 10:50:38 	php-fpm 	9647 	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed IP addresses. Reloading endpoints that may use AUVPN_VPNV4.
Sep 5 10:50:37 	rc.gateway_alarm 	62994 	>>> Gateway alarm: USAVPN_VPNV4 (Addr:8.8.4.4 Alarm:1 RTT:0ms RTTsd:0ms Loss:100%)
Sep 5 10:50:37 	check_reload_status 	422 	updating dyndns USAVPN_VPNV4
Sep 5 10:50:37 	check_reload_status 	422 	Restarting IPsec tunnels
Sep 5 10:50:37 	check_reload_status 	422 	Restarting OpenVPN tunnels/interfaces
Sep 5 10:50:37 	check_reload_status 	422 	Reloading filter
Sep 5 10:50:37 	rc.gateway_alarm 	64071 	>>> Gateway alarm: AUVPN_VPNV4 (Addr:217.138.205.98 Alarm:1 RTT:0ms RTTsd:0ms Loss:100%)
Sep 5 10:50:37 	check_reload_status 	422 	updating dyndns AUVPN_VPNV4
Sep 5 10:50:37 	check_reload_status 	422 	Restarting IPsec tunnels
Sep 5 10:50:37 	check_reload_status 	422 	Restarting OpenVPN tunnels/interfaces
Sep 5 10:50:37 	check_reload_status 	422 	Reloading filter
Sep 5 10:50:37 	php-fpm 	84533 	/rc.newwanip: Gateway, NONE AVAILABLE
Sep 5 10:50:37 	php-fpm 	84533 	/rc.newwanip: IP Address has changed, killing all states (ip_change_kill_states is set).
Sep 5 10:50:37 	php-fpm 	84533 	/rc.newwanip: Creating rrd update script
Sep 5 10:50:35 	php-fpm 	84533 	/rc.newwanip: Removing static route for monitor 8.8.4.4 and adding a new route through 10.24.0.82
Sep 5 10:50:35 	php-fpm 	84533 	/rc.newwanip: Removing static route for monitor 217.138.205.98 and adding a new route through 10.24.0.9
Sep 5 10:50:34 	php-fpm 	84533 	/rc.newwanip: rc.newwanip: Info: starting on ovpnc3.
Sep 5 10:50:34 	php-fpm 	84533 	/rc.newwanip: rc.newwanip: on (IP address: 10.24.0.10) (interface: AUVPN[opt3]) (real interface: ovpnc3).
Sep 5 10:50:33 	kernel 		ovpnc3: link state changed to UP
Sep 5 10:50:33 	check_reload_status 	422 	rc.newwanip starting ovpnc3
Sep 5 10:50:32 	kernel 		ovpnc3: link state changed to DOWN
Sep 5 10:50:32 	check_reload_status 	422 	Reloading filter

      Manually stopping the two VPN clients, stabilized everything else connecting to or through pfSense 2.7 so I'm sure I don't have a hardware issue with my box.

      One thing I noticed is from the VPN client interfaces in pfSense, I could not ping the two gateway monitor addresses even from pfSense itself. I then removed those monitor addresses and attempted to let pfSense to choose its own monitor addresses, but that didn't change anything. They still showed as down.

      I'm thinking this is some sort of routing issue that has changed between pfSense versions or even OpenVPN versions between pfSense 2.6 and 2.7 but I'm a software developer who knows only a little about networking so I'm a bit out of my depth here.

      pfSense 2.7 routing table
      c081ca63-645e-46f6-9453-b9991b27d810-image.png

      pfSense 2.6 routing table (after rollback)
      79c3f391-58bf-4609-9948-72505e71a213-image.png

      My VPN client configuration is as per below
      auvpn_1.jpg
      auvpn_2.jpg

      No settings were changed as a part of the upgrade with regard to the OpenVPN configuration. I did uncheck "Don't Pull Routes" and reconnected one of the VPN clients and then that one lit up and was working. What I noticed then was all my internet traffic even for the pfSense box itself was going over VPN Gateway instead of my WAN Group gateway. Re-ticking this box changed the behavior back to the issue.

      What am I missing with the routing tables here? Any help would be appreciated.

      I'm sure I have a misconfiguration that worked on 2.6 but no longer on 2.7, but I don't understand why. @jimp any ideas?

      M 1 Reply Last reply Reply Quote 0
      • M
        matt84 @matt84
        last edited by

        I found why I was having connectivity issues, System -> Advanced -> Misc -> Flush all states on gateway failure was set. It was a checkbox in 2.6 and now a drop down list in 2.7. It still doesn't explain why I the VPN connections cannot route.

        I'm setting up 2.7 clean in Hyper-V and I am slowly re-building what I currently have sans LAGG / VLAN / and WAN Group (4G Backup) to see if I can figure out what is causing this.

        M 1 Reply Last reply Reply Quote 0
        • M
          matt84 @matt84
          last edited by

          So an update, I manually rebuilt my config in a Hyper-V VM and well and behold it just worked. So then I upgraded again from 2.6 to 2.7 on my physical hardware and the same issue occurred.

          This time though I noticed there was mention of OpenVPN (redmine #14646) in the System Patches package so I applied all of the patches, and rebooted, and again the two OpenVPN clients did not route traffic. Strange.

          I then went in to the two OpenVPN client configuration checked all of the settings compared to the VM and the only differences I had set on the VM compared to my bare metal upgrade install were:

          • Exit Notify - set to Retry 1x
          • Ping Settings - Interval - 5
          • Ping Settings - Timeout - 30
          • Compression - Disable Compression [Omit Preference]

          I applied the above settings to the two client VPN configurations and rebooted, and the gateways came up green.

          I checked the route table between 2.7 not working bare metal and 2.7 working and they were identical.

          Maybe something in the above OpenVPN settings or in conjunction that system patch fixed it. I don't really know. At least now it seems to be working

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.