Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Constant: One or more OpenVPN tunnel endpoints may have changed its IP.

    General pfSense Questions
    7
    20
    8.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      miles267
      last edited by

      I have Pfsense 2.1 running on a Supermicro XSPA7-HF atom with 4 GB RAM fine.  Using snort, pfblocker and several packages with relatively low CPU utilization (4%) and RAM (17%) however the gateway has begun showing a yellow PACKETLOSS status (62%).  While I have several pfblocker lists from iblocklist.com and have raised my max table size to 800,000 entries, the pfsense home router doesn't appear to be breaking a sweat.  I am uploading at 5 Mbit to a cloud backup service though it's always has a healthy (green) Gateway status.  What would cause this?

      I've gone into System > Routing > Gateway and made the following changes, but OpenVPN keeps "flapping" and there aren't any active OpenVPN connections.

      Probe Interval = 10 seconds
      Down = 30 seconds

      Oct 18 09:44:46 php: rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WANGW.
      Oct 18 09:44:43 check_reload_status: Reloading filter
      Oct 18 09:44:43 check_reload_status: Restarting OpenVPN tunnels/interfaces
      Oct 18 09:44:43 check_reload_status: Restarting ipsec tunnels
      Oct 18 09:44:43 check_reload_status: updating dyndns WANGW
      Oct 18 09:44:30 php: rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WANGW.
      Oct 18 09:44:27 check_reload_status: Reloading filter
      Oct 18 09:44:27 check_reload_status: Restarting OpenVPN tunnels/interfaces
      Oct 18 09:44:27 check_reload_status: Restarting ipsec tunnels
      Oct 18 09:44:27 check_reload_status: updating dyndns WANGW
      Oct 18 09:44:16 php: rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WANGW.
      Oct 18 09:44:13 check_reload_status: Reloading filter
      Oct 18 09:44:13 check_reload_status: Restarting OpenVPN tunnels/interfaces
      Oct 18 09:44:13 check_reload_status: Restarting ipsec tunnels
      Oct 18 09:44:13 check_reload_status: updating dyndns WANGW
      Oct 18 09:44:11 bandwidthd: DNS timeout for 192.168.0.5: This problem reduces graphing performance
      Oct 18 09:44:06 php: rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WANGW.
      Oct 18 09:44:03 check_reload_status: Reloading filter
      Oct 18 09:44:03 check_reload_status: Restarting OpenVPN tunnels/interfaces
      Oct 18 09:44:03 check_reload_status: Restarting ipsec tunnels
      Oct 18 09:44:03 check_reload_status: updating dyndns WANGW
      Oct 18 09:43:31 syslogd: kernel boot file is /boot/kernel/kernel

      1 Reply Last reply Reply Quote 0
      • P
        phil.davis
        last edited by

        If you really do have significant packet loss then you need to work out what the problem is on the WAN side and ISP. As the packet loss varies "randomly" apinger is going to declare WANGW down and up, resulting in stuff that might depend on WANGW being restarted.
        If you do only have a single WAN, then you can disable gateway monitoring, since there is nothing to failover to anyway. The downside of that is that you don't get to see latency, packet loss stats. I did put a feature request in RedMine to allow apinger to do its thing, reporting the stats, but not actually take any action. That might be useful for single-WAN installs so that the WAN can be monitored without triggering useless "failover" attempts. Just need some time to think about how to implement it…

        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

        1 Reply Last reply Reply Quote 0
        • M
          miles267
          last edited by

          Thanks.  I've been using pfsense successfully at home on a single WAN config (50 mbit down/5 mbit up) for several years.  Kept gateway monitor enable by default.  No issues.  I then recently configured pfsense traffic shaper to prioritize my VOIP, cloud backup and default traffic accordingly.  No issues including packet loss or latency.  However my upstream is 100% saturated by my cloud backup upload, that's when gateway monitor reports high latency and proceeds to check status on OpenVPN, etc.  Even if it's not OpenVPN being used (even if OpenVPN interface is disabled).

          Nevertheless, if I scale back the max upstream of my cloud backup on WAN to 80-90% (down from 100%) the latency monitor reports green/OK status.  I've since disabled the gateway monitor altogether and removed the 80-90% upstream WAN limit on the cloud backup traffic and the issue has gone away.

          I guess if my connection is now working and pfsense isn't restarting the interface driving up CPU utilization, etc. I'm fine.

          1 Reply Last reply Reply Quote 0
          • P
            phil.davis
            last edited by

            You can also adjust the gateway monitoring parameters - expand the advanced section when editing the gateway parameters. You can make the latency and packet loss limits much higher so that apinger does not react when you flood the link as you describe with the cloud backup traffic. That's what I do on low speed links that are easily saturated.

            As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
            If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

            1 Reply Last reply Reply Quote 0
            • M
              miles267
              last edited by

              @phil.davis:

              You can also adjust the gateway monitoring parameters - expand the advanced section when editing the gateway parameters. You can make the latency and packet loss limits much higher so that apinger does not react when you flood the link as you describe with the cloud backup traffic. That's what I do on low speed links that are easily saturated.

              Good suggestion.  Would you mind sharing the gateway parameters you used on lower speed links so that I can test with them?  I tried more relaxed settings, but perhaps they weren't as relaxed for a saturated highspeed connection as I had thought.  Thanks!

              1 Reply Last reply Reply Quote 0
              • D
                dreamslacker
                last edited by

                Have you tried prioritizing ICMP packets?  That should prevent the pings to the gateway from being dropped when the connection is being saturated and triggering apinger.

                1 Reply Last reply Reply Quote 0
                • M
                  miles267
                  last edited by

                  @dreamslacker:

                  Have you tried prioritizing ICMP packets?  That should prevent the pings to the gateway from being dropped when the connection is being saturated and triggering apinger.

                  No, but that's a great idea.  Would I prioritize ICMP to go into my highest priority queue alongside my VOIP traffic?  Is it just a single floating rule for my WAN interface for ANY ICMP traffic?  Or does there need to be both a TCP and UDP pair?

                  1 Reply Last reply Reply Quote 0
                  • D
                    dreamslacker
                    last edited by

                    @miles267:

                    No, but that's a great idea.  Would I prioritize ICMP to go into my highest priority queue alongside my VOIP traffic?  Is it just a single floating rule for my WAN interface for ANY ICMP traffic?  Or does there need to be both a TCP and UDP pair?

                    ICMP is a protocol on its own.  Just select Out on WAN for the floating rule for any ICMP traffic.  Depending on your WAN connection type, you might also need to adjust the default Apinger parameters.  Nevertheless, it is still a good idea to scale back on the backup upload slightly.

                    1 Reply Last reply Reply Quote 0
                    • M
                      miles267
                      last edited by

                      @dreamslacker:

                      @miles267:

                      No, but that's a great idea.  Would I prioritize ICMP to go into my highest priority queue alongside my VOIP traffic?  Is it just a single floating rule for my WAN interface for ANY ICMP traffic?  Or does there need to be both a TCP and UDP pair?

                      ICMP is a protocol on its own.  Just select Out on WAN for the floating rule for any ICMP traffic.  Depending on your WAN connection type, you might also need to adjust the default Apinger parameters.  Nevertheless, it is still a good idea to scale back on the backup upload slightly.

                      So far, this single WAN-out (floating) rules for any ICMP traffic appears to have addressed my issue.  Will continue to monitor and confirm.

                      1 Reply Last reply Reply Quote 0
                      • P
                        phil.davis
                        last edited by

                        Would you mind sharing the gateway parameters you used on lower speed links so that I can test with them?

                        It looks like you have found a better solution by giving ICMP priority - I should do the same!
                        I have some very slow links (e.g. 256kbps, 384kbps) that are easily swamped by downloads. So I increase the latency to 3000-4000ms and packet loss to 30-40%, ping every 2 seconds and wait 30 seconds to declare a gateway down. The latency and packet loss parameters are a bit extreme! But it does mean that a gateway is only declared down if it really is struggling badly or is actually down. These parameters are something that really have to be tuned to what happens in real life - for modern-speed links much much lower values should be needed.

                        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                        1 Reply Last reply Reply Quote 0
                        • A
                          archedraft
                          last edited by

                          I am having the same issue with "One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints" constantly reloading. Causing the internet to go out for 3-5 minuets at a time (which can get really frustrating). I am on 2.2.4-RELEASE (amd64). I tried disabling gateway monitoring but that does not fix the issue. I would like to try this

                          So far, this single WAN-out (floating) rules for any ICMP traffic appears to have addressed my issue.

                          Would you mind posting a screenshot of that rule so I can mimic it? I am not really sure what that rule should look like.

                          1 Reply Last reply Reply Quote 0
                          • A
                            archedraft
                            last edited by

                            Anyone have any thoughts? This seems to happen the most when I access my home network through openVPN from work and then remote desktop into my home computer, but I have also had it happen when I try and log into pfSense web GUI… seems pretty random. Here are some logs of it happening.

                            9/22/2015 12:44	php-fpm[20222]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use PIA_VPN_VPNV4.
9/22/2015 12:44	php-fpm[20222]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use PIA_VPN2_VPNV4.
9/22/2015 12:44	check_reload_status: Reloading filter
9/22/2015 12:44	check_reload_status: Restarting OpenVPN tunnels/interfaces
9/22/2015 12:44	check_reload_status: Restarting ipsec tunnels
9/22/2015 12:44	check_reload_status: updating dyndns PIA_VPN_VPNV4
9/22/2015 12:44	check_reload_status: Reloading filter
9/22/2015 12:44	check_reload_status: Restarting OpenVPN tunnels/interfaces
9/22/2015 12:44	check_reload_status: Restarting ipsec tunnels
9/22/2015 12:44	check_reload_status: updating dyndns PIA_VPN2_VPNV4
9/22/2015 12:44	php-fpm: /rc.start_packages: Restarting/Starting all packages.
9/22/2015 12:44	php-fpm: /rc.start_packages: Restarting/Starting all packages.
9/22/2015 12:44	check_reload_status: Starting packages
9/22/2015 12:44	php-fpm: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.XXX.XXX.XXX -> 10.XXX.XXX.XXX - Restarting packages.
9/22/2015 12:44	php-fpm: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.XXX.XXX.XXX -> 10.XXX.XXX.XXX - Restarting packages.
9/22/2015 12:44	php-fpm: /rc.start_packages: Restarting/Starting all packages.
9/22/2015 12:44	php-fpm: /rc.start_packages: Restarting/Starting all packages.
9/22/2015 12:44	php-fpm: /rc.newwanip: Creating rrd update script
9/22/2015 12:44	php-fpm: /rc.newwanip: Creating rrd update script
9/22/2015 12:44	php-fpm: /rc.newwanip: Ignoring IPsec reload since there are no tunnels on interface opt3
9/22/2015 12:44	php-fpm: /rc.newwanip: Ignoring IPsec reload since there are no tunnels on interface opt3
9/22/2015 12:44	check_reload_status: Starting packages
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.XXX.XXX.XXX -> 10.XXX.XXX.XXX - Restarting packages.
9/22/2015 12:44	php-fpm: /rc.newwanip: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 21 leases to leases file. Listening on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Sending on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get
9/22/2015 12:44	php-fpm: /rc.newwanip: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 21 leases to leases file. Listening on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Sending on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: Creating rrd update script
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: Ignoring IPsec reload since there are no tunnels on interface opt2
9/22/2015 12:44	php-fpm: /rc.newwanip: IP has changed, killing states on former IP 10.XXX.XXX.XXX.
9/22/2015 12:44	php-fpm: /rc.newwanip: IP has changed, killing states on former IP 10.XXX.XXX.XXX.
9/22/2015 12:44	php-fpm: /rc.newwanip: rc.newwanip: on (IP address: 10.XXX.XXX.XXX) (interface: PIA_VPN2[opt3]) (real interface: ovpnc1).
9/22/2015 12:44	php-fpm: /rc.newwanip: rc.newwanip: on (IP address: 10.XXX.XXX.XXX) (interface: PIA_VPN2[opt3]) (real interface: ovpnc1).
9/22/2015 12:44	php-fpm: /rc.newwanip: rc.newwanip: Info: starting on ovpnc1.
9/22/2015 12:44	php-fpm: /rc.newwanip: rc.newwanip: Info: starting on ovpnc1.
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 21 leases to leases file. Listening on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Sending on BPF/em1/0X:3X:XX:XX:XX:XX/192.XXX.XXX.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you d
9/22/2015 12:44	check_reload_status: rc.newwanip starting ovpnc1
9/22/2015 12:44	kernel: ovpnc1: link state changed to UP
9/22/2015 12:44	check_reload_status: Reloading filter
9/22/2015 12:44	kernel: ovpnc1: link state changed to DOWN
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: IP has changed, killing states on former IP 10.XXX.XXX.XXX.
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: rc.newwanip: on (IP address: 10.XXX.XXX.XXX) (interface: PIA_VPN[opt2]) (real interface: ovpnc2).
9/22/2015 12:44	php-fpm[37931]: /rc.newwanip: rc.newwanip: Info: starting on ovpnc2.
9/22/2015 12:44	check_reload_status: rc.newwanip starting ovpnc2
9/22/2015 12:44	kernel: ovpnc2: link state changed to UP
9/22/2015 12:44	php-fpm: /rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt2).
9/22/2015 12:44	php-fpm: /rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt2).
9/22/2015 12:44	php-fpm: /rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt2).
9/22/2015 12:44	php-fpm: /rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt2).
9/22/2015 12:44	check_reload_status: Reloading filter
9/22/2015 12:44	check_reload_status: Reloading filter
9/22/2015 12:44	kernel: ovpnc2: link state changed to DOWN
9/22/2015 12:39	php-fpm: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
9/22/2015 12:39	php-fpm: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
9/22/2015 12:39	check_reload_status: Reloading filter
9/22/2015 12:39	check_reload_status: Restarting OpenVPN tunnels/interfaces
9/22/2015 12:39	check_reload_status: Restarting ipsec tunnels
9/22/2015 12:39	check_reload_status: updating dyndns WAN_DHCP
                            1 Reply Last reply Reply Quote 0
                            • P
                              pf3000
                              last edited by

                              About the ICMP rule, it's easier if you run the traffic shaper wizard and select hight priority for ICMP there; rule will be created automatically.

                              The solutions are discussed above. I had the same issue as first post, and my main fix was more relaxed settings for gateway monitoring for WAN interface, and disabling gateway monitoring for the VPN interface(s). Since then it's been completely stable for months now. The ping priority setting didn't matter as much for me.

                              I set the WAN gateway monitor to look out for disconnections only rather than a "poor quality" connection. Mine looks like this (attachment)

                              2015-03-05_093329.jpg
                              2015-03-05_093329.jpg_thumb

                              1 Reply Last reply Reply Quote 0
                              • A
                                archedraft
                                last edited by

                                @pf3000:

                                I set the WAN gateway monitor to look out for disconnections only rather than a "poor quality" connection. Mine looks like this (attachment)

                                THANK YOU! I modified my settings to look very similar to yours and also turned of VPN monitoring (I didn't realize I had that turned on). I am hopeful that this will solve the issue. I'll report back after a few days.

                                EDIT:
                                I have made it a full 24 hours without any issues! First time in months that has happened.

                                K 1 Reply Last reply Reply Quote 0
                                • K
                                  krisabc1 @archedraft
                                  last edited by

                                  Hi
                                  Can you show how you have it set up, unfortunately the images have expired
                                  BR.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Do you have a gateway group setup? That's the only place you have a trigger setting for down or latency etc.

                                    Otherwise just disable monitoring action in the advanced gateway settings.

                                    K 1 Reply Last reply Reply Quote 1
                                    • K
                                      krisabc1 @stephenw10
                                      last edited by

                                      @stephenw10 thanks for your help, looks like it helped :-) I have only 1 gateway I turned off the monitoring and the problem has not occurred since this morning :-)

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        It's usually better to disable 'monitoring action' rather than monitoring entirely so you still get data.

                                        K 1 Reply Last reply Reply Quote 1
                                        • K
                                          krisabc1 @stephenw10
                                          last edited by

                                          @stephenw10 Do you have a setting in mind like this?
                                          2a929bc6-b77b-4efd-ba1d-da141fce3afa-obraz.png

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Yes, exactly like that.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.