Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    dpinger resets at 12:40AM every day?

    Scheduled Pinned Locked Moved
    General pfSense Questions
    6
    13
    763
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RobbieTTR
      RobbieTT @coolspot
      last edited by

      @coolspot

      I've also had some weirdness but after disabling the dpinger Gateway Actions (this thread) sanity has been restored.

      I don't fully understand the underlying issues though, just the way to ameliorate it.

      ☕️

      1 Reply Last reply Reply Quote 1
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by stephenw10

        Hmm, disabling actions shouldn't affect dpinger itself though. 🤔

        Disabling monitoring entirely stops dpinger so that would stops logs like that but....

        RobbieTTR 1 Reply Last reply Reply Quote 0
        • RobbieTTR
          RobbieTT @stephenw10
          last edited by

          @stephenw10 said in dpinger resets at 12:40AM every day?:

          Hmm, disabling actions shouldn't affect dpinger itself though. 🤔

          Disabling monitoring entirely stops dpinger so that would stops logs like that but....

          Ok, I must have misinterpreted. To my eyes it looked like dpinger was restarting all packages, including itself.

          ☕️

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Hmm, it will restart if the WAN actually goes down because the IP might change. I don't expect it to restart on loss or latency though.

            1 Reply Last reply Reply Quote 0
            • C
              coolspot @stephenw10
              last edited by

              @stephenw10 said in dpinger resets at 12:40AM every day?:

              Is that the only thing logged at that time? Are those WANs static?

              They're dynamic - Cable Modem and Bell Fiber Internet (PPPoE). The Bell connection would switch IPs if the connection dropped.

              I checked my Gateway log there was no failover event.

              ov 22 01:09:16	miniupnpd	72487	remove port mapping 41642 UDP because it has expired
Nov 22 00:11:39	miniupnpd	72487	remove port mapping 41644 UDP because it has expired
Nov 21 22:10:02	miniupnpd	72487	remove port mapping 41644 UDP because it has expired
Nov 21 20:56:15	miniupnpd	72487	remove port mapping 41645 UDP because it has expired
Nov 21 20:20:50	miniupnpd	72487	remove port mapping 43723 UDP because it has expired
Nov 21 17:44:42	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 21 14:27:32	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 21 10:04:10	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 21 09:11:04	miniupnpd	72487	recv (state0): Connection reset by peer
Nov 21 09:11:04	miniupnpd	72487	recv (state0): Connection reset by peer
Nov 21 09:11:04	miniupnpd	72487	recv (state0): Connection reset by peer
Nov 21 09:11:04	miniupnpd	72487	recv (state0): Connection reset by peer
Nov 21 09:11:04	miniupnpd	72487	recv (state0): Connection reset by peer
Nov 21 04:55:08	miniupnpd	72487	remove port mapping 41644 UDP because it has expired
Nov 21 02:31:30	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 20 21:56:08	miniupnpd	72487	remove port mapping 41643 UDP because it has expired
Nov 20 20:50:50	miniupnpd	72487	remove port mapping 50201 UDP because it has expired
Nov 20 20:46:15	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 20 18:27:53	miniupnpd	72487	remove port mapping 41647 UDP because it has expired
Nov 20 16:30:53	miniupnpd	72487	sendto(udp_notify=9, 192.168.1.1): No buffer space available
Nov 20 16:21:04	miniupnpd	72487	remove port mapping 43723 UDP because it has expired
Nov 20 13:27:16	miniupnpd	72487	remove port mapping 37429 UDP because it has expired
Nov 20 13:20:51	miniupnpd	72487	remove port mapping 43723 UDP because it has expired

              I also noticed lately that pfSense will randomly "hiccup" and stop responding - do you think it's a hardware issue? No errors in logs or dmesg.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Stop responding to what?

                There's nothing else in the system log at that time?

                1 Reply Last reply Reply Quote 0
                • M
                  mr_nets
                  last edited by

                  Hi ,

                  Have you installed Suricata, If so does it happen when updating the Rule Set ? I had a similar issue with Suricata and I solved it by activating Live Rule Swap On Update in Global Settings .

                  bmeeksB 1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks @mr_nets
                    last edited by bmeeks

                    @mr_nets said in dpinger resets at 12:40AM every day?:

                    Hi ,

                    Have you installed Suricata, If so does it happen when updating the Rule Set ? I had a similar issue with Suricata and I solved it by activating Live Rule Swap On Update in Global Settings .

                    Yep. Suricata, when using the netmap device with Inline IPS Mode, will cycle the interface when Suricata stops and then restarts following a rules update job. Switching over to the "Live Rule Update" option on the GLOBAL SETTINGS tab will instead send Suricata a SIGHUP signal to let it know to reload the rules. The only downside of this option is a temporary large increase in memory usage as two copies of the entire rule set will be in memory at the same time during the swap. On some machines with limited RAM and a large number of enabled rules this setting may result in out-of-memory problems.

                    Although I have not specifically tested for it, I believe some changes made in Suricata upstream several versions back resulted in a change in how the libpcap functionality works (libpcap is used with Legacy Blocking Mode). Some users have posted evidence that even libpcap interfaces may get cycled down and back up when Suricata restarts.

                    1 Reply Last reply Reply Quote 1
                    • dennypageD
                      dennypage @coolspot
                      last edited by

                      @coolspot dpinger does not restart itself. The only way dpinger exits is upon receipt of a signal. In your case, dpinger is exiting on receipt of signal 15, which is SIGTERM. In other words, the dpinger process is being explicitly killed.

                      As previously noted this is likely happening because the interface is being bounced.

                      C 1 Reply Last reply Reply Quote 0
                      • C
                        coolspot @dennypage
                        last edited by coolspot

                        @dennypage said in dpinger resets at 12:40AM every day?:

                        As previously noted this is likely happening because the interface is being bounced.

                        That's odd, because I can't find any reason in my logs why the interface is bouncing. There are no failover messages in the Gateway tab, no log messages about the interfaces going up or down either.

                        It happened right on schedule again today:

                        
Nov 23 00:40:24	dpinger	33659	send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 198.52.175.1 bind_addr 198.52.175.28 identifier "CWAN "
Nov 23 00:40:24	dpinger	32724	send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.11.6.145 bind_addr 174.88.68.50 identifier "WAN "
Nov 23 00:40:24	dpinger	67350	exiting on signal 15
Nov 23 00:40:24	dpinger	67802	exiting on signal 15

                        My PPPoE interface has actually been up for 8 days. Not sure about the Cable Modem since it's not reported in pfSense but the IP hasn't changed for days.

                        I think I found the issue, I have a WiFi interface where my access point reboots itself every night around 12:40AM. Whenever the access point reboots, it triggers dpinger to terminate? The WiFi interface is on on a private subnet... not sure why it would affect dpinger?

                        be0c0b76-7b30-4c9e-8ba9-532430b77b7d-image.png

                        Nov 23 00:40:23	check_reload_status	451	Reloading filter
Nov 23 00:40:23	php-fpm	24447	/rc.linkup: DEVD Ethernet detached event for opt3
Nov 23 00:40:23	php-fpm	24447	/rc.linkup: Hotplug event detected for WIFI(opt3) static IP address (4: 192.168.10.1)
Nov 23 00:40:23	xinetd	25517	Reconfigured: new=0 old=2 dropped=0 (services)
Nov 23 00:40:23	xinetd	25517	readjusting service 19001-tcp
Nov 23 00:40:23	xinetd	25517	readjusting service 19000-tcp
Nov 23 00:40:23	xinetd	25517	Swapping defaults
Nov 23 00:40:23	xinetd	25517	Starting reconfiguration
Nov 23 00:40:23	php-fpm	15464	/rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: WIFI[opt3]) (real interface: igb0).
Nov 23 00:40:23	php-fpm	15464	/rc.newwanip: rc.newwanip: Info: starting on igb0.
Nov 23 00:40:22	kernel		igb0: link state changed to DOWN
Nov 23 00:40:22	check_reload_status	451	Linkup starting igb0
Nov 23 00:40:22	check_reload_status	451	Reloading filter
Nov 23 00:40:22	check_reload_status	451	rc.newwanip starting igb0
Nov 23 00:40:22	php-fpm	15464	/rc.linkup: HOTPLUG: Triggering address refresh on opt3 (igb0)
Nov 23 00:40:22	php-fpm	15464	/rc.linkup: DEVD Ethernet attached event for opt3
Nov 23 00:40:22	php-fpm	15464	/rc.linkup: Hotplug event detected for WIFI(opt3) static IP address (4: 192.168.10.1)
Nov 23 00:40:21	kernel		igb0: link state changed to UP
Nov 23 00:40:21	check_reload_status	451	Linkup starting igb0

                        Any ideas why this interface is being treated as a WAN connection?

                        Update: Seems to be this exact issue: https://forum.netgate.com/topic/174723/hotplug-event-causes-rc-start_packages-restarting-starting-all-packages/20

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Yup it's that. If an interface goes down then IP addresses on the firewall change. I agree it's a bit of a big hammer and ideally could be far more nuanced. But that's what you're seeing.

                          Interestingly there is a quirk you can probably use to stop this happening. If you enable IPv6 on the wifi interface and set it to track a WAN for v6, even if there is no IPv6, it will by ignored.

                          Steve

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post