Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Troubleshooting WAN outage

    Scheduled Pinned Locked Moved General pfSense Questions
    10 Posts 3 Posters 228 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I Online
      itsbry
      last edited by

      Hi all,
      I have frequent WAN outages and have discovered that LAN hosts do not regain a connection unless I reboot the firewall. I'd like to find out what log entries I should check to see if it is actually the ISP, or just my interface not coming up after an outage, or even just the older hardware that hosts the pfSense.

      My setup is WAN -> modem -> pfSense -> L3 switch -> LAN

      Version: 2.8.0-RELEASE (amd64)
      built on Wed May 21 19:12:00 EDT 2025
      FreeBSD 15.0-CURRENT

      I do have access to the firewall during these outages. It's function is purely firewall... L3 switch handles routing and DHCP.
      Thanks in advance for any advice!

      S 1 Reply Last reply Reply Quote 0
      • S Offline
        SteveITS Rebel Alliance @itsbry
        last edited by

        @itsbry The Gateways log shows the dpinger alerts (and status updates, like when it starts).

        Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to reboot, or more depending on packages, and device or disk speed.
        Upvote 👍 helpful posts!

        I 1 Reply Last reply Reply Quote 0
        • I Online
          itsbry @SteveITS
          last edited by

          @SteveITS
          Hi Steve.. thanks for the reply!
          I do see the dpinger entries... I am not sure exactly how to interpret them. Do these entries indicate a failure to both interfaces (WAN_DHCP and Cisco3650)?

          Below is a section that repeated until reboot:

          2025-09-01 10:39:01.940024-04:00 dpinger 42829 WAN_DHCP 9.9.9.9: sendto error: 65
          2025-09-01 10:39:00.916386-04:00 dpinger 43384 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 192.168.10.1 bind_addr 192.168.10.2 identifier "Cisco3650 "
          2025-09-01 10:39:00.914271-04:00 dpinger 42829 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 9.9.9.9 bind_addr 64.179.196.79 identifier "WAN_DHCP "
          2025-09-01 10:39:00.888785-04:00 dpinger 17634 exiting on signal 15
          2025-09-01 10:39:00.881734-04:00 dpinger 17970 exiting on signal 15
          2025-09-01 10:38:59.559298-04:00 dpinger 17970 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 192.168.10.1 bind_addr 192.168.10.2 identifier "Cisco3650 "
          2025-09-01 10:38:59.557205-04:00 dpinger 17634 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 9.9.9.9 bind_addr 64.179.196.79 identifier "WAN_DHCP "
          2025-09-01 10:38:59.532071-04:00 dpinger 51993 exiting on signal 15
          2025-09-01 10:38:43.668833-04:00 dpinger 51993 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 192.168.10.1 bind_addr 192.168.10.2 identifier "Cisco3650 "
          2025-09-01 10:38:43.654222-04:00 dpinger 36936 exiting on signal 15
          2025-09-01 10:38:43.647320-04:00 dpinger 37539 exiting on signal 15
          2025-09-01 10:38:43.602620-04:00 dpinger 36936 WAN_DHCP 9.9.9.9: sendto error: 65
          2025-09-01 10:38:43.100796-04:00 dpinger 36936 WAN_DHCP 9.9.9.9: sendto error: 65
          2025-09-01 10:38:41.600583-04:00 dpinger 37539 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 192.168.10.1 bind_addr 192.168.10.2 identifier "Cisco3650 "
          2025-09-01 10:38:41.597546-04:00 dpinger 36936 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 9.9.9.9 bind_addr 64.179.196.79 identifier "WAN_DHCP "
          2025-09-01 10:38:41.571964-04:00 dpinger 49679 exiting on signal 15
          2025-09-01 10:38:41.564964-04:00 dpinger 50057 exiting on signal 15

          S 1 Reply Last reply Reply Quote 0
          • S Offline
            SteveITS Rebel Alliance @itsbry
            last edited by

            @itsbry Error 65 is:
            https://docs.netgate.com/pfsense/en/latest/troubleshooting/gateway-errors.html#sendto-error-65

            re signal 15, see:
            https://forum.netgate.com/topic/174601/dpinger-exiting-on-signal-15/6

            ...check the main log at 2025-09-01 10:38:43 and see if it logs anything about the interface?

            The "send_interval 500ms loss_interval 2000ms ...." messages are when dpinger starts. Seems like you have two WANs so it monitors both.

            FWIW you can disable the monitoring action to not do anything if an interface has high packet loss etc. but that doesn't help if the interface is disconnecting or something.

            Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to reboot, or more depending on packages, and device or disk speed.
            Upvote 👍 helpful posts!

            1 Reply Last reply Reply Quote 0
            • stephenw10S Online
              stephenw10 Netgate Administrator
              last edited by

              Yeah I imagine the Cisco3650 is actually the downstream LAN side gateway. In which case it should probably be set to always up. Make sure it's not actually a gateway on LAN directly or pfSense will treat it as a WAN and NAT out of it by default. Unless you have disabled that.

              But, yes, check the main system log for interface link state events.

              I 1 Reply Last reply Reply Quote 0
              • I Online
                itsbry @stephenw10
                last edited by

                @stephenw10 @SteveITS
                Thanks to you both... yes, the Cisco3650 is on the LAN side (I'll check that for your recommendations).

                Here|s what I see at 10:38:43 (clearly the WAN link is down):

                2025-09-01 10:38:43.683920-04:00 check_reload_status 514 Reloading filter
                2025-09-01 10:38:43.683873-04:00 check_reload_status 514 Restarting OpenVPN tunnels/interfaces
                2025-09-01 10:38:43.683819-04:00 check_reload_status 514 Restarting IPsec tunnels
                2025-09-01 10:38:43.683649-04:00 check_reload_status 514 updating dyndns WAN_DHCP
                2025-09-01 10:38:43.681845-04:00 rc.gateway_alarm 54536 >>> Gateway alarm: WAN_DHCP (Addr:64.179.196.1 Alarm:down RTT:0ms RTTsd:0ms Loss:100%)

                1 Reply Last reply Reply Quote 0
                • I Online
                  itsbry
                  last edited by

                  Ok, so I see this:

                  706ec45d-c369-4b92-ae87-cd6735997bec-image.png

                  So I did this for Cisco3650:

                  25b9d4b2-6f85-4209-be20-7e8f2472cbbb-image.png

                  S 1 Reply Last reply Reply Quote 0
                  • S Offline
                    SteveITS Rebel Alliance @itsbry
                    last edited by

                    @itsbry Cisco3650 shows as "LAN" interface, so why does it have a gateway set?

                    Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                    When upgrading, allow 10-15 minutes to reboot, or more depending on packages, and device or disk speed.
                    Upvote 👍 helpful posts!

                    I 1 Reply Last reply Reply Quote 0
                    • I Online
                      itsbry @SteveITS
                      last edited by itsbry

                      @SteveITS Novice user in a rush, probably (me).
                      I'm remote at the moment so I hesitate to pull that from the GW list... unless there's no way it will interrupt the network?

                      FWIW, I have this:

                      43501260-bc74-4326-ad25-6c9be94713b9-image.png

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S Online
                        stephenw10 Netgate Administrator
                        last edited by stephenw10

                        It's a downstream gateway on the LAN, that's fine when you have a router there.

                        The thing it should not be is set on the LAN directly. So in Interfaces > LAN, no gateway should be set. That's what pfSense uses to determine which interfaces are "WANs".

                        Either way it's clearly not a problem currently and not the cause of a WAN outage so no need to change that yet. It would be good to do so eventually to clean up the logs and diagnosing other issues easier.

                        But, yes, dpinger shows it stops seeing ping responses from 9.9.9.9. That could be for a number of reasons though. Do you you see the WAN interface lose link or lose it's IP address in the system log when that happens?

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.