Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login
    Introducing Netgate Nexus: Multi-Instance Management at Your Fingertips.

    pfSense laggy after WAN outage

    Scheduled Pinned Locked Moved General pfSense Questions
    26 Posts 5 Posters 578 Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I Offline
      itsbry
      last edited by

      After a rare long run of no WAN outages, it happened... power brownout knocked the ISP offline for a few hours. When that was repaired, the connection came up but I struggled to access the pfSense GUI. Pings to Google DNS were intermittent, as well. Eventually, rebooting the firewall resolved the issue.

      Anyone experience this after a WAN outage?

      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        Check the system logs for the period before the reboot. Was there something stuck in a loop perhaps?

        1 Reply Last reply Reply Quote 0
        • I Offline
          itsbry
          last edited by

          This appears to be when the link came back up:

          <11>1 2026-05-24T12:33:35.817649-04:00 BnK.local.lan check_reload_status 497 - - Could not connect to /var/run/php-fpm.socket
          <11>1 2026-05-24T12:33:36.819648-04:00 BnK.local.lan check_reload_status 497 - - Could not connect to /var/run/php-fpm.socket
          <13>1 2026-05-24T12:33:36.828676-04:00 BnK.local.lan check_reload_status 497 - - Linkup starting re1
          <27>1 2026-05-24T12:33:36.930634-04:00 BnK.local.lan rtsold 61214 - - <cap_rssend> sendmsg on re1: Permission denied
          <6>1 2026-05-24T12:33:36.939622-04:00 BnK.local.lan kernel - - - re1: link state changed to UP
          <171>1 2026-05-24T12:33:37.000000-04:00 BnK.local.lan nginx - - - 2026/05/24 12:33:37 [error] 91972#100209: *5521 connect() to unix:/var/run/php-fpm.socket failed (61: Connection refused) while connecting to upstream, client: 192.168.1.5, server: , request: "POST /widgets/widgets/disks.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "192.168.10.2", referrer: "http://192.168.10.2/"
          <171>1 2026-05-24T12:33:37.000000-04:00 BnK.local.lan nginx - - - 2026/05/24 12:33:37 [error] 91972#100209: *5521 connect() to unix:/var/run/php-fpm.socket failed (61: Connection refused) while connecting to upstream, client: 192.168.1.5, server: , request: "POST /getstats.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "192.168.10.2", referrer: "http://192.168.10.2/"

          1 Reply Last reply Reply Quote 0
          • I Offline
            itsbry
            last edited by

            Then a little later in the logs:

            <11>1 2026-05-24T12:37:59.387615-04:00 BnK.local.lan check_reload_status 497 - - Could not connect to /var/run/php-fpm.socket
            <11>1 2026-05-24T12:37:59.432613-04:00 BnK.local.lan check_reload_status 497 - - Could not connect to /var/run/php-fpm.socket
            <13>1 2026-05-24T12:37:59.950848-04:00 BnK.local.lan check_reload_status 497 - - updating dyndns wan
            <27>1 2026-05-24T12:37:59.980438-04:00 BnK.local.lan php-fpm 94435 - - /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp, 6: dhcp6)
            <27>1 2026-05-24T12:37:59.980484-04:00 BnK.local.lan php-fpm 94435 - - /rc.linkup: DEVD Ethernet detached event for wan
            <11>1 2026-05-24T12:37:59.989717-04:00 BnK.local.lan check_reload_status 497 - - Could not connect to /var/run/php-fpm.socket

            GertjanG 1 Reply Last reply Reply Quote 0
            • GertjanG Offline
              Gertjan @itsbry
              last edited by Gertjan

              @itsbry

              Have a look here : things that happen when 'php' stopped.

              edit : if it's just happens ones, connect to console, and use the menu to start php again. It's essential.
              Check also the logs if you can find why this happened.
              The logs are here : /var/log/

              No "help me" PM's please. Use the forum, the community will thank you.

              I 1 Reply Last reply Reply Quote 0
              • I Offline
                itsbry @Gertjan
                last edited by

                @Gertjan Thank you... I will do some digging. I assume that if the PHP is unavailable, this is why the GUI was so very unresponsive.

                1 Reply Last reply Reply Quote 0
                • stephenw10S Offline
                  stephenw10 Netgate Administrator
                  last edited by

                  Yes without any available PHP processes you will see the login screen but nothing further in the webgui.

                  The WAN is on re1 there I assume?

                  I 1 Reply Last reply Reply Quote 0
                  • I Offline
                    itsbry @stephenw10
                    last edited by

                    @stephenw10 Correct... re1 is the WAN interface.
                    Odd though... I could navigate enough to get it rebooted without going to the closet, albeit Very Slowly.

                    luckman212L 1 Reply Last reply Reply Quote 0
                    • stephenw10S Offline
                      stephenw10 Netgate Administrator
                      last edited by

                      Hmm, well it must have had some processing still available then. At least one PHP process not completely locked. Unless you were using SSH which doesn't require php.

                      1 Reply Last reply Reply Quote 0
                      • luckman212L Offline
                        luckman212 LAYER 8 @itsbry
                        last edited by

                        @itsbry didn't see you mention what version of pfSense? That might be relevant.

                        1 Reply Last reply Reply Quote 0
                        • I Offline
                          itsbry
                          last edited by

                          @stephenw10 Not SSH...

                          @luckman212 Apologies!
                          2.8.1-RELEASE (amd64)
                          built on Tue Sep 9 12:29:00 EDT 2025
                          FreeBSD 15.0-CURRENT

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S Offline
                            stephenw10 Netgate Administrator
                            last edited by

                            Hmm, well if the link is flapping continuously it could simply be the php scripts that are run each time piling up.

                            1 Reply Last reply Reply Quote 0
                            • I Offline
                              itsbry
                              last edited by

                              We had another outage, this one due to power... remote access stayed down and the GUI was largely unresponsive until the pfSense box was rebooted. While I would love to solve the laggy GUI problem, I'm more concerned about the actual WAN link also being unavailable.

                              Below is a repeating section from the System/General logs:

                              2026-06-01 17:24:38.286730-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:24:38.286714-04:00 check_reload_status 507 Restarting OpenVPN tunnels/interfaces
                              2026-06-01 17:24:38.286694-04:00 check_reload_status 507 Restarting IPsec tunnels
                              2026-06-01 17:24:38.286604-04:00 check_reload_status 507 updating dyndns WAN_DHCP
                              2026-06-01 17:24:37.201132-04:00 php-fpm 52657 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 1 days has not passed. Not updating dynamic DNS entry.
                              2026-06-01 17:24:37.201105-04:00 php-fpm 52657 /rc.dyndns.update: Dynamic Dns (): Current WAN IP: 64.179.196.79 Cached IP: 64.179.196.79
                              2026-06-01 17:24:37.200708-04:00 php-fpm 52657 /rc.dyndns.update: Dynamic DNS custom (): _detectChange() starting.
                              2026-06-01 17:24:37.199924-04:00 php-fpm 52657 /rc.dyndns.update: Dynamic DNS: updatedns() starting
                              2026-06-01 17:24:36.224763-04:00 php-fpm 14769 /rc.linkup: DEVD Ethernet detached event for wan
                              2026-06-01 17:24:36.224738-04:00 php-fpm 14769 /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp, 6: dhcp6)
                              2026-06-01 17:24:36.224515-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:24:36.186232-04:00 check_reload_status 507 updating dyndns wan
                              2026-06-01 17:24:36.157989-04:00 rtsold 38320 <cap_rssend> sendmsg on re1: Permission denied
                              2026-06-01 17:24:35.412931-04:00 check_reload_status 507 Restarting IPsec tunnels
                              2026-06-01 17:24:35.053763-04:00 php-fpm 32861 /rc.newwanip: IP Address has changed, killing all states (ip_change_kill_states is set).
                              2026-06-01 17:24:34.385613-04:00 php-fpm 32861 /rc.newwanip: rc.newwanip: on (IP address: 64.179.196.79) (interface: WAN[wan]) (real interface: re1).
                              2026-06-01 17:24:34.385423-04:00 php-fpm 32861 /rc.newwanip: rc.newwanip: Info: starting on re1.
                              2026-06-01 17:24:33.388013-04:00 php-fpm 29473 /rc.linkup: Starting rtsold process on wan(re1)
                              2026-06-01 17:24:33.387895-04:00 php-fpm 29473 /rc.linkup: Starting DHCP6 client for interfaces re1
                              2026-06-01 17:24:33.376306-04:00 php-fpm 29473 /rc.linkup: Accept router advertisements on interface re1
                              2026-06-01 17:24:33.376236-04:00 php-fpm 29473 /rc.linkup: calling interface_dhcpv6_configure.
                              2026-06-01 17:24:33.375537-04:00 check_reload_status 507 rc.newwanip starting re1
                              2026-06-01 17:24:33.359048-04:00 kernel - re1: link state changed to UP
                              2026-06-01 17:24:33.284980-04:00 check_reload_status 507 Linkup starting re1
                              2026-06-01 17:24:29.339108-04:00 kernel - re1: link state changed to DOWN
                              2026-06-01 17:24:29.188463-04:00 check_reload_status 507 Linkup starting re1
                              2026-06-01 17:24:29.158100-04:00 php-fpm 29473 /rc.linkup: HOTPLUG: Configuring interface wan
                              2026-06-01 17:24:29.158084-04:00 php-fpm 29473 /rc.linkup: DEVD Ethernet attached event for wan
                              2026-06-01 17:24:29.158016-04:00 php-fpm 29473 /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp, 6: dhcp6)
                              2026-06-01 17:24:29.157850-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:24:29.133470-04:00 php-fpm 4891 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1780349069] unbound[4656:0] error: bind: address already in use [1780349069] unbound[4656:0] fatal error: could not open ports'
                              2026-06-01 17:24:09.564211-04:00 tailscale 97995 Unable to find device tailscale0
                              2026-06-01 17:24:04.699348-04:00 rtsold 82630 <cap_rssend> sendmsg on re1: Permission denied
                              2026-06-01 17:24:03.489767-04:00 tailscale 90301 Waiting for device tailscale0
                              2026-06-01 17:24:03.469419-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:24:03.177268-04:00 php-fpm 32861 /rc.start_packages: Restarting/Starting all packages.
                              2026-06-01 17:24:02.171606-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:24:02.171505-04:00 check_reload_status 507 Starting packages
                              2026-06-01 17:24:02.171393-04:00 php-fpm 3392 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 0.0.0.0 -> 64.179.196.79 - Restarting packages.
                              2026-06-01 17:24:00.679278-04:00 rtsold 82630 <cap_rssend> sendmsg on re1: Permission denied
                              2026-06-01 17:24:00.157050-04:00 php-fpm 3392 /rc.newwanip: Creating rrd update script
                              2026-06-01 17:24:00.152010-04:00 php-fpm 3392 /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
                              2026-06-01 17:23:59.704187-04:00 php-fpm 32861 /rc.dyndns.update: Dynamic DNS () There was an error trying to determine the public IP for interface - wan (re1 ).
                              2026-06-01 17:23:59.703493-04:00 php-fpm 32861 /rc.dyndns.update: Dynamic DNS: updatedns() starting
                              2026-06-01 17:23:59.149261-04:00 php-fpm 3392 /rc.newwanip: Dynamic DNS () There was an error trying to determine the public IP for interface - wan (re1 ).
                              2026-06-01 17:23:59.148606-04:00 php-fpm 3392 /rc.newwanip: Dynamic DNS: updatedns() starting
                              2026-06-01 17:23:58.689506-04:00 check_reload_status 507 Reloading filter
                              2026-06-01 17:23:58.689447-04:00 check_reload_status 507 Restarting OpenVPN tunnels/interfaces
                              2026-06-01 17:23:58.689374-04:00 check_reload_status 507 Restarting IPsec tunnels
                              2026-06-01 17:23:58.689201-04:00 check_reload_status 507 updating dyndns WAN_DHCP
                              2026-06-01 17:23:58.687385-04:00 rc.gateway_alarm 10170 >>> Gateway alarm: WAN_DHCP (Addr:64.179.196.1 Alarm:down RTT:0ms RTTsd:0ms Loss:100%)

                              stephenw10S tinfoilmattT 2 Replies Last reply Reply Quote 0
                              • stephenw10S Offline
                                stephenw10 Netgate Administrator
                                last edited by

                                Hmm, the log shows the WAN reconecting and correctly pulling a new lease etc.

                                But it physically lost link and reconnected. What is the NIC actually connected to? Did you have a modem reboot?

                                I 1 Reply Last reply Reply Quote 0
                                • I Offline
                                  itsbry @stephenw10
                                  last edited by

                                  @stephenw10
                                  Power was down for only 10-15 min, so the batteries held the rack up... I do not believe the modem power-cycled during this time (I do not have log access to that device, ISP locked). Layout is below:
                                  Modem -> pfSense -> Cisco3850-L3 -> LAN

                                  The firewall does monitor the WAN link but takes no action on it, assumes it is Up all the time.

                                  GertjanG 1 Reply Last reply Reply Quote 0
                                  • GertjanG Offline
                                    Gertjan @itsbry
                                    last edited by Gertjan

                                    @itsbry said in pfSense laggy after WAN outage:

                                    The firewall does monitor the WAN link but takes no action on it, assumes it is Up all the time.

                                    That's important info.
                                    This means that messages like "kernel - re1: link state changed to DOWN (or UP)" are not initiated on the pfSense side, but on the 'other' side = the modem.
                                    Modem can do so (IRC) to 'signal the downstream device that the renegotiated an upstream 'ISP' connection. Bit ... it's a long time I've seen a modem type device.

                                    Btw : the power down was locally (your place/office) or more widespread ? If it's the area or bigger : ISP can also have issues when their stuff goes off line for a while. They will, however, never admit this, so people tend to search (and destroy while doing so) the issue locally, while the real issue is 'upstream'. We've all seen this before.
                                    If it's not the DNS, it's the f**ing ISP ^^

                                    No "help me" PM's please. Use the forum, the community will thank you.

                                    I 1 Reply Last reply Reply Quote 0
                                    • I Offline
                                      itsbry @Gertjan
                                      last edited by

                                      @Gertjan
                                      This was absolutely a widespread outage... heavy rain and wind were passing through the area, downing trees etc. I'm surprised that it was only down for a short time.
                                      The ISP is still only providing a coaxial connection and they don't allow modem access so you're right, we'll only guess what is going on upstream. I will say though... the GUI and WAN link were surely to remain in the impaired state until reboot. I wonder if there is something that could trigger a soft reset or something when this happens?

                                      For clarity, I only rebooted the pfSense, nothing else. I have confirmed that all devices in the rack stayed up during the event.

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator @itsbry
                                        last edited by stephenw10

                                        @itsbry said in pfSense laggy after WAN outage:

                                        2026-06-01 17:24:33.359048-04:00 kernel - re1: link state changed to UP
                                        2026-06-01 17:24:33.284980-04:00 check_reload_status 507 Linkup starting re1
                                        2026-06-01 17:24:29.339108-04:00 kernel - re1: link state changed to DOWN
                                        2026-06-01 17:24:29.188463-04:00 check_reload_status 507 Linkup starting re1

                                        This was that outage? For 4 seconds?

                                        Do you have more that one gateway shown?

                                        I 1 Reply Last reply Reply Quote 0
                                        • I Offline
                                          itsbry @stephenw10
                                          last edited by itsbry

                                          @stephenw10
                                          Oh no, the WAN link was offline from 2026-06-01 15:57 until 2026-06-01 16:08. The power outage was shorter than that, although I am using logs from other devices to determine this.

                                          Only one gateway.

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S Offline
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Hmm OK. So do you know what bounced the link on re1 at 17:24? I assume re1 is the WAN.

                                            I 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2026 Rubicon Communications LLC (Netgate). All rights reserved.