WAN failing to properly reconnect after ISP issues



  • Having a problem where if the ISP (Time Warner Cable modem using DHCP on WAN) has an issue, however insignificant, the WAN will start flapping.  In the dashboard you can see DHCP obtain the correct IP but shortly after you see the interface go down and then come back up.  In the logs I see the following (this may be partial or there may be redundant/repeated info):

    Jan 12 07:54:35 php-fpm[68545]: /rc.linkup: DEVD Ethernet detached event for wan
    Jan 12 07:54:35 kernel: arpresolve: can't allocate llinfo for x.x.x.x on re1
    Jan 12 07:54:35 lighttpd[33869]: (connections.c.1540) SSL (error): 5 -1 1 Operation not permitted
    Jan 12 07:54:34 php-fpm[11692]: /rc.dyndns.update: DynDNS (xxxxxxx.sytes.net) There was an error trying to determine the public IP for interface - wan(re1). Probably interface is not a WAN interface.
    Jan 12 07:54:34 php-fpm[68545]: /rc.newipsecdns: WARNING: Setting i_dont_care_about_security_and_use_aggressive_mode_psk option because a phase 1 is configured using aggressive mode with pre-shared keys. This is not a secure configuration.
    Jan 12 07:54:34 php-fpm[68545]: /rc.newipsecdns: IPsec ERROR: Could not find phase 1 source for connection X. Omitting from configuration file.
    Jan 12 07:54:34 php-fpm[68545]: /rc.newipsecdns: IPsec ERROR: Could not find phase 1 source for connection Y. Omitting from configuration file.
    Jan 12 07:54:34 php-fpm[11692]: /rc.start_packages: Restarting/Starting all packages.
    Jan 12 07:54:34 check_reload_status: Configuring interface wan
    Jan 12 07:54:34 php-fpm[11692]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
    Jan 12 07:54:34 php-fpm[11692]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: re1).
    Jan 12 07:54:34 php-fpm[11692]: /rc.newwanip: rc.newwanip: Info: starting on re1.
    Jan 12 07:54:34 check_reload_status: Reloading filter
    Jan 12 07:54:34 php-fpm[68545]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Jan 12 07:54:34 check_reload_status: updating dyndns wan
    Jan 12 07:54:34 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:33 php-fpm[11692]: /rc.linkup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf re1 > /tmp/re1_output 2> /tmp/re1_error_output' returned exit code '1', the output was ''
    Jan 12 07:54:33 php-fpm[11692]: /rc.linkup: HOTPLUG: Configuring interface wan
    Jan 12 07:54:33 php-fpm[11692]: /rc.linkup: DEVD Ethernet attached event for wan
    Jan 12 07:54:33 php-fpm[11692]: /rc.start_packages: Restarting/Starting all packages.
    Jan 12 07:54:33 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:32 kernel: re1: link state changed to DOWN
    Jan 12 07:54:32 check_reload_status: Linkup starting re1
    Jan 12 07:54:32 php-fpm[11692]: /rc.linkup: DEVD Ethernet detached event for wan
    Jan 12 07:54:32 check_reload_status: updating dyndns wan
    Jan 12 07:54:32 check_reload_status: Starting packages
    Jan 12 07:54:32 php-fpm[11692]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Jan 12 07:54:32 php-fpm[11692]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Jan 12 07:54:32 php-fpm[11692]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Jan 12 07:54:31 check_reload_status: Restarting ipsec tunnels
    Jan 12 07:54:31 php-fpm[77516]: /rc.linkup: ROUTING: setting default route to X.X.X.X
    Jan 12 07:54:31 check_reload_status: rc.newwanip starting re1
    Jan 12 07:54:31 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:30 kernel: re1: link state changed to UP
    Jan 12 07:54:30 check_reload_status: Linkup starting re1
    Jan 12 07:54:30 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:29 check_reload_status: Starting packages
    Jan 12 07:54:29 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:29 php-fpm[11692]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - X.X.X.X -> X.X.X.X - Restarting packages.
    Jan 12 07:54:29 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:28 php-fpm[74616]: /rc.dyndns.update: DynDNS (xxxxxxx.sytes.net) There was an error trying to determine the public IP for interface - wan(re1). Probably interface is not a WAN interface.
    Jan 12 07:54:28 check_reload_status: Configuring interface wan
    Jan 12 07:54:28 php-fpm[74616]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
    Jan 12 07:54:28 php-fpm[74616]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: re1).
    Jan 12 07:54:28 php-fpm[74616]: /rc.newwanip: rc.newwanip: Info: starting on re1.
    Jan 12 07:54:28 php-fpm[74616]: /rc.newipsecdns: WARNING: Setting i_dont_care_about_security_and_use_aggressive_mode_psk option because a phase 1 is configured using aggressive mode with pre-shared keys. This is not a secure configuration.
    Jan 12 07:54:28 php-fpm[74616]: /rc.newipsecdns: IPsec ERROR: Could not find phase 1 source for connection X. Omitting from configuration file.
    Jan 12 07:54:28 php-fpm[74616]: /rc.newipsecdns: IPsec ERROR: Could not find phase 1 source for connection Y. Omitting from configuration file.
    Jan 12 07:54:28 check_reload_status: Reloading filter
    Jan 12 07:54:28 php-fpm[74616]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Jan 12 07:54:28 kernel: arpresolve: can't allocate llinfo for X.X.X.X on re1
    Jan 12 07:54:27 kernel: re1: link state changed to DOWN
    Jan 12 07:54:27 check_reload_status: Linkup starting re1
    Jan 12 07:54:27 php-fpm[77516]: /rc.linkup: HOTPLUG: Configuring interface wan
    Jan 12 07:54:27 php-fpm[77516]: /rc.linkup: DEVD Ethernet attached event for wan
    Jan 12 07:54:27 check_reload_status: rc.newwanip starting ovpns2
    Jan 12 07:54:27 kernel: ovpns2: link state changed to UP
    Jan 12 07:54:27 php-fpm[11692]: /rc.newwanip: Creating rrd update script
    Jan 12 07:54:27 kernel: ovpns2: link state changed to DOWN
    Jan 12 07:54:27 check_reload_status: Reloading filter
    Jan 12 07:54:26 php-fpm[11692]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Jan 12 07:54:26 php-fpm[11692]: /rc.newwanip: WARNING: Setting i_dont_care_about_security_and_use_aggressive_mode_psk option because a phase 1 is configured using aggressive mode with pre-shared keys. This is not a secure configuration.
    Jan 12 07:54:26 php-fpm[77516]: /rc.linkup: DEVD Ethernet detached event for wan
    Jan 12 07:54:26 check_reload_status: Reloading filter
    Jan 12 07:54:26 php-fpm[11692]: /rc.newwanip: Forcefully reloading IPsec
    Jan 12 07:54:26 php-fpm[77516]: /rc.start_packages: Restarting/Starting all packages.
    Jan 12 07:54:25 php-fpm[11692]: /rc.newwanip: phpDynDNS (xxxxxxx.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jan 12 07:54:25 php-fpm[77516]: /rc.dyndns.update: phpDynDNS (xxxxxxx.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jan 12 07:54:25 php-fpm[11692]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through X.X.X.X
    Jan 12 07:54:25 php-fpm[77516]: /rc.start_packages: Restarting/Starting all packages.
    Jan 12 07:54:25 php-fpm[11692]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through X.X.X.X
    Jan 12 07:54:25 php-fpm[11692]: /rc.newwanip: ROUTING: setting default route to X.X.X.X
    Jan 12 07:54:25 check_reload_status: updating dyndns wan
    Jan 12 07:54:24 check_reload_status: Restarting ipsec tunnels
    Jan 12 07:54:24 php-fpm[77516]: /rc.linkup: ROUTING: setting default route to X.X.X.X
    Jan 12 07:54:24 check_reload_status: rc.newwanip starting re1
    Jan 12 07:54:23 kernel: re1: link state changed to UP
    Jan 12 07:54:23 check_reload_status: Linkup starting re1
    Jan 12 07:54:19 check_reload_status: Linkup starting re1
    Jan 12 07:54:19 kernel: re1: link state changed to DOWN
    Jan 12 07:54:19 php-fpm[77516]: /rc.linkup: HOTPLUG: Configuring interface wan
    Jan 12 07:54:19 php-fpm[77516]: /rc.linkup: DEVD Ethernet attached event for wan
    Jan 12 07:54:19 php-fpm[11692]: /rc.newwanip: rc.newwanip: on (IP address: X.X.X.X) (interface: WAN[wan]) (real interface: re1).
    Jan 12 07:54:19 php-fpm[11692]: /rc.newwanip: rc.newwanip: Info: starting on re1.
    Jan 12 07:54:18 check_reload_status: Starting packages

    The hardware is a PCEngines APUD with 2GB RAM and running pfSense 2.2.6-RELEASE.
    Only rebooting will correct the issue, from what I have tried so far (which isn't much).  This problem is very reproducible.
    Attachment shows RRD quality graph of the interface last night as it was flapping after an ISP issue.



  • The NICs in the APU aren't the best, those Realteks don't play nicely with some modems. If it's not the modem that's the source of the link flapping on your WAN NIC (the problem could be entirely with the modem and nothing to do with anything you have plugged into it), putting a small switch between the APU and the cable modem will likely fix the issue. If it is the modem, it'll move the issue from your WAN NIC to the link between the switch and the modem, so adding a switch will either fix the issue, or narrow it down.



  • So we have replaced both the router hardware and the cable modem, both with completely different hardware, and we are still seeing the same issue when the WAN goes down, this time just referencing the different NIC:

    kernel: arpresolve: can't allocate llinfo for X.X.X.X on bge0 [this just continues to occur until the router is rebooted]

    Some other things to note:

    • there is an IPSec tunnel set up on WAN
    • there are two OpenVPN servers set up on WAN
    • there is a backup/failover connection on OPT1 that works fine throughout these issues (luckily, so I can remotely reboot the router when needed)

    Any suggestions or additional help is greatly appreciated!



  • Are you still seeing link state changes? "Linkup starting" logs like in the original post. The original problem was your WAN NIC going into constant link cycling.



  • Do you have the WAN duplex to autoselect? Mine does this for some reason if its just plain autoselect it connects I see ISP IP then 2 seconds later its blank then it shows 0000 then it shows ISP IP and repeats till I change the duplex type.

    If I set it to default autoselect it works fine but only does it on the autoselect for some reason.



  • CMB, it appears so, yes - from this AM after a very short WAN outage:

    php-fpm[65207]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:41 php-fpm[34263]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Feb 5 07:17:40 check_reload_status: Starting packages
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.0.8.1 - Restarting packages.
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: rc.newwanip: on (IP address: 10.0.8.1) (interface: []) (real interface: ovpns1).
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
    Feb 5 07:17:40 php-fpm[98159]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Feb 5 07:17:40 check_reload_status: Starting packages
    Feb 5 07:17:40 php-fpm[34263]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.x -> x.x.x.x - Restarting packages.
    Feb 5 07:17:39 php-fpm[65207]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:39 php-fpm[65207]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.0.8.1 - Restarting packages.
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: rc.newwanip: on (IP address: 10.0.8.1) (interface: []) (real interface: ovpns1).
    Feb 5 07:17:39 php-fpm[65207]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
    Feb 5 07:17:39 check_reload_status: Starting packages
    Feb 5 07:17:39 php-fpm[30650]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.x -> x.x.x.x - Restarting packages.
    Feb 5 07:17:38 check_reload_status: rc.newwanip starting ovpns2
    Feb 5 07:17:38 kernel: ovpns2: link state changed to UP
    Feb 5 07:17:38 php-fpm[34263]: /rc.newwanip: Creating rrd update script
    Feb 5 07:17:37 kernel: ovpns2: link state changed to DOWN
    Feb 5 07:17:37 check_reload_status: rc.newwanip starting ovpns1
    Feb 5 07:17:37 kernel: ovpns1: link state changed to UP
    Feb 5 07:17:37 check_reload_status: Starting packages
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Feb 5 07:17:37 check_reload_status: Starting packages
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> - Restarting packages.
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: []) (real interface: ovpns1).
    Feb 5 07:17:37 php-fpm[65207]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
    Feb 5 07:17:37 kernel: ovpns1: link state changed to DOWN
    Feb 5 07:17:37 php-fpm[34263]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Feb 5 07:17:37 check_reload_status: rc.newwanip starting ovpns2
    Feb 5 07:17:37 kernel: ovpns2: link state changed to UP
    Feb 5 07:17:37 php-fpm[34263]: /rc.newwanip: Forcefully reloading IPsec
    Feb 5 07:17:37 php-fpm[30650]: /rc.newwanip: Creating rrd update script
    Feb 5 07:17:36 kernel: ovpns2: link state changed to DOWN
    Feb 5 07:17:36 check_reload_status: rc.newwanip starting ovpns1
    Feb 5 07:17:36 kernel: ovpns1: link state changed to UP
    Feb 5 07:17:36 kernel: ovpns1: link state changed to DOWN
    Feb 5 07:17:36 php-fpm[30650]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Feb 5 07:17:36 php-fpm[65207]: /rc.dyndns.update: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:36 php-fpm[65207]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:36 php-fpm[30650]: /rc.newwanip: Forcefully reloading IPsec
    Feb 5 07:17:36 php-fpm[34263]: /rc.newwanip: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:36 php-fpm[65207]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Feb 5 07:17:36 php-fpm[34263]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:36 php-fpm[34263]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:36 php-fpm[34263]: /rc.newwanip: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.0.8.1 - Restarting packages.
    Feb 5 07:17:35 check_reload_status: Reloading filter
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: on (IP address: 10.0.8.1) (interface: []) (real interface: ovpns1).
    Feb 5 07:17:35 php-fpm[34263]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
    Feb 5 07:17:35 php-fpm[30650]: /rc.newwanip: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:35 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:35 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:35 php-fpm[30650]: /rc.newwanip: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:35 php-fpm[34263]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.1.8.1 - Restarting packages.
    Feb 5 07:17:34 check_reload_status: Reloading filter
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: on (IP address: 10.1.8.1) (interface: []) (real interface: ovpns2).
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.0.8.1 - Restarting packages.
    Feb 5 07:17:34 check_reload_status: Reloading filter
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: on (IP address: 10.0.8.1) (interface: []) (real interface: ovpns1).
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
    Feb 5 07:17:34 check_reload_status: Starting packages
    Feb 5 07:17:34 php-fpm[30650]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.x -> x.x.x.x - Restarting packages.
    Feb 5 07:17:32 php-fpm[34263]: /interfaces.php: Creating rrd update script
    Feb 5 07:17:32 check_reload_status: rc.newwanip starting ovpns2
    Feb 5 07:17:32 php-fpm[34263]: /interfaces.php: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:32 php-fpm[34263]: /interfaces.php: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:32 kernel: ovpns2: link state changed to UP
    Feb 5 07:17:32 php-fpm[30650]: /rc.newwanip: Creating rrd update script
    Feb 5 07:17:31 kernel: ovpns2: link state changed to DOWN
    Feb 5 07:17:31 check_reload_status: rc.newwanip starting ovpns1
    Feb 5 07:17:31 kernel: ovpns1: link state changed to UP
    Feb 5 07:17:31 check_reload_status: Reloading filter
    Feb 5 07:17:31 kernel: ovpns1: link state changed to DOWN
    Feb 5 07:17:31 php-fpm[30650]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Feb 5 07:17:31 php-fpm[30650]: /rc.newwanip: Forcefully reloading IPsec
    Feb 5 07:17:30 php-fpm[30650]: /rc.newwanip: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:30 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:30 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:30 php-fpm[30650]: /rc.newwanip: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:30 check_reload_status: updating dyndns wan
    Feb 5 07:17:29 php-fpm[30650]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:29 php-fpm[30650]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:29 check_reload_status: Starting packages
    Feb 5 07:17:29 php-fpm[30650]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.x -> x.x.x.x - Restarting packages.
    Feb 5 07:17:29 check_reload_status: Restarting ipsec tunnels
    Feb 5 07:17:29 php-fpm[34263]: /interfaces.php: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:29 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:28 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:27 php-fpm[34263]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Feb 5 07:17:27 check_reload_status: rc.newwanip starting ovpns2
    Feb 5 07:17:27 kernel: ovpns2: link state changed to UP
    Feb 5 07:17:27 php-fpm[30650]: /rc.newwanip: Creating rrd update script
    Feb 5 07:17:27 kernel: ovpns2: link state changed to DOWN
    Feb 5 07:17:27 check_reload_status: rc.newwanip starting ovpns1
    Feb 5 07:17:27 kernel: ovpns1: link state changed to UP
    Feb 5 07:17:26 kernel: ovpns1: link state changed to DOWN
    Feb 5 07:17:26 php-fpm[30650]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Feb 5 07:17:26 check_reload_status: Reloading filter
    Feb 5 07:17:26 php-fpm[30650]: /rc.newwanip: Forcefully reloading IPsec
    Feb 5 07:17:25 php-fpm[30650]: /rc.newwanip: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:25 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:25 php-fpm[30650]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:25 php-fpm[30650]: /rc.newwanip: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:25 check_reload_status: Starting packages
    Feb 5 07:17:25 php-fpm[98159]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.x -> x.x.x.x - Restarting packages.
    Feb 5 07:17:24 php-fpm[30650]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:24 php-fpm[30650]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:24 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:24 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:24 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:24 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:23 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:23 check_reload_status: rc.newwanip starting ovpns2
    Feb 5 07:17:23 kernel: ovpns2: link state changed to UP
    Feb 5 07:17:23 php-fpm[98159]: /rc.newwanip: Creating rrd update script
    Feb 5 07:17:22 kernel: ovpns2: link state changed to DOWN
    Feb 5 07:17:22 check_reload_status: rc.newwanip starting ovpns1
    Feb 5 07:17:22 kernel: ovpns1: link state changed to UP
    Feb 5 07:17:22 php-fpm[30650]: /rc.dyndns.update: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:22 check_reload_status: Reloading filter
    Feb 5 07:17:22 kernel: ovpns1: link state changed to DOWN
    Feb 5 07:17:22 php-fpm[98159]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Feb 5 07:17:22 check_reload_status: Reloading filter
    Feb 5 07:17:22 php-fpm[98159]: /rc.newwanip: Forcefully reloading IPsec
    Feb 5 07:17:21 check_reload_status: Restarting ipsec tunnels
    Feb 5 07:17:21 php-fpm[30650]: /rc.linkup: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:21 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:21 php-fpm[98159]: /rc.newwanip: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:21 php-fpm[30650]: /rc.linkup: HOTPLUG: Configuring interface wan
    Feb 5 07:17:21 php-fpm[30650]: /rc.linkup: DEVD Ethernet attached event for wan
    Feb 5 07:17:21 php-fpm[98159]: /rc.newwanip: Removing static route for monitor 208.67.220.222 and adding a new route through x.x.x.x
    Feb 5 07:17:21 php-fpm[98159]: /rc.newwanip: Removing static route for monitor 24.25.5.60 and adding a new route through x.x.x.x
    Feb 5 07:17:21 php-fpm[98159]: /rc.newwanip: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:20 php-fpm[98159]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:20 php-fpm[98159]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:20 php-fpm[65207]: /rc.linkup: DEVD Ethernet detached event for wan
    Feb 5 07:17:20 check_reload_status: updating dyndns wan
    Feb 5 07:17:18 check_reload_status: Restarting ipsec tunnels
    Feb 5 07:17:18 php-fpm[30650]: /rc.linkup: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:18 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:18 check_reload_status: Configuring interface wan
    Feb 5 07:17:18 php-fpm[98159]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
    Feb 5 07:17:18 php-fpm[98159]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:18 php-fpm[98159]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:18 php-fpm[30650]: /rc.linkup: HOTPLUG: Configuring interface wan
    Feb 5 07:17:18 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:18 php-fpm[30650]: /rc.linkup: DEVD Ethernet attached event for wan
    Feb 5 07:17:17 php-fpm[65207]: /rc.dyndns.update: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:17:17 php-fpm[98159]: /rc.linkup: DEVD Ethernet detached event for wan
    Feb 5 07:17:17 check_reload_status: updating dyndns wan
    Feb 5 07:17:16 check_reload_status: Restarting ipsec tunnels
    Feb 5 07:17:16 php-fpm[65207]: /rc.linkup: ROUTING: setting default route to x.x.x.x
    Feb 5 07:17:16 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:16 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:16 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:16 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:15 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:15 kernel: bge0: link state changed to UP
    Feb 5 07:17:15 check_reload_status: Linkup starting bge0
    Feb 5 07:17:15 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:15 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:14 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:14 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:14 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:12 php-fpm[98159]: /rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf bge0 > /tmp/bge0_output 2> /tmp/bge0_error_output' returned exit code '1', the output was ''
    Feb 5 07:17:12 kernel: bge0: link state changed to DOWN
    Feb 5 07:17:12 check_reload_status: Linkup starting bge0
    Feb 5 07:17:12 php-fpm[65207]: /rc.linkup: HOTPLUG: Configuring interface wan
    Feb 5 07:17:12 php-fpm[65207]: /rc.linkup: DEVD Ethernet attached event for wan
    Feb 5 07:17:08 check_reload_status: Syncing firewall
    Feb 5 07:17:08 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:17:08 php-fpm[65207]: /rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf bge0 > /tmp/bge0_output 2> /tmp/bge0_error_output' returned exit code '15', the output was ''
    Feb 5 07:17:08 php-fpm[98159]: /rc.linkup: DEVD Ethernet detached event for wan
    Feb 5 07:17:08 check_reload_status: updating dyndns wan
    Feb 5 07:17:08 check_reload_status: Linkup starting bge0
    Feb 5 07:17:08 kernel: bge0: link state changed to UP
    Feb 5 07:17:05 check_reload_status: Restarting ipsec tunnels
    Feb 5 07:17:05 php-fpm[30650]: /rc.linkup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf bge0 > /tmp/bge0_output 2> /tmp/bge0_error_output' returned exit code '1', the output was ''
    Feb 5 07:17:05 kernel: bge0: link state changed to DOWN
    Feb 5 07:17:05 check_reload_status: Linkup starting bge0
    Feb 5 07:17:04 check_reload_status: Configuring interface wan
    Feb 5 07:17:04 php-fpm[98159]: /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
    Feb 5 07:17:04 php-fpm[98159]: /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: bge0).
    Feb 5 07:17:04 php-fpm[98159]: /rc.newwanip: rc.newwanip: Info: starting on bge0.
    Feb 5 07:17:04 php-fpm[30650]: /rc.linkup: HOTPLUG: Configuring interface wan
    Feb 5 07:17:04 php-fpm[30650]: /rc.linkup: DEVD Ethernet attached event for wan
    Feb 5 07:17:04 check_reload_status: Linkup starting bge0
    Feb 5 07:17:04 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:04 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0
    Feb 5 07:17:04 kernel: bge0: link state changed to UP
    Feb 5 07:17:01 kernel: bge0: link state changed to DOWN
    Feb 5 07:17:01 check_reload_status: Linkup starting bge0
    Feb 5 07:17:01 php-fpm[30650]: /rc.dyndns.update: DynDNS (somedomain.sytes.net) There was an error trying to determine the public IP for interface - wan(bge0). Probably interface is not a WAN interface.
    Feb 5 07:17:00 php-fpm[98159]: /rc.linkup: DEVD Ethernet detached event for wan
    Feb 5 07:17:00 check_reload_status: Reloading filter
    Feb 5 07:17:00 php-fpm[65207]: /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
    Feb 5 07:16:57 php-fpm[98159]: /rc.dyndns.update: phpDynDNS (somedomain.sytes.net): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Feb 5 07:16:56 check_reload_status: rc.newwanip starting bge0
    Feb 5 07:16:56 kernel: bge0: link state changed to UP
    Feb 5 07:16:56 check_reload_status: Linkup starting bge0
    Feb 5 07:16:54 php-fpm[30650]: /rc.start_packages: Restarting/Starting all packages.
    Feb 5 07:16:54 kernel: arpresolve: can't allocate llinfo for x.x.x.x on bge0



  • So I took UnCoNoob's advice and changed the WAN NIC from Autoselect to 1000Mbps Full Duplex and the problems stopped without having to reboot the router.  I don't know if this fixed the problem or simply reset the WAN connection in a different way and it will happen again next outage.  I will let you know.

    Thank you both for the assistance.



  • you should be able to replicate a outage by unplugging the cable from your modem, The one that comes from the wall socket. I wouldnt pull out the cable between the modem and PFsense as it could be a miss match in communication from the modem and router.

    As I say mine works fine on the default autoselect (one at the very top) but not when its just plain autoselect. But if you know what it always uses then setting it manually like you have should work. Just run some test to make sure it is not affecting performance.