WAN Failure
-
Our pfSense has been running awesome for about a year. Just today we started losing the WAN connection. After a few calls to the ISP it seems the issue is on our end. LAN is working fine. Here is what keeps popping up in the log:
Oct 1 20:27:17 apinger: /usr/local/bin/rrdtool respawning too fast, waiting 300s. Oct 1 20:26:17 apinger: Error while feeding rrdtool: Broken pipe Oct 1 20:25:28 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 x20+ Oct 1 20:25:26 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:26 check_reload_status: Starting packages Oct 1 20:25:26 php: : pfSense package system has detected an ip change 192.168.100.11 -> ... Restarting packages. Oct 1 20:25:26 php: : OpenNTPD is starting up. Oct 1 20:25:26 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:26 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:25 check_reload_status: Starting packages Oct 1 20:25:25 php: : pfSense package system has detected an ip change 70.190.211.37 -> ... Restarting packages. Oct 1 20:25:25 php: : OpenNTPD is starting up. Oct 1 20:25:25 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:25 php: : Creating rrd update script Oct 1 20:25:25 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:25 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:25:25 php: : Resyncing OpenVPN instances for interface WAN. Oct 1 20:25:25 php: : Creating rrd update script Oct 1 20:25:25 php: : Resyncing OpenVPN instances for interface WAN. Oct 1 20:25:25 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 Oct 1 20:24:56 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:48 kernel: arpresolve: can't allocate llinfo for 70.190.208.1 Oct 1 20:24:45 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:40 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:38 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:37 kernel: arpresolve: can't allocate llinfo for 70.190.208.1 Oct 1 20:24:37 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:36 dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67 Oct 1 20:24:31 kernel: arpresolve: can't allocate llinfo for 70.190.208.1 x50+ Oct 1 20:24:17 kernel: arpresolve: can't allocate llinfo for 70.190.208.1I've never seen anything like this in the logs before…
Any ideas?
-
That is surely a connection drop. It could be failing hardware or an ISP problem. Does a cold reboot help?
-
Here is another thread about the llinfo message: http://forum.pfsense.org/index.php/topic,52270.0.html
You can also try Status->Interfaces and press "Release" then "Renew" on WAN DHCP. That fixes it for me on the fly.
I haven't seen it happen for a while, but haven't had my test box going consistently much lately. I am suspicious that the problem happens when I get a glitch (in power maybe?) that causes a switch attached to the WAN port to quickly go off an on - testing that theory now… -
My test Alix box is on my desk running 2.1-BETA0, the WAN connects to a dumb switch on my desk which is on the "real" office LAN, which then goes out through a pfSense to the internet. WAN is DHCP. I flicked the switch off and on quickly. The test WAN went down and up and lost its DHCP address. System log entries:
Oct 2 10:12:52 dhclient: ifconfig vr1 inet 10.49.80.167 netmask 255.255.255.0 broadcast 10.49.80.255 Oct 2 10:12:52 dhclient: New IP Address (vr1): 10.49.80.167 Oct 2 10:12:52 dhclient: New Subnet Mask (vr1): 255.255.255.0 Oct 2 10:12:52 dhclient: New Broadcast Address (vr1): 10.49.80.255 Oct 2 10:12:52 dhclient: New Routers (vr1): 10.49.80.250 Oct 2 10:12:52 dhclient: Adding new routes to interface: vr1 Oct 2 10:12:52 dhclient: /sbin/route add default 10.49.80.250 Oct 2 10:12:52 dhclient: Creating resolv.conf Oct 2 10:12:52 dhclient[32649]: bound to 10.49.80.167 -- renewal in 3600 seconds. Oct 2 10:12:52 php: : setting up lan - 0 Oct 2 10:12:52 check_reload_status: rc.newwanip starting vr1 Oct 2 10:12:52 php: : Accept router advertisements on interface vr1 Oct 2 10:12:52 check_reload_status: Reloading filter Oct 2 10:12:53 php: : Clearing states to old gateway 10.49.80.250. Oct 2 10:12:55 dhcp6c[41242]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory Oct 2 10:12:55 dhcp6c[41242]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory Oct 2 10:12:55 dhcp6c[41242]: client6_init: failed initialize control message authentication Oct 2 10:12:55 dhcp6c[41242]: client6_init: failed initialize control message authentication Oct 2 10:12:55 dhcp6c[41242]: client6_init: skip opening control port Oct 2 10:12:55 dhcp6c[41242]: client6_init: skip opening control port Oct 2 10:12:55 dhcp6c[41242]: cfparse: cfparse: fopen(/var/etc/dhcp6c_wan.conf): No such file or directory Oct 2 10:12:55 dhcp6c[41242]: cfparse: cfparse: fopen(/var/etc/dhcp6c_wan.conf): No such file or directory Oct 2 10:12:58 php: : rc.newwanip: Informational is starting vr1. Oct 2 10:12:58 php: : rc.newwanip: on (IP address: ) (interface: wan) (real interface: vr1). Oct 2 10:12:58 php: : rc.newwanip: Failed to update wan IP, restarting... Oct 2 10:12:58 check_reload_status: Configuring interface wan Oct 2 10:13:00 php: : Could not find IPv4 gateway for interface (wan). Oct 2 10:13:00 php: : Could not find IPv4 gateway for interface (wan). Oct 2 10:13:04 php: : Deny router advertisements for interface wan Oct 2 10:13:04 dhclient[51481]: dhclient already running, pid: 39992. Oct 2 10:13:04 dhclient[51481]: dhclient already running, pid: 39992. Oct 2 10:13:04 dhclient[51481]: exiting. Oct 2 10:13:04 dhclient[51481]: exiting. Oct 2 10:13:04 php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf vr1 > /tmp/vr1_output > /tmp/vr1_error_output' returned exit code '1', the output was '' Oct 2 10:13:04 php: : setting up lan - 0 Oct 2 10:13:05 php: : Accept router advertisements on interface vr1 Oct 2 10:13:05 check_reload_status: Reloading filter Oct 2 10:13:06 php: : Interface lan configured via wan type dhcp6 Oct 2 10:13:08 dhcp6c[53358]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory Oct 2 10:13:08 dhcp6c[53358]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory Oct 2 10:13:08 dhcp6c[53358]: client6_init: failed initialize control message authentication Oct 2 10:13:08 dhcp6c[53358]: client6_init: failed initialize control message authentication Oct 2 10:13:08 dhcp6c[53358]: client6_init: skip opening control port Oct 2 10:13:08 dhcp6c[53358]: client6_init: skip opening control port Oct 2 10:13:12 php: : Could not find IPv4 gateway for interface (wan). Oct 2 10:13:12 php: : Could not find IPv4 gateway for interface (wan). Oct 2 10:13:13 check_reload_status: Updating all dyndns Oct 2 10:13:13 check_reload_status: Restarting ipsec tunnels Oct 2 10:13:13 check_reload_status: Restarting OpenVPN tunnels/interfaces Oct 2 10:13:20 php: : Interface lan configured via wan type dhcp6Status->Interfaces shows no IPV4 address on WAN. Clicking "Release" then "Renew" gets it back.
This time I am not getting any "llinfo" messages, but I have had them in the past in this situation.
Maybe there is some issue with the code that takes down the interface being still running when code starts to run to bring it up again?
Not sure if this is the OP's problem, but certainly quick "glitches" in the network connected to WAN can cause issues like the OP's report.
-
Interesting. We've restarted the pfSense box many times and rebooted the modem. I also had the ISP reinitialize the modem. I'm wondering if our modem is just starting to go. I'll try to release renew next time it happens (just happened trying to post this) but since we've restarted a lot, I doubt that will fix anything.
edit: I should also add that I threw in a wireless router I had sitting around and set up the same configuration and it was still experiencing the connection drops as well. This leads me to believe it isn't the pfSense box causing the issue..
-
We got up this morning to even worse connectivity. It would be up for 30 seconds and then down for 5 minutes. I ran to the store and bought a new modem and the problem went right away. I'll keep an eye on it but the only other thing I can think would do it would be the hard drive failing in my pfSense box, though SMART data is not showing any issues.