WAN Failure



  • Our pfSense has been running awesome for about a year. Just today we started losing the WAN connection. After a few calls to the ISP it seems the issue is on our end. LAN is working fine. Here is what keeps popping up in the log:

    
    Oct 1 20:27:17 	apinger: /usr/local/bin/rrdtool respawning too fast, waiting 300s.
    Oct 1 20:26:17 	apinger: Error while feeding rrdtool: Broken pipe
    Oct 1 20:25:28	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    
    x20+
    
    Oct 1 20:25:26	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:26	check_reload_status: Starting packages
    Oct 1 20:25:26	php: : pfSense package system has detected an ip change 192.168.100.11 -> ... Restarting packages.
    Oct 1 20:25:26	php: : OpenNTPD is starting up.
    Oct 1 20:25:26	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:26	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:25	check_reload_status: Starting packages
    Oct 1 20:25:25	php: : pfSense package system has detected an ip change 70.190.211.37 -> ... Restarting packages.
    Oct 1 20:25:25	php: : OpenNTPD is starting up.
    Oct 1 20:25:25	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:25	php: : Creating rrd update script
    Oct 1 20:25:25	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:25	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:25:25	php: : Resyncing OpenVPN instances for interface WAN.
    Oct 1 20:25:25	php: : Creating rrd update script
    Oct 1 20:25:25	php: : Resyncing OpenVPN instances for interface WAN.
    Oct 1 20:25:25	kernel: arpresolve: can't allocate llinfo for 192.168.100.1
    Oct 1 20:24:56	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:48	kernel: arpresolve: can't allocate llinfo for 70.190.208.1
    Oct 1 20:24:45	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:40	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:38	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:37	kernel: arpresolve: can't allocate llinfo for 70.190.208.1
    Oct 1 20:24:37	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:36	dhclient[10035]: DHCPREQUEST on re0 to 192.168.100.1 port 67
    Oct 1 20:24:31	kernel: arpresolve: can't allocate llinfo for 70.190.208.1
    
    x50+
    
    Oct 1 20:24:17	kernel: arpresolve: can't allocate llinfo for 70.190.208.1
    
    

    I've never seen anything like this in the logs before…

    Any ideas?



  • That is surely a connection drop. It could be failing hardware or an ISP problem.  Does a cold reboot help?



  • Here is another thread about the llinfo message: http://forum.pfsense.org/index.php/topic,52270.0.html
    You can also try Status->Interfaces and press "Release" then "Renew" on WAN DHCP. That fixes it for me on the fly.
    I haven't seen it happen for a while, but haven't had my test box going consistently much lately. I am suspicious that the problem happens when I get a glitch (in power maybe?) that causes a switch attached to the WAN port to quickly go off an on - testing that theory now…



  • My test Alix box is on my desk running 2.1-BETA0, the WAN connects to a dumb switch on my desk which is on the "real" office LAN, which then goes out through a pfSense to the internet. WAN is DHCP. I flicked the switch off and on quickly. The test WAN went down and up and lost its DHCP address. System log entries:

    Oct 2 10:12:52 	dhclient: ifconfig vr1 inet 10.49.80.167 netmask 255.255.255.0 broadcast 10.49.80.255
    Oct 2 10:12:52 	dhclient: New IP Address (vr1): 10.49.80.167
    Oct 2 10:12:52 	dhclient: New Subnet Mask (vr1): 255.255.255.0
    Oct 2 10:12:52 	dhclient: New Broadcast Address (vr1): 10.49.80.255
    Oct 2 10:12:52 	dhclient: New Routers (vr1): 10.49.80.250
    Oct 2 10:12:52 	dhclient: Adding new routes to interface: vr1
    Oct 2 10:12:52 	dhclient: /sbin/route add default 10.49.80.250
    Oct 2 10:12:52 	dhclient: Creating resolv.conf
    Oct 2 10:12:52 	dhclient[32649]: bound to 10.49.80.167 -- renewal in 3600 seconds.
    Oct 2 10:12:52 	php: : setting up lan - 0
    Oct 2 10:12:52 	check_reload_status: rc.newwanip starting vr1
    Oct 2 10:12:52 	php: : Accept router advertisements on interface vr1
    Oct 2 10:12:52 	check_reload_status: Reloading filter
    Oct 2 10:12:53 	php: : Clearing states to old gateway 10.49.80.250.
    Oct 2 10:12:55 	dhcp6c[41242]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
    Oct 2 10:12:55 	dhcp6c[41242]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
    Oct 2 10:12:55 	dhcp6c[41242]: client6_init: failed initialize control message authentication
    Oct 2 10:12:55 	dhcp6c[41242]: client6_init: failed initialize control message authentication
    Oct 2 10:12:55 	dhcp6c[41242]: client6_init: skip opening control port
    Oct 2 10:12:55 	dhcp6c[41242]: client6_init: skip opening control port
    Oct 2 10:12:55 	dhcp6c[41242]: cfparse: cfparse: fopen(/var/etc/dhcp6c_wan.conf): No such file or directory
    Oct 2 10:12:55 	dhcp6c[41242]: cfparse: cfparse: fopen(/var/etc/dhcp6c_wan.conf): No such file or directory
    Oct 2 10:12:58 	php: : rc.newwanip: Informational is starting vr1.
    Oct 2 10:12:58 	php: : rc.newwanip: on (IP address: ) (interface: wan) (real interface: vr1).
    Oct 2 10:12:58 	php: : rc.newwanip: Failed to update wan IP, restarting...
    Oct 2 10:12:58 	check_reload_status: Configuring interface wan
    Oct 2 10:13:00 	php: : Could not find IPv4 gateway for interface (wan).
    Oct 2 10:13:00 	php: : Could not find IPv4 gateway for interface (wan).
    Oct 2 10:13:04 	php: : Deny router advertisements for interface wan
    Oct 2 10:13:04 	dhclient[51481]: dhclient already running, pid: 39992.
    Oct 2 10:13:04 	dhclient[51481]: dhclient already running, pid: 39992.
    Oct 2 10:13:04 	dhclient[51481]: exiting.
    Oct 2 10:13:04 	dhclient[51481]: exiting.
    Oct 2 10:13:04 	php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf vr1 > /tmp/vr1_output > /tmp/vr1_error_output' returned exit code '1', the output was ''
    Oct 2 10:13:04 	php: : setting up lan - 0
    Oct 2 10:13:05 	php: : Accept router advertisements on interface vr1
    Oct 2 10:13:05 	check_reload_status: Reloading filter
    Oct 2 10:13:06 	php: : Interface lan configured via wan type dhcp6
    Oct 2 10:13:08 	dhcp6c[53358]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
    Oct 2 10:13:08 	dhcp6c[53358]: dhcp6_ctl_authinit: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
    Oct 2 10:13:08 	dhcp6c[53358]: client6_init: failed initialize control message authentication
    Oct 2 10:13:08 	dhcp6c[53358]: client6_init: failed initialize control message authentication
    Oct 2 10:13:08 	dhcp6c[53358]: client6_init: skip opening control port
    Oct 2 10:13:08 	dhcp6c[53358]: client6_init: skip opening control port
    Oct 2 10:13:12 	php: : Could not find IPv4 gateway for interface (wan).
    Oct 2 10:13:12 	php: : Could not find IPv4 gateway for interface (wan).
    Oct 2 10:13:13 	check_reload_status: Updating all dyndns
    Oct 2 10:13:13 	check_reload_status: Restarting ipsec tunnels
    Oct 2 10:13:13 	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Oct 2 10:13:20 	php: : Interface lan configured via wan type dhcp6
    

    Status->Interfaces shows no IPV4 address on WAN. Clicking "Release" then "Renew" gets it back.
    This time I am not getting any "llinfo" messages, but I have had them in the past in this situation.
    Maybe there is some issue with the code that takes down the interface being still running when code starts to run to bring it up again?
    Not sure if this is the OP's problem, but certainly quick "glitches" in the network connected to WAN can cause issues like the OP's report.




  • Interesting. We've restarted the pfSense box many times and rebooted the modem. I also had the ISP reinitialize the modem. I'm wondering if our modem is just starting to go. I'll try to release renew next time it happens (just happened trying to post this) but since we've restarted a lot, I doubt that will fix anything.

    edit: I should also add that I threw in a wireless router I had sitting around and set up the same configuration and it was still experiencing the connection drops as well. This leads me to believe it isn't the pfSense box causing the issue..



  • We got up this morning to even worse connectivity. It would be up for 30 seconds and then down for 5 minutes. I ran to the store and bought a new modem and the problem went right away. I'll keep an eye on it but the only other thing I can think would do it would be the hard drive failing in my pfSense box, though SMART data is not showing any issues.


Log in to reply