Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    WAN dropped connection on 2.1

    Scheduled Pinned Locked Moved General pfSense Questions
    37 Posts 8 Posters 9.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B Offline
      badams
      last edited by

      Hey everyone,
      hoping someone can provide some insight with a problem ive been having for a while now.

      I have a pfSense box running, but every once in a while (completely random), the WAN connection with go down.
      Originally I had it set with a static IP as we are paying for one through our ISP, but I changed it to DHCP to see if that made a difference.
      I have a switch between the modem and pfSense (we have multiple IPs so the switch allows us to connect multiple routers to the modem).
      The second router (not pfSense) never has this dropping issue.
      The only way I have been able to reconnect once it goes down is by rebooting the whole pfSense box. Rebooting the switch or the modem has no effect.

      I also have these logs to hopefully better explain:
      Dec 18 01:07:27 apinger: ALARM: WAN(216.x6.1xx.1) *** down ***
      Dec 18 08:12:23 apinger: Starting Alarm Pinger, apinger(22865)
      Dec 18 08:12:24 apinger: SIGHUP received, reloading configuration.
      Dec 18 11:43:46 apinger: ALARM: WAN(216.x6.1xx.1) *** delay ***
      Dec 18 11:43:57 apinger: alarm canceled: WAN(216.x6.1xx.1) *** delay ***

      You can see in the logs, it went down at 1am, and didnt come back up until pfSense was rebooted after 8am.

      Im at a loss as to where I start looking…. Is it a configuration issue, or potentially hardware issue?

      1 Reply Last reply Reply Quote 0
      • V Offline
        vincom
        last edited by

        my first guess is a bad nic, swith the nics wan<>lan and see if lan drops out

        1 Reply Last reply Reply Quote 0
        • T Offline
          trussent
          last edited by

          Yep, I have the same problem with 2.1.  I have changed the nics, twice, and tested with and without multi-wan, and individually connected and disconnected my power supply filtering of dsl & cable sources.  Apinger is showing drops of the wan and opt1 every 30-60 min each.  This is not a hardware problem.  Here is a sample from the gateway logs:

          Jan 13 20:02:34 apinger: alarm canceled: OPT1_DHCP(8.8.8.8) *** down ***
          Jan 13 20:02:45 apinger: SIGHUP received, reloading configuration.
          Jan 13 20:28:12 apinger: ALARM: WAN_DHCP(8.8.4.4) *** down ***
          Jan 13 20:39:15 apinger: alarm canceled: WAN_DHCP(8.8.4.4) *** down ***
          Jan 13 20:39:18 apinger: SIGHUP received, reloading configuration.
          Jan 13 20:39:21 apinger: SIGHUP received, reloading configuration.
          Jan 13 20:41:14 apinger: ALARM: OPT1_DHCP(8.8.8.8) *** down ***
          Jan 13 20:45:28 apinger: SIGHUP received, reloading configuration.
          Jan 13 20:45:28 apinger: alarm canceled: OPT1_DHCP(8.8.8.8) *** down **

          This behavior is simply reliably ridiculous.  I am about to mod the code for gateway.widgets.php to call status_interfaces.php?action=Renew whenever this happens and rely on the web interface to fix the gateway fails unless someone here has a better idea.

          :-\

          1 Reply Last reply Reply Quote 0
          • P Offline
            peterlinuxgeek
            last edited by

            PLS, Next time start a new thread rather than hijacking an existing one…

            Had once simular situation, ISP replaced a switch down the street, problem solved.
            Took me 4 days to convince them that problem was on their end...

            Each time apinger went down, rules in pfSense got reloaded when it came back up.
            Took long enough to make for a bunch of unhappy users.

            You can alter the apinger treshold so it doesn't panic that fast.

            It can be a NIC/driver problem on your side.
            or the nic in the ISPs modem...

            Good luck.

            Peter

            1 Reply Last reply Reply Quote 0
            • T Offline
              trussent
              last edited by

              Whatever.

              Look, I have two ISPs.  Not one.  Both fail.  It is not some hardware fail at the ISP.  It is either a dump from the ue usb nic driver or it is a general problem with multiple gateways in 2.1.  I never had this problem with 2.0.1 rel.

              What I need right now is code to cause an apinger alert to reload the interface, or a real fix to whatever was broken in the 2.1 build for multi wan applications.

              stand or deliver.

              1 Reply Last reply Reply Quote 0
              • T Offline
                trussent
                last edited by

                Dropped back to 1.2.3 and axe0,1 for wan and opt1.  No problems yet.

                J'accuse, the ue usb nic driver in 2.1 as the unreliable driver.

                1 Reply Last reply Reply Quote 0
                • stephenw10S Online
                  stephenw10 Netgate Administrator
                  last edited by

                  If you're using USB NICs that's the first suspect in this sort of case. Is there no way you can use something else?
                  Your logs indicate that the connection simply goes directly down rather than first experiencing delay or packet loss is that correct?
                  You could try disabling gateway monitoring for the bad WAN connection and see if that makes any difference.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • T Offline
                    trussent
                    last edited by

                    I can't test this without usb nics easily with the equipment I currently have available.  Right now, I have Pfsense 2.1 and 1.2.3 installed on SD cards connected to an old EEEpc with unusable a SSD c drive.  Both 2.1 and 1.2.3 are configured with one onboard nic to the lan and two usb nics to the wan and opt1.  One of the usb nics is a dlink and the other is from belkin.  Both worked fine until 2.1.  Both are working fine now on a clean 1.2.3 install from last night.  While the hardware may be faulty in some odd way, having swapped connectors, computers, and operating system versions makes hardware an unlikely source of the problem.

                    I noted that when firing up 2.1 both usb nics came in on the ue usb driver, and that in 1.2.3 both came up on the axe driver.  The onboard nic is using ale in 1.2.3 and 2.1.  My current hypothesis is that the ue driver incorporated in 2.1 is flaky.  I have been a UNIX sysadmin since SCO in the 80s; so, am somewhat comfortable under the hood, but do not know how pfsense/bsd selects the driver appropriate for the usb nic, nor how to force it to choose an older/different driver.  If there were a way to do that, then I could test my hypothesis.

                    Specifically, regarding your question about the connections just dropping out, yes that is correct.  No warning, delays or odd behavior.  Further, using the interface status page to release and renew the interface always works to bring it back up; although, whatever is causing the connection to drop still occurs.

                    I have tried using and disabling gateway monitoring.  It has no effect on the behavior in 2.1.  The log you see above is from a time when I had it on.  I am in the process of setting up gateways for 1.2.3 to see if it makes any difference with that OS version.

                    Thanks Steve.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S Online
                      stephenw10 Netgate Administrator
                      last edited by

                      They are still using the axe(4) driver it's just now wrapped in the ue nomenclature for some reason I've not understood.
                      There may well have been a regression in the driver for your particular device. If you google freebsd axe regression there's no shortage of reports!
                      You're not seeing either of the axe diagnostic outputs: axe0: watchdog timeout  or  axe0: no memory for rx list?

                      Steve

                      1 Reply Last reply Reply Quote 0
                      • T Offline
                        trussent
                        last edited by

                        Well s**t Steve, I never seen any axe diagnostics, but where should I find them, on the console standard out like gui admin login reports where axe diagnostics are not, or in some special log where I don't know to look?

                        Glad to hear that axe is around in 2.1.  I think all I need is to properly assign it to the usb nic interface, but I don't know how to do that at this point.

                        Until then though, 1.2.3 seems to be doing the job I need it to do.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S Online
                          stephenw10 Netgate Administrator
                          last edited by

                          Any output from the axe driver should appear in the system logs. Since you didn't post it above I assumed you didn't but worth mentioning in case.

                          You could try the new 2.1.1 snapshots. A lot of bug fixes have gone into that, maybe one relevant to you.
                          https://forum.pfsense.org/index.php/topic,71546.0.html

                          Steve

                          1 Reply Last reply Reply Quote 0
                          • T Offline
                            trussent
                            last edited by

                            nope.  I just updated from 1.2.3 to  2.1.3-RELEASE (i386) and the usb nic driver problem is still present.

                            I'll mess with it for a few days, but then drop back to 1.2.3 if there is no resolution.

                            1 Reply Last reply Reply Quote 0
                            • T Offline
                              trussent
                              last edited by

                              While it is probably a coincidence, both my wan and opt1 links served by the ue usb driver dropped nearly simultaneously after almost exactly 3000 minutes of up time (+/- 5 minutes).  So, it could be a bad constant somewhere, or more likely, the "bad" event happened on the router and knocked both connections down in short order and it just happened to be on a unusual number boundary.

                              1 Reply Last reply Reply Quote 0
                              • T Offline
                                trussent
                                last edited by

                                Here is the system log, where you can see both connections dropping and not restarting

                                May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                                May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                May 21 20:20:54 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                                May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 15
                                May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 15
                                May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 14
                                May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 14
                                May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                                May 21 20:20:54 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                                May 21 21:05:13 check_reload_status: rc.newwanip starting ue0
                                May 21 21:05:17 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                                May 21 21:05:17 php: rc.newwanip: rc.newwanip: on (IP address: 50.130.29.69) (interface: OPT1[opt1]) (real interface: ue0).
                                May 21 21:05:19 php: rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.130.28.1
                                May 21 21:05:19 php: rc.newwanip: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                May 21 21:05:19 php: rc.newwanip: MONITOR: GW_OPT1 is down, removing from routing group combine
                                May 21 21:05:19 php: rc.newwanip: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                May 21 21:05:23 check_reload_status: rc.newwanip starting ue1
                                May 21 21:05:24 php: rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
                                May 21 21:05:24 php: rc.newwanip: Creating rrd update script
                                May 21 21:05:26 php: rc.newwanip: rc.newwanip: Informational is starting ue1.
                                May 21 21:05:26 php: rc.newwanip: rc.newwanip: on (IP address: 172.23.23.53) (interface: WAN[wan]) (real interface: ue1).
                                May 21 21:05:26 php: rc.newwanip: ROUTING: setting default route to 172.23.23.23
                                May 21 21:05:26 php: rc.newwanip: Removing static route for monitor 76.29.224.1 and adding a new route through 172.23.23.23
                                May 21 21:05:26 php: rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.130.28.1
                                May 21 21:05:26 php: rc.newwanip: pfSense package system has detected an ip change 0.0.0.0 -> 50.130.29.69 … Restarting packages.
                                May 21 21:05:26 check_reload_status: Starting packages
                                May 21 21:05:26 check_reload_status: Reloading filter
                                May 21 21:05:30 check_reload_status: updating dyndns GW_OPT1
                                May 21 21:05:30 check_reload_status: Restarting ipsec tunnels
                                May 21 21:05:30 check_reload_status: Restarting OpenVPN tunnels/interfaces
                                May 21 21:05:30 check_reload_status: Reloading filter
                                May 21 21:05:30 php: rc.start_packages: Restarting/Starting all packages.
                                May 21 21:05:32 php: rc.newwanip: Resyncing OpenVPN instances for interface WAN.
                                May 21 21:05:32 php: rc.newwanip: Creating rrd update script
                                May 21 21:05:35 php: rc.newwanip: pfSense package system has detected an ip change 0.0.0.0 -> 172.23.23.53 ... Restarting packages.

                                1 Reply Last reply Reply Quote 0
                                • T Offline
                                  trussent
                                  last edited by

                                  And as usual, restarting the interfaces merely required punching the release and restart button on the interface form (http://192.168.3.1/status_interfaces.php) for WAN and OPT1.

                                  1 Reply Last reply Reply Quote 0
                                  • T Offline
                                    trussent
                                    last edited by

                                    and here are the log entries starting at the fail at 20:20:14

                                    May 21 18:55:37 check_reload_status: Reloading filter
                                    May 21 20:20:14 check_reload_status: Linkup starting ue0
                                    May 21 20:20:14 kernel: ue0: link state changed to DOWN
                                    May 21 20:20:14 kernel: ue0: link state changed to UP
                                    May 21 20:20:14 kernel: ue1: link state changed to DOWN
                                    May 21 20:20:14 kernel: ue1: link state changed to UP
                                    May 21 20:20:14 check_reload_status: Linkup starting ue0
                                    May 21 20:20:14 check_reload_status: Linkup starting ue1
                                    May 21 20:20:14 check_reload_status: Linkup starting ue1
                                    May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for opt1
                                    May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for wan
                                    May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface wan
                                    May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for wan
                                    May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for opt1
                                    May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface opt1
                                    May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:23 php: rc.linkup: Clearing states to old gateway 172.23.23.23.
                                    May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:24 check_reload_status: rc.newwanip starting ue0
                                    May 21 20:20:24 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:24 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:24 php: rc.linkup: Clearing states to old gateway 50.130.28.1.
                                    May 21 20:20:24 check_reload_status: rc.newwanip starting ue1
                                    May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: on (IP address: ) (interface: OPT1[opt1]) (real interface: ue0).
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: Failed to update opt1 IP, restarting…
                                    May 21 20:20:27 check_reload_status: Configuring interface opt1
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: Informational is starting ue1.
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: ue1).
                                    May 21 20:20:27 php: rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
                                    May 21 20:20:28 check_reload_status: Configuring interface wan
                                    May 21 20:20:28 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:28 check_reload_status: updating dyndns wan
                                    May 21 20:20:28 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:28 php: rc.linkup: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid ale0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.6 Copyright 2004-2014 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 17 leases to leases file. Can't backup lease database /var/db/dhcpd.leases to /var/db/dhcpd.leases~: File exists Listening on BPF/ale0/00:23:54:0b:73:24/192.168.3.0/24 Sending on BPF/ale0/00:23:54:0b:73:24/192.168.3.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get this soft
                                    May 21 20:20:28 check_reload_status: updating dyndns opt1
                                    May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:30 php: rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_opt1.conf ue0 > /tmp/ue0_output 2> /tmp/ue0_error_output' returned exit code '1', the output was ''
                                    May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:32 php: rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf ue1 > /tmp/ue1_output 2> /tmp/ue1_error_output' returned exit code '1', the output was ''
                                    May 21 20:20:32 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:32 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:43 check_reload_status: updating dyndns GW_WAN
                                    May 21 20:20:43 check_reload_status: Restarting ipsec tunnels
                                    May 21 20:20:43 check_reload_status: Restarting OpenVPN tunnels/interfaces
                                    May 21 20:20:43 check_reload_status: Reloading filter
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:43 check_reload_status: updating dyndns GW_OPT1
                                    May 21 20:20:43 check_reload_status: Restarting ipsec tunnels
                                    May 21 20:20:43 check_reload_status: Restarting OpenVPN tunnels/interfaces
                                    May 21 20:20:43 check_reload_status: Reloading filter
                                    May 21 20:20:44 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:44 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:50 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:50 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_WAN is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_OPT1 is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_WAN is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_OPT1 is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                    May 21 20:20:50 php: rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                    May 21 20:20:50 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:50 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                                    May 21 20:20:50 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                    May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                                    May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                                    May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                                    May 21 20:20:54 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                                    May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                                    May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 15
                                    May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 15
                                    May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 14
                                    May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 14
                                    May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                                    May 21 20:20:54 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                                    May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                                    May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                                    May 21 21:05:13 check_reload_status: rc.newwanip starting ue0
                                    May 21 21:05:17 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                                    May 21 21:05:17 php: rc.newwanip: rc.newwanip: on (IP address: 50.130.29.69) (interface: OPT1[opt1]) (real interface: ue0).

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S Online
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      @trussent:

                                      May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for opt1
                                      May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for wan
                                      May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface wan
                                      May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for wan
                                      May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for opt1
                                      May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface opt1

                                      It looks like the driver is seeing the actual ethernet link go down as though the cable has been unplugged or the upstream device has been rebooted. What are the WAN and OPT1 interfaces connected to?

                                      There is a bug in rc.renewwanip which may be relevant here:
                                      https://forum.pfsense.org/index.php?topic=76735.msg421118#msg421118

                                      Steve

                                      1 Reply Last reply Reply Quote 0
                                      • T Offline
                                        trussent
                                        last edited by

                                        Steve,

                                        Thanks for your quick response.  I was surprised that this thread was still open.  I just saw that there was a new release and thought I would upgrade and try it out.  Unfortunately, my problem with the USB nic drivers is apparently still present.

                                        Specifically, answering your questions, the hardware was not touched or unplugged or changed in any way at that time.

                                        The WAN is plugged into an ATT dsl 2WIRE model modem, which has its wireless features disabled; is plugged into a phone jack in the wall; and is powered by a ups.  It is set to provide a 172.23.x.x address in response to a dhcp request from the pfsense box.

                                        The OPT1 is plugged into a motorola surfboard cable modem, which is plugged into the cable coax in the wall; is also powered by a UPS; and delivers an external IP address in response to a dhcp request from the pfsense box.

                                        It is unlikely that both networks dropped connection to my house simultaneously, two days after upgrading from 1.2.3, when they have been stable for the 6 months I have not been using pfsense 2.1.

                                        I will check into the bug report this evening.

                                        -ram

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S Online
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          I agree, more likely that the USB driver is resetting the PHY for some reason.
                                          pfSense 1.2.3 was built on FreeBSD 7.X. pfSense 2.0.X on 8.1 and 2.1.X on 8.3, both FreeBSD 8 branch so the drivers are probably similar if not identical. You might try a 2.2 snapshot because they are built on FreeBSD 10 so will likely have better drivers, less bugs (but also may have new bugs!  ;))
                                          http://snapshots.pfsense.org/

                                          Steve

                                          1 Reply Last reply Reply Quote 0
                                          • K Offline
                                            kpa
                                            last edited by

                                            My experience with USB NICs on FreeBSD and pfSense is pretty much that USB NICs should be used only when no other option is available. They are not in the same league with PCI/PCIx/PCIe NICs in terms of reliability and performance.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.