Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    WAN dropped connection on 2.1

    Scheduled Pinned Locked Moved General pfSense Questions
    37 Posts 8 Posters 9.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T Offline
      trussent
      last edited by

      Dropped back to 1.2.3 and axe0,1 for wan and opt1.  No problems yet.

      J'accuse, the ue usb nic driver in 2.1 as the unreliable driver.

      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        If you're using USB NICs that's the first suspect in this sort of case. Is there no way you can use something else?
        Your logs indicate that the connection simply goes directly down rather than first experiencing delay or packet loss is that correct?
        You could try disabling gateway monitoring for the bad WAN connection and see if that makes any difference.

        Steve

        1 Reply Last reply Reply Quote 0
        • T Offline
          trussent
          last edited by

          I can't test this without usb nics easily with the equipment I currently have available.  Right now, I have Pfsense 2.1 and 1.2.3 installed on SD cards connected to an old EEEpc with unusable a SSD c drive.  Both 2.1 and 1.2.3 are configured with one onboard nic to the lan and two usb nics to the wan and opt1.  One of the usb nics is a dlink and the other is from belkin.  Both worked fine until 2.1.  Both are working fine now on a clean 1.2.3 install from last night.  While the hardware may be faulty in some odd way, having swapped connectors, computers, and operating system versions makes hardware an unlikely source of the problem.

          I noted that when firing up 2.1 both usb nics came in on the ue usb driver, and that in 1.2.3 both came up on the axe driver.  The onboard nic is using ale in 1.2.3 and 2.1.  My current hypothesis is that the ue driver incorporated in 2.1 is flaky.  I have been a UNIX sysadmin since SCO in the 80s; so, am somewhat comfortable under the hood, but do not know how pfsense/bsd selects the driver appropriate for the usb nic, nor how to force it to choose an older/different driver.  If there were a way to do that, then I could test my hypothesis.

          Specifically, regarding your question about the connections just dropping out, yes that is correct.  No warning, delays or odd behavior.  Further, using the interface status page to release and renew the interface always works to bring it back up; although, whatever is causing the connection to drop still occurs.

          I have tried using and disabling gateway monitoring.  It has no effect on the behavior in 2.1.  The log you see above is from a time when I had it on.  I am in the process of setting up gateways for 1.2.3 to see if it makes any difference with that OS version.

          Thanks Steve.

          1 Reply Last reply Reply Quote 0
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            They are still using the axe(4) driver it's just now wrapped in the ue nomenclature for some reason I've not understood.
            There may well have been a regression in the driver for your particular device. If you google freebsd axe regression there's no shortage of reports!
            You're not seeing either of the axe diagnostic outputs: axe0: watchdog timeout  or  axe0: no memory for rx list?

            Steve

            1 Reply Last reply Reply Quote 0
            • T Offline
              trussent
              last edited by

              Well s**t Steve, I never seen any axe diagnostics, but where should I find them, on the console standard out like gui admin login reports where axe diagnostics are not, or in some special log where I don't know to look?

              Glad to hear that axe is around in 2.1.  I think all I need is to properly assign it to the usb nic interface, but I don't know how to do that at this point.

              Until then though, 1.2.3 seems to be doing the job I need it to do.

              1 Reply Last reply Reply Quote 0
              • stephenw10S Offline
                stephenw10 Netgate Administrator
                last edited by

                Any output from the axe driver should appear in the system logs. Since you didn't post it above I assumed you didn't but worth mentioning in case.

                You could try the new 2.1.1 snapshots. A lot of bug fixes have gone into that, maybe one relevant to you.
                https://forum.pfsense.org/index.php/topic,71546.0.html

                Steve

                1 Reply Last reply Reply Quote 0
                • T Offline
                  trussent
                  last edited by

                  nope.  I just updated from 1.2.3 to  2.1.3-RELEASE (i386) and the usb nic driver problem is still present.

                  I'll mess with it for a few days, but then drop back to 1.2.3 if there is no resolution.

                  1 Reply Last reply Reply Quote 0
                  • T Offline
                    trussent
                    last edited by

                    While it is probably a coincidence, both my wan and opt1 links served by the ue usb driver dropped nearly simultaneously after almost exactly 3000 minutes of up time (+/- 5 minutes).  So, it could be a bad constant somewhere, or more likely, the "bad" event happened on the router and knocked both connections down in short order and it just happened to be on a unusual number boundary.

                    1 Reply Last reply Reply Quote 0
                    • T Offline
                      trussent
                      last edited by

                      Here is the system log, where you can see both connections dropping and not restarting

                      May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                      May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                      May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                      May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                      May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                      May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                      May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                      May 21 20:20:54 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                      May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                      May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 15
                      May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 15
                      May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 14
                      May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 14
                      May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                      May 21 20:20:54 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                      May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                      May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                      May 21 21:05:13 check_reload_status: rc.newwanip starting ue0
                      May 21 21:05:17 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                      May 21 21:05:17 php: rc.newwanip: rc.newwanip: on (IP address: 50.130.29.69) (interface: OPT1[opt1]) (real interface: ue0).
                      May 21 21:05:19 php: rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.130.28.1
                      May 21 21:05:19 php: rc.newwanip: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                      May 21 21:05:19 php: rc.newwanip: MONITOR: GW_OPT1 is down, removing from routing group combine
                      May 21 21:05:19 php: rc.newwanip: Gateways status could not be determined, considering all as up/active. (Group: combine)
                      May 21 21:05:23 check_reload_status: rc.newwanip starting ue1
                      May 21 21:05:24 php: rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
                      May 21 21:05:24 php: rc.newwanip: Creating rrd update script
                      May 21 21:05:26 php: rc.newwanip: rc.newwanip: Informational is starting ue1.
                      May 21 21:05:26 php: rc.newwanip: rc.newwanip: on (IP address: 172.23.23.53) (interface: WAN[wan]) (real interface: ue1).
                      May 21 21:05:26 php: rc.newwanip: ROUTING: setting default route to 172.23.23.23
                      May 21 21:05:26 php: rc.newwanip: Removing static route for monitor 76.29.224.1 and adding a new route through 172.23.23.23
                      May 21 21:05:26 php: rc.newwanip: Removing static route for monitor 8.8.8.8 and adding a new route through 50.130.28.1
                      May 21 21:05:26 php: rc.newwanip: pfSense package system has detected an ip change 0.0.0.0 -> 50.130.29.69 … Restarting packages.
                      May 21 21:05:26 check_reload_status: Starting packages
                      May 21 21:05:26 check_reload_status: Reloading filter
                      May 21 21:05:30 check_reload_status: updating dyndns GW_OPT1
                      May 21 21:05:30 check_reload_status: Restarting ipsec tunnels
                      May 21 21:05:30 check_reload_status: Restarting OpenVPN tunnels/interfaces
                      May 21 21:05:30 check_reload_status: Reloading filter
                      May 21 21:05:30 php: rc.start_packages: Restarting/Starting all packages.
                      May 21 21:05:32 php: rc.newwanip: Resyncing OpenVPN instances for interface WAN.
                      May 21 21:05:32 php: rc.newwanip: Creating rrd update script
                      May 21 21:05:35 php: rc.newwanip: pfSense package system has detected an ip change 0.0.0.0 -> 172.23.23.53 ... Restarting packages.

                      1 Reply Last reply Reply Quote 0
                      • T Offline
                        trussent
                        last edited by

                        And as usual, restarting the interfaces merely required punching the release and restart button on the interface form (http://192.168.3.1/status_interfaces.php) for WAN and OPT1.

                        1 Reply Last reply Reply Quote 0
                        • T Offline
                          trussent
                          last edited by

                          and here are the log entries starting at the fail at 20:20:14

                          May 21 18:55:37 check_reload_status: Reloading filter
                          May 21 20:20:14 check_reload_status: Linkup starting ue0
                          May 21 20:20:14 kernel: ue0: link state changed to DOWN
                          May 21 20:20:14 kernel: ue0: link state changed to UP
                          May 21 20:20:14 kernel: ue1: link state changed to DOWN
                          May 21 20:20:14 kernel: ue1: link state changed to UP
                          May 21 20:20:14 check_reload_status: Linkup starting ue0
                          May 21 20:20:14 check_reload_status: Linkup starting ue1
                          May 21 20:20:14 check_reload_status: Linkup starting ue1
                          May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for opt1
                          May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for wan
                          May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface wan
                          May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for wan
                          May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for opt1
                          May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface opt1
                          May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:19 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:23 php: rc.linkup: Clearing states to old gateway 172.23.23.23.
                          May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:23 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:24 check_reload_status: rc.newwanip starting ue0
                          May 21 20:20:24 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:24 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:24 php: rc.linkup: Clearing states to old gateway 50.130.28.1.
                          May 21 20:20:24 check_reload_status: rc.newwanip starting ue1
                          May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:25 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:26 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: on (IP address: ) (interface: OPT1[opt1]) (real interface: ue0).
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: Failed to update opt1 IP, restarting…
                          May 21 20:20:27 check_reload_status: Configuring interface opt1
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: Informational is starting ue1.
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:27 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: ue1).
                          May 21 20:20:27 php: rc.newwanip: rc.newwanip: Failed to update wan IP, restarting…
                          May 21 20:20:28 check_reload_status: Configuring interface wan
                          May 21 20:20:28 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:28 check_reload_status: updating dyndns wan
                          May 21 20:20:28 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:28 php: rc.linkup: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid ale0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.6 Copyright 2004-2014 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 17 leases to leases file. Can't backup lease database /var/db/dhcpd.leases to /var/db/dhcpd.leases~: File exists Listening on BPF/ale0/00:23:54:0b:73:24/192.168.3.0/24 Sending on BPF/ale0/00:23:54:0b:73:24/192.168.3.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get this soft
                          May 21 20:20:28 check_reload_status: updating dyndns opt1
                          May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:29 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:30 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:30 php: rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_opt1.conf ue0 > /tmp/ue0_output 2> /tmp/ue0_error_output' returned exit code '1', the output was ''
                          May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:31 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:32 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:32 php: rc.interfaces_wan_configure: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf ue1 > /tmp/ue1_output 2> /tmp/ue1_error_output' returned exit code '1', the output was ''
                          May 21 20:20:32 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:32 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:33 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:34 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:35 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:36 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:37 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:38 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:39 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:40 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:41 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:42 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:43 check_reload_status: updating dyndns GW_WAN
                          May 21 20:20:43 check_reload_status: Restarting ipsec tunnels
                          May 21 20:20:43 check_reload_status: Restarting OpenVPN tunnels/interfaces
                          May 21 20:20:43 check_reload_status: Reloading filter
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:43 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:43 check_reload_status: updating dyndns GW_OPT1
                          May 21 20:20:43 check_reload_status: Restarting ipsec tunnels
                          May 21 20:20:43 check_reload_status: Restarting OpenVPN tunnels/interfaces
                          May 21 20:20:43 check_reload_status: Reloading filter
                          May 21 20:20:44 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:44 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:45 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:46 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:47 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:48 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:49 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:50 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:50 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:50 php: rc.dyndns.update: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_WAN is down, removing from routing group combine
                          May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_OPT1 is down, removing from routing group combine
                          May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_WAN is down, removing from routing group combine
                          May 21 20:20:50 php: rc.dyndns.update: MONITOR: GW_OPT1 is down, removing from routing group combine
                          May 21 20:20:50 php: rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: combine)
                          May 21 20:20:50 php: rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: combine)
                          May 21 20:20:50 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:50 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                          May 21 20:20:50 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                          May 21 20:20:50 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                          May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:51 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:52 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:53 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                          May 21 20:20:53 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 172.23.23.23
                          May 21 20:20:54 kernel: arpresolve: can't allocate llinfo for 50.130.28.1
                          May 21 20:20:54 php: rc.filter_configure_sync: The command '/sbin/route change -inet default dynamic' returned exit code '68', the output was 'route: bad address: dynamic'
                          May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_WAN is down, removing from routing group combine
                          May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 15
                          May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 15
                          May 21 20:20:54 lighttpd[34060]: (network_writev.c.107) writev failed: Operation not permitted 14
                          May 21 20:20:54 lighttpd[34060]: (connections.c.619) connection closed: write failed on fd 14
                          May 21 20:20:54 php: rc.filter_configure_sync: MONITOR: GW_OPT1 is down, removing from routing group combine
                          May 21 20:20:54 php: rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: combine)
                          May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (wan).
                          May 21 20:20:54 php: rc.filter_configure_sync: Could not find IPv4 gateway for interface (opt1).
                          May 21 21:05:13 check_reload_status: rc.newwanip starting ue0
                          May 21 21:05:17 php: rc.newwanip: rc.newwanip: Informational is starting ue0.
                          May 21 21:05:17 php: rc.newwanip: rc.newwanip: on (IP address: 50.130.29.69) (interface: OPT1[opt1]) (real interface: ue0).

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S Offline
                            stephenw10 Netgate Administrator
                            last edited by

                            @trussent:

                            May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for opt1
                            May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for wan
                            May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface wan
                            May 21 20:20:18 php: rc.linkup: DEVD Ethernet detached event for wan
                            May 21 20:20:18 php: rc.linkup: DEVD Ethernet attached event for opt1
                            May 21 20:20:18 php: rc.linkup: HOTPLUG: Configuring interface opt1

                            It looks like the driver is seeing the actual ethernet link go down as though the cable has been unplugged or the upstream device has been rebooted. What are the WAN and OPT1 interfaces connected to?

                            There is a bug in rc.renewwanip which may be relevant here:
                            https://forum.pfsense.org/index.php?topic=76735.msg421118#msg421118

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • T Offline
                              trussent
                              last edited by

                              Steve,

                              Thanks for your quick response.  I was surprised that this thread was still open.  I just saw that there was a new release and thought I would upgrade and try it out.  Unfortunately, my problem with the USB nic drivers is apparently still present.

                              Specifically, answering your questions, the hardware was not touched or unplugged or changed in any way at that time.

                              The WAN is plugged into an ATT dsl 2WIRE model modem, which has its wireless features disabled; is plugged into a phone jack in the wall; and is powered by a ups.  It is set to provide a 172.23.x.x address in response to a dhcp request from the pfsense box.

                              The OPT1 is plugged into a motorola surfboard cable modem, which is plugged into the cable coax in the wall; is also powered by a UPS; and delivers an external IP address in response to a dhcp request from the pfsense box.

                              It is unlikely that both networks dropped connection to my house simultaneously, two days after upgrading from 1.2.3, when they have been stable for the 6 months I have not been using pfsense 2.1.

                              I will check into the bug report this evening.

                              -ram

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S Offline
                                stephenw10 Netgate Administrator
                                last edited by

                                I agree, more likely that the USB driver is resetting the PHY for some reason.
                                pfSense 1.2.3 was built on FreeBSD 7.X. pfSense 2.0.X on 8.1 and 2.1.X on 8.3, both FreeBSD 8 branch so the drivers are probably similar if not identical. You might try a 2.2 snapshot because they are built on FreeBSD 10 so will likely have better drivers, less bugs (but also may have new bugs!  ;))
                                http://snapshots.pfsense.org/

                                Steve

                                1 Reply Last reply Reply Quote 0
                                • K Offline
                                  kpa
                                  last edited by

                                  My experience with USB NICs on FreeBSD and pfSense is pretty much that USB NICs should be used only when no other option is available. They are not in the same league with PCI/PCIx/PCIe NICs in terms of reliability and performance.

                                  1 Reply Last reply Reply Quote 0
                                  • T Offline
                                    trussent
                                    last edited by

                                    Kpa,

                                    Well thanks for your thoughts, but I can't just shove a couple cards into the eeepc's I've been using for pfsense over the past several years.  Further, I have been using USB nics in them since 2009, and have had no problems with them.  In fact, my 701 surf, which is running 1.2.3, has been up for over a year without a single problem, and its USB NICs have been in continuous use since 2009.  My experience says that USB NICs perform well, are robust, and reliable.

                                    I cannot say the same for the ue drivers bundled in pfsense 2.1.X  They have been a constant headache, since I first upgraded from 2.0.1 rel, and still are.  I am hoping Steve's note that 2.2 is based on a different build will bring some relief.

                                    -ram

                                    1 Reply Last reply Reply Quote 0
                                    • M Offline
                                      matrix200
                                      last edited by

                                      I want to concur with Trussent here.
                                      Only in my case problems started after upgrading to 2.1.3
                                      Before that I was running 2.1 in a VM and a usb adapter in pass through mode and they performed great.
                                      Sure there were multiple interface down and up once in a while but they seemed never to affect the actual connection.
                                      Since I installed 2.1.3 it has been nothing but pain and frustration.
                                      First the same ue0 down and up events are now actually causing the wan connection to reset.
                                      Secondly the pppoe dialer has a bug so it is unable to reconnect so I wrote a php script to invoke configure_interface function whenever this happens (to be more precise just put it at cronjob running each minute.
                                      Third I just attempted something that may alleviate this problem.
                                      I simply commented out interface_bring_down function from the the /etc/rc.linkup.
                                      This of course breaks unplug replug cable functionality, but hopefully it will stop reacting to the usb nic driver behaving badly and showing as if network cable got disconnected.
                                      Can't comment yet on the results as I haven't had ue0 event just yet so it remains to be seen how this works.

                                      Current network "hardware" :
                                      Running 2.2RC in Virtualbox 4.2.16.

                                      Retired:
                                      ALIX2C2 , 4 gigabyte disk cf card running 2.0 (official release).

                                      1 Reply Last reply Reply Quote 0
                                      • N Offline
                                        nug
                                        last edited by

                                        How did you go with this Matrix?

                                        I've just jumped into the world of pfSense and started with the latest version 2.1.3.  I'm using a Belkin F4U047bt network card for my WAN connection to my Netgear DG834G in modem mode.  I know the modem and cable are both fine because they ran fine for years under ClearOS.

                                        The USB NIC is new however and it’s plugged into a EEEbox (same as Trussent).

                                        I’m having exactly the same symptoms but I’m not really as keen to go back so many versions to get it to work consistently.

                                        1 Reply Last reply Reply Quote 0
                                        • M Offline
                                          matrix200
                                          last edited by

                                          Unfortunately I didn't
                                          Since I run pfsense in a VM I did the following:
                                          1. I installed an old intel 100 pro nic into the host machine.
                                          2. Using Virtualbox I briged it with a virtual adapter (intel pro 1000).
                                          3. Hooked up the modem to this nic.
                                          4. In order to make sure the nic is exclusively used only by the VM I removed any ip addresses that might have been assigned to it.

                                          No Internet disconnects since

                                          Current network "hardware" :
                                          Running 2.2RC in Virtualbox 4.2.16.

                                          Retired:
                                          ALIX2C2 , 4 gigabyte disk cf card running 2.0 (official release).

                                          1 Reply Last reply Reply Quote 0
                                          • T Offline
                                            trussent
                                            last edited by

                                            Steve,

                                            I have been messing with 2.2 alpha builds for a while now, and they are just too unstable for me to be able to know if the USB drivers are better.  (By unstable I mean that they throw exceptions and drop into the db when reloading the interfaces after a reboot, or a reconfiguration of the network architecture.)  Perhaps as 2.2 matures, I can do some system tests that can add information about this problem with respect to 2.2.

                                            As for 2.1.3, I have swapped the wan and opt1 and done a few other experiments.  None have changed the fundamental problem with the ue usb drivers reporting a HOTPLUG event within a short time (5 minutes to 2 days).  The only new symptom is that prior to dropping out the gateway monitor is reporting 66% packet loss for a while (at least a few minutes), but there is nothing about this in the logs.  I can't really say if it is new or I just didn't notice it before.

                                            Matrix200,

                                            Did your code change:

                                            I simply commented out interface_bring_down function from the the /etc/rc.linkup.

                                            work?  I haven't tried that yet.

                                            -ram

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.