Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Since a few day's losing the wan (dhcp) ip address

    Scheduled Pinned Locked Moved 2.1 Snapshot Feedback and Problems - RETIRED
    52 Posts 11 Posters 24.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      TravisD
      last edited by

      Update: Stable overnite running on 2.0.1. Will continue to monitor.

      1 Reply Last reply Reply Quote 0
      • T
        tworush
        last edited by

        woutervb
        I have the same problem my work around is creating a script which pings google and if packet lost my network card restarts

        copy and save this file in the root home directory called wan_check.sh

        #!/usr/bin/env bash
        #/usr/local/bin/network_check
        
        # Script to monitor and restart network card
        
        maxPloss=10 #Maximum percent packet loss before a restart
        
        restart_networking() {
                # Add any commands need to get network back up and running
                sh /root/wan.sh
        
        }
        
        # First make sure we can resolve google, otherwise 'ping -w' would hang
        if ! $(host -W5 www.google.com > /dev/null 2>&1); then
                #Make a note in syslog
                logger "network_check: Network connection is down, restarting network ..."
                restart_networking
                exit
        fi
        
        # Initialize to a value that would force a restart
        # (just in case ping gives an error and ploss doesn't get set)
        ploss=101
        # now ping google for 10 seconds and count packet loss
        ploss=$(ping -q -w10 www.google.com | grep -o "[0-9]*%" | tr -d %) > /dev/null 2>&1
        
        if [ "$ploss" -gt "$maxPloss" ]; then
                logger "Packet loss ($ploss%) exceeded $maxPloss, restarting network ..."
                restart_networking
        fi
        
        

        This code will restart Wan if ping loses packets.  change de1 to your wan interface
        name this file wan.sh

        ifconfig de1 down
        sleep 5
        ifconfig de1 up
        dhclient de1
        
        

        change the attriputes (Chmod) to 777 on both files
        and setup cron job to check the wan interface every 30 min
        30  0  *  *  *  root  /root/wan_check.sh

        Note I had to make the folowing change because running under Hyper-v.  Change Speed and duplex to "100 base tx full duplex" this located under interface–> WAN

        1 Reply Last reply Reply Quote 0
        • A
          abeowitz
          last edited by

          What do you all have powerd set to?

          Mine seems to fail when powerd is checked.  But that could be just coincidence…  (Looking for patterns.)

          1 Reply Last reply Reply Quote 0
          • Z
            Zorac
            last edited by

            mine is unchecked (off)

            1 Reply Last reply Reply Quote 0
            • A
              abeowitz
              last edited by

              So, I updated from 2012-11-29 to today and the problem returned.

              However, reconfiguring from scratch, I made the WAN & OPT IP's static, and it seems stable.  (Though static means I have to NAT again through the modems.)

              1 Reply Last reply Reply Quote 0
              • T
                TravisD
                last edited by

                Do you guys have OpenVPN configured (but perhaps not used) by any chance? I'm starting to think that this may have started for me about the time I started messing with OpenVPN (I got interested when the IOS OpenVPN App came out). I came across a post here somewhere that seemed to correlate one of the various error messages with OpenVPN.

                I just whacked the OpenVPN config on my system to see if it stabilizes at all. We'll see what happens.

                1 Reply Last reply Reply Quote 0
                • Z
                  Zorac
                  last edited by

                  I do have openvpn configured.  I use it for my tablet when I'm away from home.

                  1 Reply Last reply Reply Quote 0
                  • T
                    TravisD
                    last edited by

                    I just did a 100% clean install of 2.1.  Didn't bring in any old config, just configured a handfull of NATs that I need. No VPN, etc. We'll see how this one goes…

                    1 Reply Last reply Reply Quote 0
                    • T
                      TravisD
                      last edited by

                      At least I seem to be moving the ball around…  (Oh, my apologies for spreading this around to multiple areas of the support forum. Since I'm now on 2.1, I'll try and put things here and redirect the others to here. Bad forum behavior, I know - sorry!)

                      Fresh, clean, no-legacy install of 2.1 -- still fails, but failed differently this time.

                      
                      Feb 28 23:26:00 gw check_reload_status: Linkup starting ue0
                      Feb 28 23:26:00 gw kernel: ue0: link state changed to DOWN
                      Feb 28 23:26:00 gw kernel: ue0: link state changed to UP
                      Feb 28 23:26:01 gw check_reload_status: Linkup starting ue0
                      Feb 28 23:26:03 gw php: : DEVD Ethernet detached event for wan
                      Feb 28 23:26:03 gw dhclient[52833]: connection closed
                      Feb 28 23:26:03 gw dhclient[52833]: exiting.
                      Feb 28 23:26:03 gw php: : DEVD Ethernet attached event for wan
                      Feb 28 23:26:03 gw php: : HOTPLUG: Configuring interface wan
                      Feb 28 23:26:03 gw dhclient: PREINIT
                      Feb 28 23:26:03 gw kernel: arpresolve: can't allocate llinfo for 68.48.154.1
                      Feb 28 23:26:03 gw kernel: arpresolve: can't allocate llinfo for 68.48.154.1
                      Feb 28 23:26:03 gw dhclient: Starting delete_old_states()
                      Feb 28 23:26:03 gw dhclient: Comparing IPs: Old: New: 
                      Feb 28 23:26:03 gw dhclient: Comparing Routers: Old: New: 
                      Feb 28 23:26:03 gw dhclient[68325]: DHCPREQUEST on ue0 to 255.255.255.255 port 67
                      Feb 28 23:26:03 gw dhclient[68325]: DHCPACK from 73.129.106.1
                      Feb 28 23:26:03 gw dhclient: REBOOT
                      Feb 28 23:26:03 gw dhclient: Starting delete_old_states()
                      Feb 28 23:26:03 gw dhclient: Comparing IPs: Old: New: 68.48.155.52
                      Feb 28 23:26:03 gw dhclient: Comparing Routers: Old: New: 68.48.154.1
                      Feb 28 23:26:03 gw dhclient: Removing states through old gateway '' (new gateway '68.48.154.1')
                      Feb 28 23:26:03 gw dhclient: Starting add_new_address()
                      Feb 28 23:26:03 gw dhclient: ifconfig ue0 inet 68.48.155.52 netmask 255.255.254.0 broadcast 255.255.255.255 
                      Feb 28 23:26:03 gw dhclient: New IP Address (ue0): 68.48.155.52
                      Feb 28 23:26:03 gw dhclient: New Subnet Mask (ue0): 255.255.254.0
                      Feb 28 23:26:03 gw dhclient: New Broadcast Address (ue0): 255.255.255.255
                      Feb 28 23:26:03 gw dhclient: New Routers (ue0): 68.48.154.1
                      Feb 28 23:26:03 gw dhclient: Adding new routes to interface: ue0
                      Feb 28 23:26:03 gw dhclient: /sbin/route add default 68.48.154.1
                      Feb 28 23:26:03 gw dhclient: Creating resolv.conf
                      Feb 28 23:26:03 gw check_reload_status: rc.newwanip starting ue0
                      Feb 28 23:26:03 gw dhclient[68325]: bound to 68.48.155.52 -- renewal in 163608 seconds.
                      Feb 28 23:26:04 gw php: : Clearing states to old gateway 68.48.154.1.
                      Feb 28 23:26:06 gw php: : rc.newwanip: Informational is starting ue0.
                      Feb 28 23:26:06 gw php: : rc.newwanip: on (IP address: ) (interface: wan) (real interface: ue0).
                      Feb 28 23:26:06 gw php: : rc.newwanip: Failed to update wan IP, restarting…
                      Feb 28 23:26:06 gw check_reload_status: Configuring interface wan
                      Feb 28 23:26:08 gw dhclient[78874]: dhclient already running, pid: 76098.
                      Feb 28 23:26:08 gw dhclient[78874]: exiting.
                      Feb 28 23:26:08 gw php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf ue0 > /tmp/ue0_output 2> /tmp/ue0_error_output' returned exit code '1', the output was '' 
                      Feb 28 23:26:22 gw check_reload_status: Updating all dyndns
                      Feb 28 23:26:22 gw check_reload_status: Restarting ipsec tunnels
                      Feb 28 23:26:22 gw check_reload_status: Restarting OpenVPN tunnels/interfaces
                      Feb 28 23:26:22 gw check_reload_status: Reloading filter
                      Feb 28 23:26:25 gw php: : DynDns: updatedns() starting
                      Feb 28 23:26:25 gw php: : DynDNS (travisd.dyndns.org): running get_failover_interface for wan. found ue0
                      Feb 28 23:26:25 gw php: : DynDNS (travisd.dyndns.org) There was an error trying to determine the public IP for interface - wan(ue0). Probably interface is not a WAN interface.
                      Feb 28 23:26:25 gw php: : Could not find IPv4 gateway for interface (wan).
                      Feb 28 23:26:25 gw php: : Could not find IPv4 gateway for interface (wan).
                      Feb 28 23:26:25 gw php: : Could not find IPv4 gateway for interface (wan).
                      Feb 28 23:26:25 gw php: : Could not find IPv4 gateway for interface (wan).
                      Feb 28 23:26:25 gw php: : Could not find IPv4 gateway for interface (wan).
                      Feb 28 23:26:26 gw php: : DynDns: updatedns() starting
                      Feb 28 23:26:26 gw php: : DynDNS (travisd.dyndns.org): running get_failover_interface for wan. found ue0
                      Feb 28 23:26:26 gw php: : DynDNS (travisd.dyndns.org) There was an error trying to determine the public IP for interface - wan(ue0). Probably interface is not a WAN interface.
                      
                      

                      Files as they looked before release/renew:

                      
                      [2.1-BETA1][root@gw.tubas.net]/root(7): cat /tmp/ue0*
                      fe80::201:5cff:fe22:1281
                      dhclient already running, pid: 76098.
                      exiting.
                      
                      [2.1-BETA1][root@gw.tubas.net]/var/etc(13): cat dhclient_wan.conf 
                      interface "ue0" {
                      timeout 60;
                      retry 15;
                      select-timeout 0;
                      initial-interval 1;
                      
                       script "/sbin/dhclient-script";
                      }
                      
                      

                      No significant DHCP traffic on tcpdump at this time on the WAN interface (filtering on port 67). Removing the filter, I saw normal looking arp traffic.

                      tcpdump while doing release/renew (at least that works again to restore service!) produced minimal output, but it looks reasonable to me:

                      
                      23:32:43.335315 IP (tos 0x10, ttl 16, id 0, offset 0, flags [none], proto UDP (17), length 328)
                          c-68-48-155-52.hsd1.md.comcast.net.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:50:b6:59:70:f5 (oui Unknown), length 300, xid 0xba01a458, Flags [none] (0x0000)
                         Client-Ethernet-Address 00:50:b6:59:70:f5 (oui Unknown) [|bootp]
                      23:32:43.354619 IP (tos 0x0, ttl 64, id 60020, offset 0, flags [none], proto UDP (17), length 338)
                          73.129.106.1.bootps > c-68-48-155-52.hsd1.md.comcast.net.bootpc: BOOTP/DHCP, Reply, length 310, hops 1, xid 0xba01a458, Flags [none] (0x0000)
                         Your-IP c-68-48-155-52.hsd1.md.comcast.net
                         Gateway-IP 73.129.106.1
                         Client-Ethernet-Address 00:50:b6:59:70:f5 (oui Unknown) [|bootp]
                      
                      

                      And again the post-renew output from the various dhcp files:

                      
                      [2.1-BETA1][root@gw.tubas.net]/var/etc(18): cat /tmp/ue0*
                      fe80::201:5cff:fe22:1281
                      dhclient: PREINIT
                      dhclient: Starting delete_old_states()
                      OLD_IP: not found
                      dhclient: Comparing IPs: Old: New: 
                      dhclient: Comparing Routers: Old: New: 
                      DHCPREQUEST on ue0 to 255.255.255.255 port 67
                      DHCPACK from 73.129.106.1
                      bound to 68.48.155.52 -- renewal in 163409 seconds.
                      68.48.154.1
                      [2.1-BETA1][root@gw.tubas.net]/var/etc(19): cat dhc
                      dhclient_wan.conf dhcp6c_wan_script.sh* 
                      [2.1-BETA1][root@gw.tubas.net]/var/etc(19): cat dhclient_wan.conf 
                      interface "ue0" {
                      timeout 60;
                      retry 15;
                      select-timeout 0;
                      initial-interval 1;
                      
                       script "/sbin/dhclient-script";
                      }
                      
                      

                      So, something changed by doing a fresh install of the 2.1 snapshot, but things still seme broken.

                      1 Reply Last reply Reply Quote 0
                      • A
                        abeowitz
                        last edited by

                        Even with static IPs on WAN and OPT1, I still get lots of these in my dmesg logs:

                        ue1: link state changed to DOWN
                        ue1: link state changed to UP
                        ue0: link state changed to DOWN
                        ue0: link state changed to UP
                        pfsync0: promiscuous mode disabled
                        ue0: link state changed to DOWN
                        ue0: link state changed to UP
                        ue0: link state changed to DOWN
                        ue0: link state changed to UP
                        pfsync0: promiscuous mode enabled

                        I'm getting small % packet loss, which seems to correlate to the link states.  But there's no interruption in web surfing.

                        I don't have OpenVPN enabled/configured.  I do have the export keys module installed, but haven't done anything with that yet.

                        1 Reply Last reply Reply Quote 0
                        • T
                          TravisD
                          last edited by

                          So I had configured a couple of dynamic DNS updaters, but decided to pull them out for one reason or another.

                          Upon deleting them, this appears in the system.log:

                          
                          Mar  1 01:05:29 gw php: /services_dyndns.php: The command '/bin/rm /conf/dyndns_*.cache' returned exit code '1', the output was 'rm: /conf/dyndns_wandyndns'travisd.dyndns.org'0.cache: Read-only file system rm: /conf/dyndns_wanopendns'travisd.dyndns.org'1.cache: Read-only file system' 
                          
                          

                          …and the WAN link dies again (loses it's IP address)

                          Manually trying to remove these replicates the issue:

                          
                          [2.1-BETA1][root@gw.tubas.net]/conf(12): rm -f dyndns_*
                          rm: dyndns_wandyndns'travisd.dyndns.org'0.cache: Read-only file system
                          rm: dyndns_wanopendns'travisd.dyndns.org'1.cache: Read-only file system
                          
                          

                          Edit: More logs, this is what spits out now when it dies (and before manual recovery via release/renew)

                          
                          Mar  1 01:34:58 gw check_reload_status: Linkup starting ue0
                          Mar  1 01:34:58 gw kernel: ue0: link state changed to DOWN
                          Mar  1 01:34:58 gw kernel: ue0: link state changed to UP
                          Mar  1 01:34:58 gw check_reload_status: Linkup starting ue0
                          Mar  1 01:34:58 gw kernel: ue0: link state changed to DOWN
                          Mar  1 01:34:58 gw kernel: ue0: link state changed to UP
                          Mar  1 01:34:58 gw check_reload_status: Linkup starting ue0
                          Mar  1 01:34:58 gw check_reload_status: Linkup starting ue0
                          Mar  1 01:35:01 gw php: : DEVD Ethernet detached event for wan
                          Mar  1 01:35:01 gw php: : DEVD Ethernet attached event for wan
                          Mar  1 01:35:01 gw dhclient[23035]: connection closed
                          Mar  1 01:35:01 gw dhclient[23035]: exiting.
                          Mar  1 01:35:01 gw php: : DEVD Ethernet detached event for wan
                          Mar  1 01:35:01 gw php: : HOTPLUG: Configuring interface wan
                          Mar  1 01:35:01 gw php: : DEVD Ethernet attached event for wan
                          Mar  1 01:35:01 gw php: : HOTPLUG: Configuring interface wan
                          Mar  1 01:35:01 gw dhclient: PREINIT
                          Mar  1 01:35:01 gw dhclient[44035]: dhclient already running, pid: 43420.
                          Mar  1 01:35:01 gw dhclient[44035]: exiting.
                          Mar  1 01:35:01 gw php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf ue0 > /tmp/ue0_output 2> /tmp/ue0_error_output' returned exit code '1', the output was '' 
                          Mar  1 01:35:01 gw dhclient: Starting delete_old_states()
                          Mar  1 01:35:01 gw dhclient: Comparing IPs: Old:  New: 
                          Mar  1 01:35:01 gw dhclient: Comparing Routers: Old:  New: 
                          Mar  1 01:35:01 gw dhclient[43420]: DHCPREQUEST on ue0 to 255.255.255.255 port 67
                          Mar  1 01:35:01 gw dhclient[43420]: DHCPACK from 73.129.106.1
                          Mar  1 01:35:01 gw dhclient: REBOOT
                          Mar  1 01:35:01 gw dhclient: Starting delete_old_states()
                          Mar  1 01:35:01 gw dhclient: Comparing IPs: Old:  New: 68.48.155.52
                          Mar  1 01:35:01 gw dhclient: Comparing Routers: Old:  New: 68.48.154.1
                          Mar  1 01:35:01 gw dhclient: Removing states through old gateway '' (new gateway '68.48.154.1')
                          Mar  1 01:35:01 gw dhclient: Starting add_new_address()
                          Mar  1 01:35:01 gw dhclient: ifconfig ue0 inet 68.48.155.52 netmask 255.255.254.0 broadcast 255.255.255.255 
                          Mar  1 01:35:01 gw dhclient: New IP Address (ue0): 68.48.155.52
                          Mar  1 01:35:01 gw dhclient: New Subnet Mask (ue0): 255.255.254.0
                          Mar  1 01:35:01 gw dhclient: New Broadcast Address (ue0): 255.255.255.255
                          Mar  1 01:35:01 gw dhclient: New Routers (ue0): 68.48.154.1
                          Mar  1 01:35:01 gw dhclient: Adding new routes to interface: ue0
                          Mar  1 01:35:01 gw dhclient: /sbin/route add default 68.48.154.1
                          Mar  1 01:35:01 gw dhclient: Creating resolv.conf
                          Mar  1 01:35:01 gw check_reload_status: rc.newwanip starting ue0
                          Mar  1 01:35:01 gw dhclient[43420]: bound to 68.48.155.52 -- renewal in 159739 seconds.
                          Mar  1 01:35:02 gw php: : Clearing states to old gateway 68.48.154.1.
                          Mar  1 01:35:04 gw php: : rc.newwanip: Informational is starting ue0.
                          Mar  1 01:35:04 gw php: : rc.newwanip: on (IP address: ) (interface: wan) (real interface: ue0).
                          Mar  1 01:35:04 gw php: : rc.newwanip: Failed to update wan IP, restarting...
                          Mar  1 01:35:04 gw check_reload_status: Configuring interface wan
                          Mar  1 01:35:06 gw dhclient[54110]: dhclient already running, pid: 51797.
                          Mar  1 01:35:06 gw dhclient[54110]: exiting.
                          Mar  1 01:35:06 gw php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf ue0 > /tmp/ue0_output 2> /tmp/ue0_error_output' returned exit code '1', the output was '' 
                          Mar  1 01:35:22 gw check_reload_status: Updating all dyndns
                          Mar  1 01:35:22 gw check_reload_status: Restarting ipsec tunnels
                          Mar  1 01:35:22 gw check_reload_status: Restarting OpenVPN tunnels/interfaces
                          Mar  1 01:35:22 gw check_reload_status: Reloading filter
                          Mar  1 01:35:25 gw php: : Could not find IPv4 gateway for interface (wan).
                          Mar  1 01:35:25 gw php: : Could not find IPv4 gateway for interface (wan).
                          Mar  1 01:35:25 gw php: : Could not find IPv4 gateway for interface (wan).
                          Mar  1 01:35:25 gw php: : Could not find IPv4 gateway for interface (wan).
                          Mar  1 01:35:25 gw php: : Could not find IPv4 gateway for interface (wan).
                          
                          

                          Edit 2: From the next time it died, here's the dhclient output:

                          
                          [2.1-BETA1][root@gw.tubas.net]/tmp(62): cat ue*
                          dhclient already running, pid: 69584.
                          exiting.
                          
                          

                          And yes, it is indeed running:

                          _dhcp  69584  0.0  0.1  3388  1464  ??  SNs   1:52AM   0:00.01 dhclient: ue0 (dhclient)

                          Edit 3: The next time it died, I tried running dhclient manually:

                          
                          [2.1-BETA1][root@gw.tubas.net]/sbin(74):  /sbin/dhclient -c /var/etc/dhclient_wan.conf ue0
                          dhclient already running, pid: 54736.
                          exiting.
                          
                          [2.1-BETA1][root@gw.tubas.net]/sbin(75): ps auxw | grep dhcl
                          root   46928  0.0  0.1  3388  1332  ??  INs   1:58AM   0:00.00 dhclient: ue0 [priv] (dhclient)
                          _dhcp  54736  0.0  0.1  3388  1464  ??  SNs   1:58AM   0:00.06 dhclient: ue0 (dhclient)
                          root   94481  0.0  0.1  3536  1248   1  S+    2:00AM   0:00.00 grep dhcl
                          
                          [2.1-BETA1][root@gw.tubas.net]/sbin(76): kill -KILL 54736 46928
                          
                          [2.1-BETA1][root@gw.tubas.net]/sbin(77): /sbin/dhclient -c /var/etc/dhclient_wan.conf ue0
                          dhclient: PREINIT
                          ifconfig: ioctl (SIOCAIFADDR): File exists
                          dhclient: Starting delete_old_states()
                          OLD_IP: not found
                          dhclient: Comparing IPs: Old:  New: 
                          dhclient: Comparing Routers: Old:  New: 
                          DHCPREQUEST on ue0 to 255.255.255.255 port 67
                          DHCPACK from 73.129.106.1
                          bound to 68.48.155.52 -- renewal in 158952 seconds.
                          [2.1-BETA1][root@gw.tubas.net]/sbin(78): 
                          
                          

                          …and this brought it back up.

                          1 Reply Last reply Reply Quote 0
                          • W
                            wallabybob
                            last edited by

                            @TravisD:

                            Since I'm now on 2.1,

                            But what particular build of 2.1?

                            @TravisD:

                            
                            Feb 28 23:26:00 gw check_reload_status: Linkup starting ue0
                            Feb 28 23:26:00 gw kernel: ue0: link state changed to DOWN
                            Feb 28 23:26:00 gw kernel: ue0: link state changed to UP
                            Feb 28 23:26:01 gw check_reload_status: Linkup starting ue0
                            Feb 28 23:26:03 gw php: : DEVD Ethernet detached event for wan
                            Feb 28 23:26:03 gw dhclient[52833]: connection closed
                            Feb 28 23:26:03 gw dhclient[52833]: exiting.
                            Feb 28 23:26:03 gw php: : DEVD Ethernet attached event for wan
                            Feb 28 23:26:03 gw php: : HOTPLUG: Configuring interface wan
                            Feb 28 23:26:03 gw dhclient: PREINIT
                            Feb 28 23:26:03 gw kernel: arpresolve: can't allocate llinfo for 68.48.154.1
                            Feb 28 23:26:03 gw kernel: arpresolve: can't allocate llinfo for 68.48.154.1
                            Feb 28 23:26:03 gw dhclient: Starting delete_old_states()
                            Feb 28 23:26:03 gw dhclient: Comparing IPs: Old: New: 
                            Feb 28 23:26:03 gw dhclient: Comparing Routers: Old: New: 
                            Feb 28 23:26:03 gw dhclient[68325]: DHCPREQUEST on ue0 to 255.255.255.255 port 67
                            Feb 28 23:26:03 gw dhclient[68325]: DHCPACK from 73.129.106.1
                            Feb 28 23:26:03 gw dhclient: REBOOT
                            Feb 28 23:26:03 gw dhclient: Starting delete_old_states()
                            Feb 28 23:26:03 gw dhclient: Comparing IPs: Old: New: 68.48.155.52
                            Feb 28 23:26:03 gw dhclient: Comparing Routers: Old: New: 68.48.154.1
                            Feb 28 23:26:03 gw dhclient: Removing states through old gateway '' (new gateway '68.48.154.1')
                            Feb 28 23:26:03 gw dhclient: Starting add_new_address()
                            Feb 28 23:26:03 gw dhclient: ifconfig ue0 inet 68.48.155.52 netmask 255.255.254.0 broadcast 255.255.255.255 
                            Feb 28 23:26:03 gw dhclient: New IP Address (ue0): 68.48.155.52
                            Feb 28 23:26:03 gw dhclient: New Subnet Mask (ue0): 255.255.254.0
                            Feb 28 23:26:03 gw dhclient: New Broadcast Address (ue0): 255.255.255.255
                            Feb 28 23:26:03 gw dhclient: New Routers (ue0): 68.48.154.1
                            Feb 28 23:26:03 gw dhclient: Adding new routes to interface: ue0
                            Feb 28 23:26:03 gw dhclient: /sbin/route add default 68.48.154.1
                            Feb 28 23:26:03 gw dhclient: Creating resolv.conf
                            Feb 28 23:26:03 gw check_reload_status: rc.newwanip starting ue0
                            Feb 28 23:26:03 gw dhclient[68325]: bound to 68.48.155.52 -- renewal in 163608 seconds.
                            Feb 28 23:26:04 gw php: : Clearing states to old gateway 68.48.154.1.
                            [b]Feb 28 23:26:06 gw php: : rc.newwanip: Informational is starting ue0.[/b]
                            Feb 28 23:26:06 gw php: : rc.newwanip: on (IP address: ) (interface: wan) (real interface: ue0).
                            Feb 28 23:26:06 gw php: : rc.newwanip: Failed to update wan IP, restarting…
                            
                            

                            The log extract you posted shows multiple dhclient copies running. This seems like a bad idea - they could easily interfere with each other in unpredictable ways. I think I have seen recent posts addressing this issue. The multiple reports: Feb 28 23:26:06 gw php: : rc.newwanip: Informational is starting ue0. in the original log you posted seem strange in that ue0 shouldn't need to be started (not sure exactly what that started means) immediately after the interface IP address has been set and routes added etc. PERHAPS there is a synchronisation issue between multiple processes and it generally works OK in some systems (slower systems like mine) but doesn't generally work OK on faster systems.

                            @abeowitz:

                            Even with static IPs on WAN and OPT1, I still get lots of these in my dmesg logs:

                            ue1: link state changed to DOWN
                            ue1: link state changed to UP
                            ue0: link state changed to DOWN
                            ue0: link state changed to UP
                            pfsync0: promiscuous mode disabled
                            ue0: link state changed to DOWN
                            ue0: link state changed to UP
                            ue0: link state changed to DOWN
                            ue0: link state changed to UP
                            pfsync0: promiscuous mode enabled

                            Its likely that for a time the NICs didn't detect carrier from the devices they are connected to.

                            @TravisD:

                            Manually trying to remove these replicates the issue:

                            
                            [2.1-BETA1][root@gw.tubas.net]/conf(12): rm -f dyndns_*
                            rm: dyndns_wandyndns'travisd.dyndns.org'0.cache: Read-only file system
                            rm: dyndns_wanopendns'travisd.dyndns.org'1.cache: Read-only file system
                            
                            

                            Are you running nanoBSD based variant of pfSense? Please post the version string from the home page of your pfSense box.

                            1 Reply Last reply Reply Quote 0
                            • T
                              TravisD
                              last edited by

                              Yes, I'm running a nanoBSD variant. Currently:

                              2.1-BETA1 (i386)
                              built on Thu Feb 28 04:26:51 EST 2013

                              (but over the last several days I've been trying 2.0.1, 2.0.2, 2.0.3-SNAPSHOT, and a few 2.1 snapshots.

                              This is running on:  Intel(R) Atom(TM) CPU 330 @ 1.60GHz

                              It definitely appears to be a dhclient thing at this point. If I manually kill the running dhclient process and re-start it, things come right back up.

                              
                              [2.1-BETA1][root@gw.tubas.net]/sbin(86): /sbin/dhclient -c /var/etc/dhclient_wan.conf ue0
                              dhclient already running, pid: 72467.
                              exiting.
                              [2.1-BETA1][root@gw.tubas.net]/sbin(87): kill -KILL 72467
                              [2.1-BETA1][root@gw.tubas.net]/sbin(88): /sbin/dhclient -c /var/etc/dhclient_wan.conf ue0
                              dhclient: PREINIT
                              ifconfig: ioctl (SIOCAIFADDR): File exists
                              dhclient: Starting delete_old_states()
                              OLD_IP: not found
                              dhclient: Comparing IPs: Old:  New: 
                              dhclient: Comparing Routers: Old:  New: 
                              DHCPREQUEST on ue0 to 255.255.255.255 port 67
                              DHCPACK from 73.129.106.1
                              bound to 68.48.155.52 -- renewal in 158108 seconds.
                              
                              
                              1 Reply Last reply Reply Quote 0
                              • T
                                TravisD
                                last edited by

                                Beginning to believe I have two problems at play.  :-\

                                #1 is that the USB Ethernet adaptor doesn't seem to be very stable. This is the 2nd adaptor I've tried but same make/model as the first. I swapped the former outside (ue0) and inside (re0) interfaces around so now ue0 is the LAN.  After a little while, I saw it hiccup again, but since it's static it survived:

                                
                                Mar  1 04:01:38 gw check_reload_status: Linkup starting ue0
                                Mar  1 04:01:38 gw kernel: ue0: link state changed to DOWN
                                Mar  1 04:01:38 gw kernel: ue0: link state changed to UP
                                Mar  1 04:01:38 gw check_reload_status: Linkup starting ue0
                                Mar  1 04:01:40 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                Mar  1 04:01:40 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                Mar  1 04:01:41 gw check_reload_status: rc.newwanip starting ue0
                                Mar  1 04:01:43 gw php: : rc.newwanip: Informational is starting ue0.
                                Mar  1 04:01:43 gw php: : rc.newwanip: on (IP address: 192.168.100.1) (interface: lan) (real interface: ue0).
                                Mar  1 04:01:44 gw check_reload_status: Reloading filter
                                Mar  1 04:01:49 gw php: : Resyncing OpenVPN instances for interface LAN.
                                Mar  1 04:01:49 gw php: : Creating rrd update script
                                Mar  1 04:01:51 gw php: : pfSense package system has detected an ip change 0.0.0.0 ->  192.168.100.1 ... Restarting packages.
                                Mar  1 04:01:51 gw check_reload_status: Starting packages
                                Mar  1 04:01:54 gw php: : Restarting/Starting all packages.
                                
                                

                                The second problem seems to be that the quick down/up apparently plays havoc with the dhclient (what we're seeing in the earlier posts).  Related, but different issues from what I see.

                                I don't consider this fixed - it's just a bandaid solution. What other info can I provide to help here?

                                1 Reply Last reply Reply Quote 0
                                • Z
                                  Zorac
                                  last edited by

                                  i can confirm disabling openvpn doesnt solve the problem.   :-[

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    TravisD
                                    last edited by

                                    @Zorac:

                                    i can confirm disabling openvpn doesnt solve the problem.   :-[
                                    [/quote]

                                    Yeah, I left it out of my clean re-install and no change.

                                    Since I moved the USB interface to a static addressed interface, I'm still seeing it flap occasionally, but it doesn't get hung up with dhcp like it did on the WAN side. I think that eliminates my cablemodem from the cause of the flapping problem (it was connected directly to the cablemodem; now ue0 connects to a cisco switch).

                                    ue(4) driver problem? usb issue?

                                    When it flaps, it happens quickly enough where I don't even lose any pings when running a continuous test from another host.

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      TravisD
                                      last edited by

                                      Sure looks like this would be the bug in question, but seems like the fix may not be working.

                                      http://redmine.pfsense.org/issues/2792

                                      Edit: Multiple restarts at once?

                                      
                                      ar  2 20:02:54 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to DOWN
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to UP
                                      Mar  2 20:02:54 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to DOWN
                                      Mar  2 20:02:54 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to UP
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to DOWN
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to UP
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to DOWN
                                      Mar  2 20:02:54 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:54 gw kernel: ue0: link state changed to UP
                                      Mar  2 20:02:55 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:55 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:55 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:55 gw check_reload_status: Linkup starting ue0
                                      Mar  2 20:02:57 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:57 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:57 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:57 gw check_reload_status: rc.newwanip starting ue0
                                      Mar  2 20:02:58 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:58 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:58 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:58 gw check_reload_status: rc.newwanip starting ue0
                                      Mar  2 20:02:58 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:58 gw php: : Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.100.1 )
                                      Mar  2 20:02:58 gw check_reload_status: rc.newwanip starting ue0
                                      Mar  2 20:02:58 gw check_reload_status: rc.newwanip starting ue0
                                      Mar  2 20:02:59 gw php: : rc.newwanip: Informational is starting ue0.
                                      Mar  2 20:02:59 gw php: : rc.newwanip: on (IP address: 192.168.100.1) (interface: lan) (real interface: ue0).
                                      Mar  2 20:03:00 gw php: : rc.newwanip: Informational is starting ue0.
                                      Mar  2 20:03:00 gw php: : rc.newwanip: on (IP address: 192.168.100.1) (interface: lan) (real interface: ue0).
                                      Mar  2 20:03:00 gw php: : rc.newwanip: Informational is starting ue0.
                                      Mar  2 20:03:00 gw php: : rc.newwanip: on (IP address: 192.168.100.1) (interface: lan) (real interface: ue0).
                                      Mar  2 20:03:00 gw php: : rc.newwanip: Informational is starting ue0.
                                      Mar  2 20:03:00 gw php: : rc.newwanip: on (IP address: 192.168.100.1) (interface: lan) (real interface: ue0).
                                      Mar  2 20:03:01 gw check_reload_status: Reloading filter
                                      Mar  2 20:03:01 gw dhcpleases: kqueue error: unkown
                                      Mar  2 20:03:06 gw php: : Resyncing OpenVPN instances for interface LAN.
                                      Mar  2 20:03:06 gw php: : Resyncing OpenVPN instances for interface LAN.
                                      Mar  2 20:03:07 gw php: : Resyncing OpenVPN instances for interface LAN.
                                      Mar  2 20:03:07 gw php: : Resyncing OpenVPN instances for interface LAN.
                                      
                                      
                                      1 Reply Last reply Reply Quote 0
                                      • Z
                                        Zorac
                                        last edited by

                                        i gave up and went back to 2.0.1, hopefully ill be problem free

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          TravisD
                                          last edited by

                                          @Zorac:

                                          i gave up and went back to 2.0.1, hopefully ill be problem free

                                          I tried that, didn't seem to help, but curious what you find.

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            TravisD
                                            last edited by

                                            Since the "bad" interface now connects into my switch, I can look at the other end of the link. No sign of it flapping/bouncing on the switchport side.  No errors of any sort, and this is thru a couple of flapping events on the pfSense side:

                                            
                                            GigabitEthernet0/8 is up, line protocol is up (connected)
                                              Hardware is Gigabit Ethernet, address is 000f.f752.6308 (bia 000f.f752.6308)
                                              MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, 
                                                 reliability 255/255, txload 1/255, rxload 1/255
                                              Encapsulation ARPA, loopback not set
                                              Keepalive set (10 sec)
                                              Full-duplex, 100Mb/s
                                              input flow-control is off, output flow-control is off
                                              ARP type: ARPA, ARP Timeout 04:00:00
                                              Last input never, output 00:00:01, output hang never
                                              Last clearing of "show interface" counters 00:33:36
                                              Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
                                              Queueing strategy: fifo
                                              Output queue: 0/40 (size/max)
                                              5 minute input rate 637000 bits/sec, 37 packets/sec
                                              5 minute output rate 23000 bits/sec, 24 packets/sec
                                                 172453 packets input, 245147952 bytes, 0 no buffer
                                                 Received 30 broadcasts (0 multicast)
                                                 0 runts, 0 giants, 0 throttles
                                                 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
                                                 0 watchdog, 0 multicast, 0 pause input
                                                 0 input packets with dribble condition detected
                                                 110118 packets output, 10263992 bytes, 0 underruns
                                                 0 output errors, 0 collisions, 0 interface resets
                                                 0 babbles, 0 late collision, 0 deferred
                                                 0 lost carrier, 0 no carrier, 0 PAUSE output
                                                 0 output buffer failures, 0 output buffers swapped out
                                            
                                            
                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.