Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Link cycling on ue NIC

    Scheduled Pinned Locked Moved
    2.3-RC Snapshot Feedback and Issues - ARCHIVED
    2
    5
    1.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      iamzam
      last edited by

      @David_W:

      Your issue happens because your USB network interface keeps going down and coming back up. It really would be in your interests to get away from the problem by fixing or changing the interface if possible as an interface really should not be behaving this way.

      The root cause is that nothing is terminating the dhcpd / dhcpleases6 / dhcpleases processes when the interface goes down. It may well be that interface_bring_down() (/etc/inc/interfaces.inc from around line 1219) should do this, though it would be important to check for possible side effects before making this change because this function is used in /etc/rc.linkup when a hotpluggable interface goes down (as in your scenario) but also in many other places.

      Thanks, I will start another thread but the really strange thing that I just realized is that this interface stops doing this when I close the main pfsense page.  You can see that the syslog errors are from the php-fpm process:

      
[2.3-BETA][admin@pfs.dv.loc]/root: clog /var/log/system.log | grep linkup | tail -n 12
Jan 30 14:18:13 pfs php-fpm[82724]: /rc.linkup: HOTPLUG: Configuring interface opt1
Jan 30 14:18:15 pfs php-fpm[82724]: /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1454181495] unbound[73272:0] error: bind: address already in use [1454181495] unbound[73272:0] fatal error: could not open ports'
Jan 30 14:18:16 pfs php-fpm[82724]: /rc.linkup: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em0 ue0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 1 leases to leases file. Listening on BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Sending on   BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Listening on BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Sending on   BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf.   Also make sure you are not running HP JetAdmin software, which includes a bootp server.  If you did not get this software
Jan 30 14:40:29 pfs php-fpm[32332]: /rc.linkup: DEVD Ethernet detached event for opt1
Jan 30 14:40:29 pfs php-fpm[32332]: /rc.linkup: DEVD Ethernet attached event for opt1
Jan 30 14:40:29 pfs php-fpm[32332]: /rc.linkup: HOTPLUG: Configuring interface opt1
Jan 30 14:40:31 pfs php-fpm[32332]: /rc.linkup: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em0 ue0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 1 leases to leases file. Listening on BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Sending on   BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Listening on BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Sending on   BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf.   Also make sure you are not running HP JetAdmin software, which includes a bootp server.  If you did not get this software
Jan 30 14:40:32 pfs php-fpm[76534]: /rc.linkup: DEVD Ethernet detached event for opt1
Jan 30 14:40:32 pfs php-fpm[83145]: /rc.linkup: DEVD Ethernet attached event for opt1
Jan 30 14:40:32 pfs php-fpm[83145]: /rc.linkup: HOTPLUG: Configuring interface opt1
Jan 30 14:40:34 pfs php-fpm[83145]: /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1454182834] unbound[97426:0] error: bind: address already in use [1454182834] unbound[97426:0] fatal error: could not open ports'
Jan 30 14:40:35 pfs php-fpm[83145]: /rc.linkup: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em0 ue0' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.8 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 0 deleted host decls to leases file. Wrote 0 new dynamic host decls to leases file. Wrote 1 leases to leases file. Listening on BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Sending on   BPF/ue0/a0:ce:c8:01:c6:ca/172.22.23.0/24 Listening on BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Sending on   BPF/em0/00:90:fb:38:84:96/172.22.22.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf.   Also make sure you are not running HP JetAdmin software, which includes a bootp server.  If you did not get this software

      If I have the main dashboard open it will continuously do the DEVD detached/attached and unbound restart…

      1 Reply Last reply Reply Quote 0
      • C
        cmb
        last edited by

        The root issue is the NIC link cycling, which has no relation to the other thread so I split this to its own thread. Having traffic going to the host itself has caused link cycling on crappy NICs in the past. php-fpm is the source of the resulting logs because it's what handles things post-link cycle, not because the web interface has any relation.

        Does it definitely never cycle if you aren't logged into the web interface (or SSH or anything else to the box itself)?

        I'd suggest a better NIC.

        1 Reply Last reply Reply Quote 0
        • I
          iamzam
          last edited by

          Hey,  sorry I wasn't able to concentrate on this problem because I had a few other issues that have been worked out now, so now this one is pretty clear to me.

          On the dashboard, the Traffic Interface Statistics widget is the culprit.  I wouldn't say the USB interface is rock solid if I'm not on the dashboard with that widget, but with widget vs. without widget, the difference is stark.

          If I enable the widget, and just leave the dashboard open, the interface reloads at least once a minute or maybe two.  Without that widget on, maybe once or twice a day I will notice that there are multiple copies of dhcpd -6 and dhcpleases6 and I can also see in the syslog a /rc.linkup: DEVD message like:

          
2/5/16 04:56:33.000 php-fpm[48967]: /rc.linkup: DEVD Ethernet detached event for opt1
2/5/16 04:56:33.000 php-fpm[50517]: /rc.linkup: DEVD Ethernet attached event for opt1
2/5/16 04:56:33.000 php-fpm[50517]: /rc.linkup: HOTPLUG: Configuring interface opt1

          That was actually the last time this happened in the syslog (time is EST) but if I enabled that widget it would happen all day.  The fan on my Mac mini will even startup when the widget is on and open in Firefox, not that I have tested any other browsers yet, but anyway.

          I don't doubt that the root issue is related to the crappy USB but the widget is definitely a contributing factor.

          If you wanted to test it for yourself, here is the actual interface from usbconfig (the last one):

          
[2.3-BETA][admin@pfs.dv.loc]/root: usbconfig
ugen0.1: <ehci root="" hub="" intel="">at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen1.1: <ehci root="" hub="" intel="">at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen1.2: <product 0x0020="" vendor="" 0x8087="">at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.2: <product 0x0020="" vendor="" 0x8087="">at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.3: <usb receiver="" logitech="">at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (98mA)
ugen0.4: <ax88179 asix="" elec.="" corp.="">at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (248mA)</ax88179></usb></product></product></ehci></ehci>

          Available here and other places:
          https://duckduckgo.com/?q=AX88179+cable+matters+usb+ethernet&t=ffsb&ia=products
          https://www.google.com/search?tbm=shop&q=AX88179+cable+matters+usb+ethernet

          • Edited to change Traffic Statistics widget to Interface Statistics widget
          1 Reply Last reply Reply Quote 0
          • C
            cmb
            last edited by

            @iamzam:

            On the dashboard, the Traffic Interface Statistics widget is the culprit.  I wouldn't say the USB interface is rock solid if I'm not on the dashboard with that widget, but with widget vs. without widget, the difference is stark.

            Hm, so something in get_interface_info function, which just reads statistics from the interface, apparently causes link cycling with that driver. That's your LAN NIC if I recall? It's configured with a static IP I presume? Is it bridged? The code paths are somewhat diff there with static vs. DHCP vs. PPPoE, etc.

            1 Reply Last reply Reply Quote 0
            • I
              iamzam
              last edited by

              I am using it as an OPT1 (additional LAN) interface with a single wireless AP plugged into it through a POE injector, no switch.  The IPv4 of the interface is static with DHCP on the subnet.  IPv6 tracks the WAN interface and gets an additional /64 from the delegated /60.  Not bridged in any way.

              1 Reply Last reply Reply Quote 0
              • First post
                Last post