LAN interface flapping after (re)boot

asdjklfjkdslfdsaklj · Jul 15, 2023, 2:31 AM

23.05.1-RELEASE.

After rebooting, or power cycling, after about ~30sec, the LAN interface starts bouncing endlessly.

I got it to stop after nuking the interface configuration previously, and another time by running /etc/linkup stop em1 followed by start of the same.

Don't see any obvious precursor in the logs, just suddenly this... endless down/up cycle:

Jul 14 19:07:35 router check_reload_status[428]: Linkup starting em1
Jul 14 19:07:35 router kernel: em1: link state changed to DOWN
Jul 14 19:07:35 router kernel: em1.4: link state changed to DOWN
Jul 14 19:07:35 router check_reload_status[428]: Linkup starting em1.4
Jul 14 19:07:36 router php-fpm[398]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Jul 14 19:07:36 router php-fpm[398]: /rc.linkup: DEVD Ethernet detached event for lan
Jul 14 19:07:36 router php-fpm[399]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Jul 14 19:07:36 router php-fpm[399]: /rc.linkup: DEVD Ethernet detached event for opt1
Jul 14 19:07:36 router check_reload_status[428]: Reloading filter
Jul 14 19:07:36 router check_reload_status[428]: Reloading filter
Jul 14 19:07:38 router kernel: em1: link state changed to UP
Jul 14 19:07:38 router kernel: em1.4: link state changed to UP
Jul 14 19:07:38 router check_reload_status[428]: Linkup starting em1
Jul 14 19:07:38 router check_reload_status[428]: Linkup starting em1.4
Jul 14 19:07:39 router php-fpm[85068]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Jul 14 19:07:39 router php-fpm[85068]: /rc.linkup: DEVD Ethernet attached event for lan
Jul 14 19:07:39 router php-fpm[85068]: /rc.linkup: HOTPLUG: Configuring interface lan
Jul 14 19:07:39 router php-fpm[399]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Jul 14 19:07:39 router php-fpm[399]: /rc.linkup: DEVD Ethernet attached event for opt1
Jul 14 19:07:39 router php-fpm[399]: /rc.linkup: HOTPLUG: Configuring interface opt1

I plugged the LAN interface into a temporary dumb switch, with nothing else plugged in, to rule out my downstream switch being involved, and same cycle.

@jimp this sounds like what you mentioned here

stephenw10 · Jul 25, 2023, 3:54 PM

After that log does it just show link down again? Can we see logs showing at least two cycles?

asdjklfjkdslfdsaklj · Jul 26, 2023, 1:22 AM

@stephenw10 after intervention (linkup action I mentioned previously) the interface is up, and functional, otherwise sans intervention that log simply continues, seemingly endlessly. Steps to repeat are simply to reboot the device.

As an aside, L2 or even L1 reset of any kind during the "event" (i.e., switch interface disable, cable pull) causes the interface to stay down, flapping ends. When switch connection is restored, the cycle continues (or perhaps starts anew?)

I almost went down an exhaustive switch-side path, until plugging pfSense LAN into a dumb switch (no other connections, no STP, no nothing) produced the same results.

I'll attempt to gather logs just as soon as my schedule permits. Thanks for your time.

asdjklfjkdslfdsaklj · Aug 27, 2023, 3:37 AM

@stephenw10 correcting my previous post(s), an interface shutdown via ifconfig seemed to be the "kick".

Recent logs:

Aug 26 16:23:39 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:39 router kernel: em1: link state changed to DOWN
Aug 26 16:23:39 router kernel: em1.4: link state changed to DOWN
Aug 26 16:23:39 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:39 router kernel: em1: promiscuous mode enabled
Aug 26 16:23:40 router php-fpm[770]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:40 router php-fpm[770]: /rc.linkup: DEVD Ethernet detached event for opt1
Aug 26 16:23:40 router php-fpm[20820]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:40 router php-fpm[20820]: /rc.linkup: DEVD Ethernet detached event for lan
Aug 26 16:23:40 router check_reload_status[800]: Reloading filter
Aug 26 16:23:40 router check_reload_status[800]: Reloading filter
Aug 26 16:23:42 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:42 router kernel: em1: link state changed to UP
Aug 26 16:23:42 router kernel: em1.4: link state changed to UP
Aug 26 16:23:42 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:43 router php-fpm[20820]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:43 router php-fpm[20820]: /rc.linkup: DEVD Ethernet attached event for opt1
Aug 26 16:23:43 router php-fpm[20820]: /rc.linkup: HOTPLUG: Configuring interface opt1
Aug 26 16:23:43 router php-fpm[15597]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:43 router php-fpm[15597]: /rc.linkup: DEVD Ethernet attached event for lan
Aug 26 16:23:43 router php-fpm[15597]: /rc.linkup: HOTPLUG: Configuring interface lan
Aug 26 16:23:43 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:43 router kernel: em1: link state changed to DOWN
Aug 26 16:23:43 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:43 router kernel: em1.4: link state changed to DOWN
Aug 26 16:23:43 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:43 router check_reload_status[800]: updating dyndns lan
Aug 26 16:23:43 router check_reload_status[800]: Reloading filter
Aug 26 16:23:43 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:43 router check_reload_status[800]: updating dyndns opt1
Aug 26 16:23:44 router php-fpm[91883]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:44 router php-fpm[91883]: /rc.linkup: DEVD Ethernet detached event for lan
Aug 26 16:23:44 router php-fpm[15597]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:44 router php-fpm[15597]: /rc.linkup: DEVD Ethernet detached event for opt1
Aug 26 16:23:44 router check_reload_status[800]: Reloading filter
Aug 26 16:23:46 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:46 router kernel: em1: link state changed to UP
Aug 26 16:23:46 router kernel: em1.4: link state changed to UP
Aug 26 16:23:46 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:47 router php-fpm[15597]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:47 router php-fpm[15597]: /rc.linkup: DEVD Ethernet attached event for lan
Aug 26 16:23:47 router php-fpm[15597]: /rc.linkup: HOTPLUG: Configuring interface lan
Aug 26 16:23:47 router php-fpm[20820]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:47 router php-fpm[20820]: /rc.linkup: DEVD Ethernet attached event for opt1
Aug 26 16:23:47 router php-fpm[20820]: /rc.linkup: HOTPLUG: Configuring interface opt1
Aug 26 16:23:47 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:47 router kernel: em1: link state changed to DOWN
Aug 26 16:23:47 router kernel: em1.4: link state changed to DOWN
Aug 26 16:23:47 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:47 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:47 router check_reload_status[800]: updating dyndns opt1
Aug 26 16:23:47 router check_reload_status[800]: Reloading filter
Aug 26 16:23:47 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:47 router check_reload_status[800]: updating dyndns lan
Aug 26 16:23:47 router check_reload_status[800]: Reloading filter
Aug 26 16:23:48 router rtsold[59361]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Aug 26 16:23:48 router php-fpm[770]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:48 router php-fpm[770]: /rc.linkup: DEVD Ethernet detached event for lan
Aug 26 16:23:48 router php-fpm[91883]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:48 router php-fpm[91883]: /rc.linkup: DEVD Ethernet detached event for opt1
Aug 26 16:23:49 router php-fpm[15597]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Aug 26 16:23:49 router php-fpm[15597]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:8801:...:6805) (interface: wan) (real interface: em0).
Aug 26 16:23:50 router check_reload_status[800]: Linkup starting em1
Aug 26 16:23:50 router kernel: em1: link state changed to UP
Aug 26 16:23:50 router kernel: em1.4: link state changed to UP
Aug 26 16:23:50 router check_reload_status[800]: Linkup starting em1.4
Aug 26 16:23:51 router php-fpm[770]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Aug 26 16:23:51 router php-fpm[770]: /rc.linkup: DEVD Ethernet attached event for opt1
Aug 26 16:23:51 router php-fpm[770]: /rc.linkup: HOTPLUG: Configuring interface opt1
Aug 26 16:23:51 router php-fpm[20820]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Aug 26 16:23:51 router php-fpm[20820]: /rc.linkup: DEVD Ethernet attached event for lan
Aug 26 16:23:51 router php-fpm[20820]: /rc.linkup: HOTPLUG: Configuring interface lan
Aug 26 16:23:51 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:51 router check_reload_status[800]: updating dyndns opt1
Aug 26 16:23:51 router check_reload_status[800]: Reloading filter
Aug 26 16:23:51 router check_reload_status[800]: Restarting IPsec tunnels
Aug 26 16:23:51 router check_reload_status[800]: updating dyndns lan
Aug 26 16:23:51 router check_reload_status[800]: Reloading filter
Aug 26 16:23:54 router rtsold[74753]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Aug 26 16:23:55 router php-fpm[770]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Aug 26 16:23:55 router php-fpm[770]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:8801:...:6805) (interface: wan) (real interface: em0).
Aug 26 16:23:58 router check_reload_status[800]: Reloading filter
Aug 26 16:25:00 router php-cgi[50692]: [Suricata] Suricata signalled with SIGHUP for LAN (em1)...

asdjklfjkdslfdsaklj · Aug 27, 2023, 3:37 AM

Anecdotally, and without much deep diving, I'm starting to suspect this is happening when the interfaces are put into promiscuous mode c/o Suricata (6.0.13).

I disabled Suricata on a whim, rebooted, and haven't seen the problem yet.

stephenw10 · Aug 27, 2023, 2:23 PM

Are you able to test disabling the em1.4 (opt1) such that only LAN looses link?

It appears it could be the fact that both interfaces are restarted that causes a problem. Though I have interfaces with many VLANs that don't do that.

asdjklfjkdslfdsaklj · Nov 3, 2023, 10:20 PM

@stephenw10 Sorry for the late reply. I removed the VLAN sub-interface from em1, and put the VLAN into pfSense (access switch-side) on em2. Things flapped briefly after turning Suricate on (individually, on each phy). tl;dr, looks like problem occurs w/sub-interface in play.

Nov  3 15:10:05 router php[32364]: [Suricata] Building new sid-msg.map file for HOME...
Nov  3 15:10:06 router php[32364]: [Suricata] Updating rules configuration for: HOME_GUEST ...
Nov  3 15:10:07 router php[32364]: [Suricata] Enabling any flowbit-required rules for: HOME_GUEST...
Nov  3 15:10:07 router php[32364]: [Suricata] Building new sid-msg.map file for HOME_GUEST...
Nov  3 15:10:08 router php[32364]: [Suricata] Suricata START for HOME_GUEST(em2)...
Nov  3 15:10:24 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:24 router kernel: em1: link state changed to DOWN
Nov  3 15:10:24 router kernel: em1: promiscuous mode enabled
Nov  3 15:10:25 router lldpd[94864]: unable to send packet on real device for em1: Network is down
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:25 router check_reload_status[874]: Reloading filter
Nov  3 15:10:27 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:27 router kernel: em1: link state changed to UP
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:28 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:28 router kernel: em1: link state changed to DOWN
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:28 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:28 router check_reload_status[874]: Reloading filter
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:29 router check_reload_status[874]: Reloading filter
Nov  3 15:10:30 router rtsold[50036]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:10:31 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:31 router kernel: em1: link state changed to UP
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:32 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:32 router check_reload_status[874]: Reloading filter
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router check_reload_status[874]: Reloading filter
Nov  3 15:11:16 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:16 router kernel: em2: link state changed to DOWN
Nov  3 15:11:16 router kernel: em2: promiscuous mode enabled
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:17 router lldpd[94864]: unable to send packet on real device for em2: Network is down
Nov  3 15:11:17 router check_reload_status[874]: Reloading filter
Nov  3 15:11:19 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:19 router kernel: em2: link state changed to UP
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:20 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:20 router kernel: em2: link state changed to DOWN
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:20 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:20 router check_reload_status[874]: Reloading filter
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:21 router check_reload_status[874]: Reloading filter
Nov  3 15:11:22 router rtsold[22483]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:11:23 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:23 router kernel: em2: link state changed to UP
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:24 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:24 router check_reload_status[874]: Reloading filter
Nov  3 15:11:25 router php-fpm[15443]: /rc.dyndns.update: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:25 router php-fpm[88809]: /rc.filter_configure_sync: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router check_reload_status[874]: Reloading filter
Nov  3 15:10:03 router php[32364]: [Suricata] Updating rules configuration for: HOME ...
Nov  3 15:10:05 router php[32364]: [Suricata] Enabling any flowbit-required rules for: HOME...
Nov  3 15:10:05 router php[32364]: [Suricata] Building new sid-msg.map file for HOME...
Nov  3 15:10:06 router php[32364]: [Suricata] Updating rules configuration for: HOME_GUEST ...
Nov  3 15:10:07 router php[32364]: [Suricata] Enabling any flowbit-required rules for: HOME_GUEST...
Nov  3 15:10:07 router php[32364]: [Suricata] Building new sid-msg.map file for HOME_GUEST...
Nov  3 15:10:08 router php[32364]: [Suricata] Suricata START for HOME_GUEST(em2)...
Nov  3 15:10:24 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:24 router kernel: em1: link state changed to DOWN
Nov  3 15:10:24 router kernel: em1: promiscuous mode enabled
Nov  3 15:10:25 router lldpd[94864]: unable to send packet on real device for em1: Network is down
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:25 router check_reload_status[874]: Reloading filter
Nov  3 15:10:27 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:27 router kernel: em1: link state changed to UP
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:28 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:28 router kernel: em1: link state changed to DOWN
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:28 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:28 router check_reload_status[874]: Reloading filter
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:29 router check_reload_status[874]: Reloading filter
Nov  3 15:10:30 router rtsold[50036]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:10:31 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:31 router kernel: em1: link state changed to UP
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:32 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:32 router check_reload_status[874]: Reloading filter
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router check_reload_status[874]: Reloading filter
Nov  3 15:11:16 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:16 router kernel: em2: link state changed to DOWN
Nov  3 15:11:16 router kernel: em2: promiscuous mode enabled
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:17 router lldpd[94864]: unable to send packet on real device for em2: Network is down
Nov  3 15:11:17 router check_reload_status[874]: Reloading filter
Nov  3 15:11:19 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:19 router kernel: em2: link state changed to UP
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:20 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:20 router kernel: em2: link state changed to DOWN
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:20 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:20 router check_reload_status[874]: Reloading filter
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:21 router check_reload_status[874]: Reloading filter
Nov  3 15:11:22 router rtsold[22483]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:11:23 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:23 router kernel: em2: link state changed to UP
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:24 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:24 router check_reload_status[874]: Reloading filter
Nov  3 15:11:25 router php-fpm[15443]: /rc.dyndns.update: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:25 router php-fpm[88809]: /rc.filter_configure_sync: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router check_reload_status[874]: Reloading filter
Nov  3 15:10:05 router php[32364]: [Suricata] Building new sid-msg.map file for HOME...
Nov  3 15:10:06 router php[32364]: [Suricata] Updating rules configuration for: HOME_GUEST ...
Nov  3 15:10:07 router php[32364]: [Suricata] Enabling any flowbit-required rules for: HOME_GUEST...
Nov  3 15:10:07 router php[32364]: [Suricata] Building new sid-msg.map file for HOME_GUEST...
Nov  3 15:10:08 router php[32364]: [Suricata] Suricata START for HOME_GUEST(em2)...
Nov  3 15:10:24 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:24 router kernel: em1: link state changed to DOWN
Nov  3 15:10:24 router kernel: em1: promiscuous mode enabled
Nov  3 15:10:25 router lldpd[94864]: unable to send packet on real device for em1: Network is down
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:25 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:25 router check_reload_status[874]: Reloading filter
Nov  3 15:10:27 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:27 router kernel: em1: link state changed to UP
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:28 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:28 router kernel: em1: link state changed to DOWN
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:28 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:28 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:28 router check_reload_status[874]: Reloading filter
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:29 router php-fpm[836]: /rc.linkup: DEVD Ethernet detached event for lan
Nov  3 15:10:29 router check_reload_status[874]: Reloading filter
Nov  3 15:10:30 router rtsold[50036]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:10:31 router php-fpm[88809]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:10:31 router check_reload_status[874]: Linkup starting em1
Nov  3 15:10:31 router kernel: em1: link state changed to UP
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Hotplug event detected for HOME(lan) dynamic IP address (4: 192.168.0.1, 6: track6)
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: DEVD Ethernet attached event for lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: HOTPLUG: Configuring interface lan
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router php-fpm[836]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:10:32 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:10:32 router check_reload_status[874]: updating dyndns lan
Nov  3 15:10:32 router check_reload_status[874]: Reloading filter
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router php-fpm[88809]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:10:34 router check_reload_status[874]: Reloading filter
Nov  3 15:11:16 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:16 router kernel: em2: link state changed to DOWN
Nov  3 15:11:16 router kernel: em2: promiscuous mode enabled
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:17 router php-fpm[15443]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:17 router lldpd[94864]: unable to send packet on real device for em2: Network is down
Nov  3 15:11:17 router check_reload_status[874]: Reloading filter
Nov  3 15:11:19 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:19 router kernel: em2: link state changed to UP
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:20 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:20 router kernel: em2: link state changed to DOWN
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router php-fpm[88809]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:20 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:20 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:20 router check_reload_status[874]: Reloading filter
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:21 router php-fpm[837]: /rc.linkup: DEVD Ethernet detached event for opt1
Nov  3 15:11:21 router check_reload_status[874]: Reloading filter
Nov  3 15:11:22 router rtsold[22483]: Received RA specifying route fe80::3e8b:cdff:fec1:f131 for interface wan(em0)
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Nov  3 15:11:23 router php-fpm[836]: /rc.newwanipv6: rc.newwanipv6: on (IP address: 2600:xxxx:fab6) (interface: wan) (real interface: em0).
Nov  3 15:11:23 router check_reload_status[874]: Linkup starting em2
Nov  3 15:11:23 router kernel: em2: link state changed to UP
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Hotplug event detected for HOME_GUEST(opt1) dynamic IP address (4: 192.168.4.1, 6: track6)
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: DEVD Ethernet attached event for opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: HOTPLUG: Configuring interface opt1
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router php-fpm[15443]: /rc.linkup: Gateway, NONE AVAILABLE
Nov  3 15:11:24 router check_reload_status[874]: Restarting IPsec tunnels
Nov  3 15:11:24 router check_reload_status[874]: updating dyndns opt1
Nov  3 15:11:24 router check_reload_status[874]: Reloading filter
Nov  3 15:11:25 router php-fpm[15443]: /rc.dyndns.update: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:25 router php-fpm[88809]: /rc.filter_configure_sync: dpinger: No dpinger session running for gateway WAN_COX_DHCP6
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router php-fpm[836]: /rc.newwanipv6: Gateway, NONE AVAILABLE
Nov  3 15:11:27 router check_reload_status[874]: Reloading filter

stephenw10 · Nov 7, 2023, 3:23 PM

Do you have Suricata running in in-line mode on either em or the VLANs on it? That bounces the interface a lot more by inserting netmap.

asdjklfjkdslfdsaklj · Nov 7, 2023, 3:23 PM

@stephenw10 nope, legacy.

stephenw10 · Nov 7, 2023, 3:32 PM

Hmm, there must be something specific to your config applying to those interfaces triggering the race condition there...

asdjklfjkdslfdsaklj · Nov 8, 2023, 7:56 PM

@stephenw10 Aye, not exactly a burning community issue, and given this is home easy enough to do what I've done.

If this warrants effort on your side in the future, I'm happy to help further.