"igb1: link state changed to UP/DOWN" is it benign ?

JKnott

@chudak said in "igb1: link state changed to UP/DOWN" is it benign ?:

https://www.amazon.com/gp/product/B07GRG63P6/ref=ox_sc_saved_title_8?smid=ATVPDKIKX0DER&psc=1

Avoid TP-Link. Some models don't handle VLANs properly. If you're not planning on using VLANs, it's probably OK, but there are other brands in the same price range that don't have the VLAN problem.

However, a managed switch isn't necessary unless you want to use VLANs etc.

chudak

@jknott said in "igb1: link state changed to UP/DOWN" is it benign ?:

@chudak

Check your Ethernet cables & switch. If you're getting those errors, it indicates a problem in that area.

So far it did not work, still see errors :(

JKnott

@chudak

Well, it has something to do with the Ethernet connection. That means the NIC, cable or switch. How often does it happen? Any pattern?

chudak

@jknott

It's possible that I am paying more attention to the logs after 2.5.0, I see no "visible" problems.

See the frequency here:

Feb 27 12:03:52	kernel		igb1: link state changed to UP
Feb 27 12:03:49	kernel		igb1: link state changed to DOWN
Feb 27 10:19:28	kernel		igb1: link state changed to UP
Feb 27 10:19:24	kernel		igb1: link state changed to DOWN
Feb 27 09:07:20	kernel		igb1: link state changed to UP
Feb 27 09:07:16	kernel		igb1: link state changed to DOWN
Feb 27 09:03:58	kernel		igb1: link state changed to UP
Feb 27 09:03:55	kernel		igb1: link state changed to DOWN
Feb 27 07:00:56	kernel		igb1: link state changed to UP
Feb 27 07:00:52	kernel		igb1: link state changed to DOWN
Feb 27 03:41:55	kernel		igb1: link state changed to UP
Feb 27 03:41:52	kernel		igb1: link state changed to DOWN
Feb 27 03:20:36	kernel		igb1: link state changed to UP
Feb 27 03:20:32	kernel		igb1: link state changed to DOWN
Feb 27 02:53:23	kernel		igb1: link state changed to UP
Feb 27 02:53:19	kernel		igb1: link state changed to DOWN
Feb 27 00:33:20	kernel		igb1: link state changed to UP
Feb 27 00:33:16	kernel		igb1: link state changed to DOWN
Feb 26 23:57:50	kernel		igb1: link state changed to UP
Feb 26 23:57:46	kernel		igb1: link state changed to DOWN
Feb 26 22:01:58	kernel		ovpns1: link state changed to UP
Feb 26 22:01:58	kernel		ovpns1: link state changed to DOWN
Feb 26 21:01:54	kernel		igb1: link state changed to UP
Feb 26 21:01:50	kernel		igb1: link state changed to DOWN
Feb 26 19:47:01	kernel		igb1: link state changed to UP
Feb 26 19:46:58	kernel		igb1: link state changed to DOWN
Feb 26 19:09:28	kernel		igb1: link state changed to UP
Feb 26 19:09:25	kernel		igb1: link state changed to DOWN
Feb 26 16:05:55	kernel		igb1: link state changed to UP
Feb 26 16:05:51	kernel		igb1: link state changed to DOWN
Feb 26 14:24:08	kernel		igb1: link state changed to UP
Feb 26 14:24:05	kernel		igb1: link state changed to DOWN
Feb 26 11:02:24	kernel		igb1: link state changed to UP
Feb 26 11:02:20	kernel		igb1: link state changed to DOWN

See no obvious pattern...

The port and cable were changed around Feb 27 09:07

JKnott

@chudak

Those do seem frequent and it appears the link goes down for 3-4 seconds. Do you notice any effect when this happens? Does it show in the switch or NIC lights?

I just checked my system and haven't seen a link change since around noon yesterday, when I was doing some testing with openvpn. That would be around 29.5 hours since the last one. As an experiment, I just tried connecting and disconnecting my test LAN NIC to my notebook. Here's the result:

Feb 26 12:04:57 firewall kernel: ovpns1: link state changed to UP
Feb 27 17:32:10 firewall kernel: igb2: link state changed to UP
Feb 27 17:32:10 firewall kernel: igb2: link state changed to DOWN
Feb 27 17:32:11 firewall php-fpm[349]: /rc.linkup: DEVD Ethernet attached event for opt2
Feb 27 17:32:11 firewall php-fpm[349]: /rc.linkup: HOTPLUG: Configuring interface opt2
Feb 27 17:32:14 firewall kernel: igb2: link state changed to UP
Feb 27 17:32:15 firewall php-fpm[348]: /rc.linkup: DEVD Ethernet detached event for opt2
Feb 27 17:32:15 firewall php-fpm[51629]: /rc.linkup: DEVD Ethernet attached event for opt2
Feb 27 17:32:15 firewall php-fpm[51629]: /rc.linkup: HOTPLUG: Configuring interface opt2
Feb 27 17:32:17 firewall kernel: igb2: link state changed to DOWN
Feb 27 17:32:20 firewall php-fpm[349]: /rc.linkup: DEVD Ethernet detached event for opt2

I included one line from when I was testing my VPN, so you can see how long I went without any link state messages. This also shows what happens with a complete disconnection of the Ethernet cable.

Are you running any packages that might affect the connections?

chudak

@jknott

I need to keep my eye on this, so far I didn’t see any related problems, or at least did not register them as such

Don’t see much lights issues but I’m not watching my switch all the time

Run all standard packages

No errors in last ~4 h

Interesting...

chudak

@jknott

Update - I see no link state changed messages on LAN since yesterday.

What changed? I reinstalled all packages after seeing some warnings in the log.

However I see now this on WG:

Feb 28 09:09:40	kernel		wg0: link state changed to UP
Feb 28 09:09:40	kernel		wg0: link state changed to DOWN
Feb 28 09:09:39	kernel		wg0: link state changed to UP
Feb 28 09:09:39	kernel		wg0: link state changed to DOWN
Feb 28 09:09:38	kernel		wg0: link state changed to DOWN
Feb 27 12:03:52	kernel		igb1: link state changed to UP

The fact that this is on wg0 makes me think it maybe s/w related not cables etc.

JKnott

@chudak

You're running Wireguard and OpenVPN?

chudak

@jknott said in "igb1: link state changed to UP/DOWN" is it benign ?:

@chudak

You're running Wireguard and OpenVPN?

Yes

chudak

@jknott

Still see them:

Mar 20 07:20:31	check_reload_status	378	Reloading filter
Mar 20 07:20:31	check_reload_status	378	rc.newwanip starting igb1
Mar 20 07:20:31	php-fpm	74793	/rc.linkup: Hotplug event detected for LAN(lan) static IP (192.168.90.1 )
Mar 20 07:20:30	kernel		igb1: link state changed to UP
Mar 20 07:20:30	check_reload_status	378	Linkup starting igb1
Mar 20 07:20:27	check_reload_status	378	Reloading filter
Mar 20 07:20:27	php-fpm	71393	/rc.linkup: Hotplug event detected for LAN(lan) static IP (192.168.90.1 )
Mar 20 07:20:26	kernel		igb1: link state changed to DOWN
Mar 20 07:20:26	check_reload_status	378	Linkup starting igb1
Mar 20 06:52:21	arpwatch	79456	bogon 10.10.10.1 00:0e:c4:d1:6f:28
Mar 20 06:52:20	check_reload_status	378	Reloading filter

and nothing else ...
:( ???

stephenw10

Try swapping the LAN to a different port. Something other than igb1.

Steve

chudak

@stephenw10

Yeah that's probably only an option left.
Current assignment:
LAN - igb1

I do have igb3 NIC/port available.

Would you assign a new LAN2 -> igb3 (and then copy all FW rules to it)
or
is it safe simply change LAN -> igb3 ?

and connect an Ethernet cable to a new port after...

PS: Trying to make sure not to do anything stupid :)

Thx

stephenw10

Just reassign LAN as igb3 and all your existing rules etc will follow it.

chudak

@stephenw10

What can I tell?!
What was going to be a simple step, turned out to be not so easy :(

Changed LAN -> igb3, system after save does not show igb3, still shows igb1. No usual apply setting banner/button or Interface has been changed etc. Everything seems broken. I maybe pushing too hard enough, but was not able to make a new assignment to work.

Restored back to the previous configuration via Auto Config Setup.

I guess "link state changed to UP/DOWN" does not sound too bad after whole system is down :) that's a good output.

But still scratching my head WTH ? ...

stephenw10

Mmm, hard to say. I wouldn't expect an issue as long as the LAN was reconnected to the new NIC correctly.

chudak

@stephenw10

I think I could not change LAN -> igb3 because it was seeing by pfSense as "disabled".

Not sure why pfSense did show it in the drop down as available network port. Maybe there is some point, but actually feels like a bug to me.

Anyway, meanwhile added new LAN2 -> igb3 and setup with DHCP and one open to all FW rule. Will watch this for awhile and then will switch:

LAN2 -> igb1
LAN -> igb3

chudak

Update:

Assigned a new interface LAN2 -> igb3 (Logged a bug )

Connected only one laptop to it

Every time when laptop tuns on I see:

ligb3: ink state changed to UP

and when powered off -

igb3: link state changed to DOWN

This laptop connected by a long 20-30 cable, so it's possible some issues and it's not easy to replace. But overall troubleshooting was helpful.

It's still possible that my router box is faulty (which is unlikely) or pfSense throws false errors sometimes. Hard to say.

Thx for your help @stephenw10

stephenw10

Looks like igb3 works exactly as expect there then.