strange errorThere were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured

pfsss

error reports in the Notice of wegui, it says "There were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured - The line in question reads [0]: @ 2024-02-25 23:21:20"

After the error, my Wan is down and cannot resume. Only after I reboot my router, Wan comes correct.
This error occurs especially when I download files in a very hight speed, for example 80MB/s.

The system is not crashed, only the Wan is error.

looking for help!

pfsss

@pfsss said in strange errorThere were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured:

pfctl: SIOCGIFGROUP: Device not configured

@stephenw10

stephenw10

Is that after upgrading?

The WAN shows as down in the gateway monitoring? In the Interface Status? What sort of WAN is it?

Steve

pfsss

@stephenw10
my Wan network is pppoe via a vlan to the ISP. The interface status page cannot connect to the ISP, and the gateway page shows both Wans (PPPOE and SLAAC) are pending.

stephenw10

If the parent interface still linked when that happens? What does ifconfig show?

What do the ppp logs show when the connection attempt fails?

It seems likely that pfctl error is a result of the WAN failing rather than the cause. Though the time-stamps in the logs should should that.

pfsss

@stephenw10 here is one log got from the filter logs

Feb 27 23:19:26	ppp	25955	[wan] SECDNS XXX.XXX.XXX.XXX
Feb 27 23:19:26	ppp	25955	[wan] IPCP: state change Ack-Sent --> Opened
Feb 27 23:19:26	ppp	25955	[wan] IPCP: LayerUp
Feb 27 23:19:26	ppp	25955	[wan] XXX.XXX.XXX.176 -> XXX.XXX.XXX.1
Feb 27 23:24:17	ppp	25955	caught fatal signal TERM
Feb 27 23:24:17	ppp	25955	[wan] IFACE: Close event
Feb 27 23:24:17	ppp	25955	[wan] IPCP: Close event
Feb 27 23:24:17	ppp	25955	[wan] IPCP: state change Opened --> Closing
Feb 27 23:24:17	ppp	25955	[wan] IPCP: SendTerminateReq #4
Feb 27 23:24:17	ppp	25955	[wan] IPCP: LayerDown
Feb 27 23:24:18	ppp	25955	[wan] IFACE: Removing IPv4 address from pppoe1 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address
Feb 27 23:24:18	ppp	25955	[wan] IPV6CP: Close event
Feb 27 23:24:18	ppp	25955	[wan] IPV6CP: state change Opened --> Closing
Feb 27 23:24:18	ppp	25955	[wan] IPV6CP: SendTerminateReq #3
Feb 27 23:24:18	ppp	25955	[wan] IPV6CP: LayerDown
Feb 27 23:24:19	ppp	25955	[wan] IFACE: Down event
Feb 27 23:24:19	ppp	25955	[wan] IFACE: Rename interface pppoe1 to pppoe1
Feb 27 23:24:19	ppp	25955	[wan] IFACE: Set description "WAN"
Feb 27 23:24:19	ppp	25955	[wan] IPCP: SendTerminateReq #5
Feb 27 23:24:20	ppp	25955	[wan] IPV6CP: SendTerminateReq #4
Feb 27 23:24:20	ppp	54740	Multi-link PPP daemon for FreeBSD
Feb 27 23:24:20	ppp	54740	process 54740 started, version 5.9
Feb 27 23:24:20	ppp	54740	waiting for process 25955 to die...
Feb 27 23:24:21	ppp	25955	[wan] Bundle: Shutdown
Feb 27 23:24:21	ppp	25955	[wan_link0] Link: Shutdown
Feb 27 23:24:21	ppp	25955	process 25955 terminated
Feb 27 23:24:21	ppp	54740	web: web is not running
Feb 27 23:24:21	ppp	54740	[wan] Bundle: Interface ng0 created
Feb 27 23:24:21	ppp	54740	[wan_link0] Link: OPEN event
Feb 27 23:24:21	ppp	54740	[wan_link0] LCP: Open event
Feb 27 23:24:21	ppp	54740	[wan_link0] LCP: state change Initial --> Starting
Feb 27 23:24:21	ppp	54740	[wan_link0] LCP: LayerStart
Feb 27 23:24:21	ppp	54740	[wan_link0] PPPoE: Connecting to ''
Feb 27 23:24:30	ppp	54740	[wan_link0] PPPoE connection timeout after 9 seconds
Feb 27 23:24:30	ppp	54740	[wan_link0] Link: DOWN event
Feb 27 23:24:30	ppp	54740	[wan_link0] LCP: Down event
Feb 27 23:24:30	ppp	54740	[wan_link0] Link: reconnection attempt 1 in 1 seconds
Feb 27 23:24:31	ppp	54740	[wan_link0] Link: reconnection attempt 1
Feb 27 23:24:31	ppp	54740	[wan_link0] PPPoE: Connecting to ''
Feb 27 23:24:40	ppp	54740	[wan_link0] PPPoE connection timeout after 9 seconds
Feb 27 23:24:40	ppp	54740	[wan_link0] Link: DOWN event
Feb 27 23:24:40	ppp	54740	[wan_link0] LCP: Down event
Feb 27 23:24:40	ppp	54740	[wan_link0] Link: reconnection attempt 2 in 3 seconds
Feb 27 23:24:43	ppp	54740	caught fatal signal TERM
Feb 27 23:24:43	ppp	54740	[wan] IFACE: Close event
Feb 27 23:24:43	ppp	54740	[wan] IPCP: Close event
Feb 27 23:24:43	ppp	54740	[wan] IPV6CP: Close event
Feb 27 23:24:43	ppp	54740	[wan_link0] LCP: Close event
Feb 27 23:24:43	ppp	54740	[wan_link0] LCP: state change Starting --> Initial
Feb 27 23:24:43	ppp	54740	[wan_link0] LCP: LayerFinish
Feb 27 23:24:45	ppp	54740	[wan] Bundle: Shutdown
Feb 27 23:24:45	ppp	54740	[wan_link0] Link: Shutdown
Feb 27 23:24:45	ppp	54740	process 54740 terminated
Feb 27 23:24:47	ppp	80052	Multi-link PPP daemon for FreeBSD
Feb 27 23:24:47	ppp	80052	process 80052 started, version 5.9
Feb 27 23:24:47	ppp	80052	web: web is not running
Feb 27 23:24:47	ppp	80052	[wan] Bundle: Interface ng0 created

the above log shows that crush happens at Feb 27 23:24:17

the other error log came from the system log as below:

Feb 27 23:37:19	root	55800	Bootup complete
Feb 27 23:37:21	login	64471	login on ttyv0 as root
Feb 27 23:37:21	sshguard	67007	Now monitoring attacks.
Feb 28 00:01:43	rc.gateway_alarm	74290	>>> Gateway alarm: WAN_PPPOE (Addr:XXX.XXX.XXX.1 Alarm:1 RTT:5.823ms RTTsd:12.159ms Loss:21%)
Feb 28 00:01:43	check_reload_status	430	updating dyndns WAN_PPPOE
Feb 28 00:01:43	check_reload_status	430	Restarting IPsec tunnels
Feb 28 00:01:43	check_reload_status	430	Restarting OpenVPN tunnels/interfaces
Feb 28 00:01:43	check_reload_status	430	Reloading filter
Feb 28 00:01:44	rc.gateway_alarm	76965	>>> Gateway alarm: WAN_SLAAC (Addr:fe80::XXXX:XXXX:XXXX:XXXX%pppoe1 Alarm:1 RTT:5.072ms RTTsd:8.538ms Loss:22%)
Feb 28 00:01:44	check_reload_status	430	updating dyndns WAN_SLAAC
Feb 28 00:01:44	check_reload_status	430	Restarting IPsec tunnels
Feb 28 00:01:44	check_reload_status	430	Restarting OpenVPN tunnels/interfaces
Feb 28 00:01:44	check_reload_status	430	Reloading filter
Feb 28 00:01:45	php-fpm	398	/rc.openvpn: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
Feb 28 00:01:45	php-fpm	398	/rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. 'WAN_SLAAC'
Feb 28 00:01:45	php-fpm	399	/rc.dyndns.update: phpDynDNS (***********): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:46	php-fpm	398	/rc.dyndns.update: phpDynDNS (***********): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:47	php-fpm	18183	/rc.openvpn: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
Feb 28 00:01:47	php-fpm	18183	/rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. 'WAN_SLAAC'
Feb 28 00:01:47	php-fpm	399	/rc.dyndns.update: phpDynDNS (^^^^^^^^^): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:48	php-fpm	398	/rc.dyndns.update: phpDynDNS (^^^^^^^^^): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:49	php-fpm	399	/rc.dyndns.update: phpDynDNS (~~~~~~~~~~): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:50	php-fpm	398	/rc.dyndns.update: phpDynDNS (~~~~~~~~~~): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Feb 28 00:01:51	ppp	26054	[wan_link0] LCP: no reply to 1 echo request(s)
Feb 28 00:01:52	check_reload_status	430	Linkup starting re0
Feb 28 00:01:52	kernel		re0: watchdog timeout
Feb 28 00:01:52	kernel		re0: link state changed to DOWN
Feb 28 00:01:52	kernel		re0.1154: link state changed to DOWN
Feb 28 00:01:52	check_reload_status	430	Linkup starting re0.1233
Feb 28 00:01:53	check_reload_status	430	Reloading filter
Feb 28 00:01:53	ppp	26054	caught fatal signal TERM
Feb 28 00:01:53	ppp	26054	[wan] IFACE: Close event
Feb 28 00:01:53	ppp	26054	[wan] IPCP: Close event
Feb 28 00:01:53	ppp	26054	[wan] IPCP: state change Opened --> Closing
Feb 28 00:01:53	ppp	26054	[wan] IPCP: SendTerminateReq #4
Feb 28 00:01:53	ppp	26054	[wan] IPCP: LayerDown
Feb 28 00:01:54	php-cgi	29860	rc.kill_states: rc.kill_states: Removing states for IP XXX.XXX.XXX.XXX/32
Feb 28 00:01:54	php-cgi	29860	rc.kill_states: rc.kill_states: Removing states for interface pppoe1
Feb 28 00:01:54	check_reload_status	430	Rewriting resolv.conf
Feb 28 00:01:54	ppp	26054	[wan] IFACE: Removing IPv4 address from pppoe1 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address
Feb 28 00:01:54	ppp	26054	[wan] IPV6CP: Close event
Feb 28 00:01:54	ppp	26054	[wan] IPV6CP: state change Opened --> Closing
Feb 28 00:01:54	ppp	26054	[wan] IPV6CP: SendTerminateReq #3
Feb 28 00:01:54	ppp	26054	[wan] IPV6CP: LayerDown
Feb 28 00:01:55	php-cgi	41578	rc.kill_states: rc.kill_states: Removing states for IP fe80::XXXX:XXXX:XXXX:XXXX%pppoe1/32
Feb 28 00:01:55	php-cgi	41578	rc.kill_states: rc.kill_states: Removing states for interface pppoe1
Feb 28 00:01:55	check_reload_status	430	Linkup starting re0
Feb 28 00:01:55	kernel		re0: link state changed to UP
Feb 28 00:01:55	kernel		re0.1154: link state changed to UP
Feb 28 00:01:55	check_reload_status	430	Linkup starting re0.1233
Feb 28 00:01:55	check_reload_status	430	Rewriting resolv.conf
Feb 28 00:01:55	ppp	26054	[wan] IFACE: Down event
Feb 28 00:01:55	ppp	26054	[wan] IFACE: Rename interface pppoe1 to pppoe1
Feb 28 00:01:55	ppp	26054	[wan] IFACE: Set description "WAN"
Feb 28 00:01:56	ppp	26054	[wan] IPCP: SendTerminateReq #5
Feb 28 00:01:56	ppp	26054	[wan] IPV6CP: SendTerminateReq #4
Feb 28 00:01:56	check_reload_status	430	Reloading filter
Feb 28 00:01:56	ppp	69952	Multi-link PPP daemon for FreeBSD
Feb 28 00:01:56	ppp	69952	process 69952 started, version 5.9
Feb 28 00:01:56	ppp	69952	waiting for process 26054 to die...
Feb 28 00:01:56	php-fpm	18183	/rc.linkup: calling interface_dhcpv6_configure.
Feb 28 00:01:57	php-fpm	18183	/rc.linkup: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
Feb 28 00:01:57	php-fpm	18183	/rc.linkup: Gateway, none 'available' for inet6, use the first one configured. 'WAN_SLAAC'
Feb 28 00:01:57	check_reload_status	430	Restarting IPsec tunnels
Feb 28 00:01:57	check_reload_status	430	updating dyndns wan
Feb 28 00:01:57	ppp	26054	[wan] Bundle: Shutdown
Feb 28 00:01:57	ppp	26054	[wan_link0] Link: Shutdown
Feb 28 00:01:57	ppp	26054	process 26054 terminated
Feb 28 00:01:57	ppp	69952	web: web is not running
Feb 28 00:01:57	ppp	69952	[wan] Bundle: Interface ng0 created
Feb 28 00:01:57	ppp	69952	[wan_link0] Link: OPEN event
Feb 28 00:01:57	kernel		ng0: changing name to 'pppoe1'
Feb 28 00:01:57	ppp	69952	[wan_link0] LCP: Open event
Feb 28 00:01:57	ppp	69952	[wan_link0] LCP: state change Initial --> Starting
Feb 28 00:01:57	ppp	69952	[wan_link0] LCP: LayerStart
Feb 28 00:01:57	ppp	69952	[wan_link0] PPPoE: Connecting to ''

the above log shows the crash happens at Feb 28 00:01:43.

stephenw10

@pfsss said in strange errorThere were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured:

Feb 28 00:01:52 kernel re0: watchdog timeout
Feb 28 00:01:52 kernel re0: link state changed to DOWN

Ok, that looks like a NIC/driver issue. You should try using the kmod-realtek driver package.

Ellis Michael Lieberman

Just got this in an (Notifications in this message: 1) email this morning.

16:33:32 There were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured - The line in question reads [0]:

... BUT

I have been seeing a bouncing of the connection between the WAN Ethernet port on the dedicated unit that runs pfSense+ as it connects to the ISP bridge for PPOE. ever since I upgraded to:

Version	23.09.1-RELEASE (amd64)
built on Wed Jan 10 7:58:00 -08 2024
FreeBSD 14.0-CURRENT

I am running:

Intel(R) Celeron(R) N5105 @ 2.00GHz
Current: 2795 MHz, Max: 1996 MHz
4 CPUs : 1 package(s) x 4 core(s)

The BIOS version on my dedicated device is:

Vendor: Techvision, LLC.
Version: 5.19
Release Date: Wed Sep 7 2022

A log entry says:

Feb 28 16:33:41	php-fpm	76679	/rc.newwanip: Netgate pfSense Plus package system has detected an IP change or dynamic WAN reconnection - 58.69.150.45 -> 58.69.150.45 - Restarting packages.

and then

Feb 28 16:33:42	php-fpm	72862	/rc.start_packages: Configuration Change: (system): pfBlockerNG: saving DNSBL changes

BUT I have a static IP address from the ISP and is does NOT change. I have had this address for many years now and I never saw bouncing on a WAN interface until the latest BSD release.
PLEASE pay attention to this.

stephenw10

Check the system log for link state changes.

Ellis Michael Lieberman

@stephenw10
I did and you see all there is within the log file related to that in my post above. There are previous ones, but they are no different. None existed prior to the software upgrade,

Ellis Michael Lieberman

@stephenw10
I should also note that I have an active network monitoring system running on a server. I can see when something is bouncing. This is what that part of the map looks like when the interface is up.
alt text
It goes read and sounds an alarm with the interface goes down, and just so lucky iI caught the very end of it as I was writing this!

The log shows only this.

Feb 28 20:25:00	sshguard	25376	Exiting on signal.
Feb 28 20:25:00	sshguard	64179	Now monitoring attacks.
Feb 28 20:26:03	kernel		arp: 00:0c:43:e1:76:29 is using my IP address 192.168.1.1 on igc3!
Feb 28 20:26:05	kernel		arp: 00:0c:43:e1:76:29 is using my IP address 192.168.1.1 on igc3!
Feb 28 20:32:00	sshguard	64179	Exiting on signal.

And like normal there was no "email" notification.
AND there is NO 00:0c:43:e1:76:29 seen when I type 'arp -n -a' on the pfSense+ unit. (I am happy to upload that is needed. :-)

stephenw10

If the interface actually goes down, bounces as you say, then there will be log messages indicating that.

The logs you posted above show simply that the WAN reconnected at some point but not the cause of that which would have been before that in that logs.

What version did you upgrade from?

Ellis Michael Lieberman

@stephenw10
I honestly don't remember the version but it was probably the most recent.

Here is a sample from the system gateway log. Once again there errors did not appear before the upgrade I did in January. They started and the bouncing immediately following the upgraded version.

Feb 28 16:33:28	dpinger	4624	WAN_PPPOE 58.69.145.0: sendto error: 65
Feb 28 16:33:28	dpinger	4624	exiting on signal 15
Feb 28 16:33:38	dpinger	17224	send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 58.69.145.0 bind_addr 58.69.150.45 identifier "WAN_PPPOE "
Feb 28 23:18:21	dpinger	17224	WAN_PPPOE 58.69.145.0: Alarm latency 1607111us stddev 3527976us loss 0%
Feb 28 23:18:23	dpinger	17224	WAN_PPPOE 58.69.145.0: sendto error: 65
Feb 28 23:18:23	dpinger	17224	WAN_PPPOE 58.69.145.0: sendto error: 65
Feb 28 23:18:24	dpinger	17224	WAN_PPPOE 58.69.145.0: sendto error: 65
Feb 28 23:18:24	dpinger	17224	exiting on signal 15
Feb 28 23:18:39	dpinger	89624	send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000m

stephenw10

The system log at that time would show more. Again the gateway log only shows the consequences of something else. Latency hits 1.6s so it throws an alarm. Then 3s later dpinger is killed and then restarted presumably after the ppp session reconnects.

Ellis Michael Lieberman

@stephenw10
I understand that the log 'should' but it doesn't. Do I need to send a larger snippet to prove it?

stephenw10

I mean it's possible the WAN failed because of something upstream. Maybe your WAN latency is very sensitive to traffic? Check the Status > Monitoring graphs for traffic flow at that time.

Ellis Michael Lieberman

@stephenw10
Do you see the RED "The Wall" in the second graphic above. That is what you see when an 'Device drops not an interface. And what is it monitoring? Intermapper is monitoring the address 192.168.1.1 on the device (the LAN port). So the drops are NOT an upstream issue. Maybe they're the phantom MAC address that is NOT on my system. (Unless you want to argue that the ISP bridge, on occasion forget's it's a bridge and starts announcing it self as my gateway.) So maybe? the drop can be related to an upstream issue, but it is the device does we see drop. The software I am using is Intermapper from Help Systems. It is getting its information via SNMP from the router/firewall.I never saw the issued related to the WAN port before, I never saw the device drops I see now, until the software upgrade occurred. The issues stared within hours.

Until I got the 'email notification,' I had nothing in the logs other than a phantom MAC address that doesn't exist on my network to look at. But the email got me writing this.

For what it's worth the manufacturer associated with the MAC address does not have an obvious connection to the devices on my network..(See below). All such devices that are of the same tye but different manufacturer are assigned proper IP addresses and can be found on the network.

From https://en.wikipedia.org/wiki/Ralink
Ralink Technology, Corp. is a Wi-Fi chipset manufacturer mainly known for their IEEE 802.11 (Wireless LAN) chipsets. Ralink was founded in 2001 in Cupertino, California, then moved its headquarters to Hsinchu, Taiwan. On 5 May 2011, Ralink was acquired by MediaTek.

Some of Ralink's 802.11n RT2800 chipsets have been accepted into the Wi-Fi Alliance 802.11n draft 2.0 core technology testbed. They have also been selected in the Wi-Fi Protected Setup (WPS) and Wireless Multimedia Extensions Power Save (WMM-PS) testbeds. Ralink was a participant in the Wi-Fi Alliance and the IEEE 802.11 standards committees.[1] Ralink chipsets are used in various consumer-grade routers made by Gigabyte Technology, Linksys, D-Link, Asus and Belkin, as well as Wi-Fi adaptors for USB, PCI, ExpressCard, PC Card, and PCI Express interfaces. An example of an adapter is the Nintendo Wi-Fi USB Connector which uses the Ralink RT2570 chipset to allow a Nintendo DS or Wii to be internetworked via a home computer.

stephenw10

OK well if it's monitoring the LAN side then the reported LAN IP conflict could certainly be causing it:
arp: 00:0c:43:e1:76:29 is using my IP address 192.168.1.1 on igc3

How often do you see that logged?

Is igc3 the LAN interface?

Check the MAC address in the ARP table of the monitoring system when it shows as down.

The device is on your network somehow. Using that IP makes it likely to be a router/firewall of some sort.

Ellis Michael Lieberman

@stephenw10
The MAC address does NOT appear on the arp table for: (1)the netgate device; (2) the DHCP server (a Linux server); (3) my desktop; the arp table of my managed switch. Angry IP scanner does not see it. There are 53 hosts plus mobile units, but in each case I know both the MAC address and the assigned IP.

So either this is a phantom but real MAC address that only decided to come alive as soon as the last Netgate/FreeBSD code was installed (as nothing had at that time been added to the network, or it is the Netgate/FreeBSD code. Are you wanting to argue that the phantom MAC address and the attached gateway IP was always there and earlier Netgate/FreeBSD code wasn't good enough to see it? I am confused.

While at the same time, there was the appearance of the PPPoE issue regarding error 65 on the gateway.

stephenw10

@Ellis-Michael-Lieberman said in strange errorThere were error(s) loading the rules: pfctl: SIOCGIFGROUP: Device not configured:

The MAC address does NOT appear on the arp table for: (1)the netgate device; (2) the DHCP server (a Linux server); (3) my desktop; the arp table of my managed switch. Angry IP scanner does not see it.

That is when the firewall shows as down in the monitoring system?

It could be coincidental that the monitoring system stops being able to pull responses from the firewall and the firewall is logging another device using it's IP.

I would guess something is falling back to a default config or perhaps using that IP/MAC when it boots and then switching to something else.

However something on the LAN using the same IP would not cause a problem on the WAN side. That assumes igc3 is the LAN as I asked above?