some services show can't start

yon 0

After entering this command, nothing happens.


[23.09-BETA][admin@]/root: pfSense-repoc
[23.09-BETA][admin@]/root:

stephenw10

That's expected. Can it now upgrade?

yon 0

@stephenw10

waiting for this long time

[23.09-BETA][admin@]/root: pfSense-upgrade -dy
ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense
Error updating repositories!

stephenw10

IIRC you previously had issues with IPv6 failing there. Try: pfSense-upgrade -d4

yon 0

@stephenw10

[23.09-BETA][admin@]/root: pfSense-upgrade -d4
ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
pkg-static: An error occured while fetching package
pkg-static: Impossible to get the value from Last-Modified HTTP header
Unable to update repository pfSense
Error updating repositories!

stephenw10

And I assume the same if you force v6?

yon 0

@stephenw10

Oct 15 01:31:53 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s).
Oct 15 01:31:53 check_reload_status 507 Syncing firewall
Oct 15 01:31:53 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s).
Oct 15 01:31:53 check_reload_status 507 Syncing firewall
Oct 15 01:31:54 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard).
Oct 15 01:31:54 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard).
Oct 15 01:31:55 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard).
Oct 15 01:31:55 check_reload_status 507 Syncing firewall
Oct 15 01:31:55 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard).
Oct 15 01:31:55 check_reload_status 507 Syncing firewall
Oct 15 01:31:56 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary.

yon 0

@stephenw10

I successfully upgraded once after restarting the system, but the problem remains the same. It can only succeed occasionally.

yon 0

Since frr8 requires that the fe80:: address must be configured, but pfsense appears to block such addresses

Oct 15 01:31:57 kernel wg0: changing name to 'tun_wg0'
Oct 15 01:31:57 php_wg 53450 /usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/ifconfig tun_wg0 inet6 'fe80::981f:60ff:fee9:56d3' -alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Oct 15 01:31:57 kernel tun_wg0: link state changed to UP
Oct 15 01:31:58 kernel [fib_algo] inet6.0 (radix6_lockless#844) rebuild_fd_flm: switching algo to radix6

yon 0

my wireguard config connect to DTLS tunnle local forward to remote server, pfsense service seem can't allow this. but wiregaurd allow this, only pfsense limited it.

yon 0

pfsense frr bgp session incorrectly monitors the wiregaurd service status according to pfsense and interrupts the connection of frr bgp sessions. In fact, all wiregaurd and frr bgp can be connected normally. The biggest problem at present is a series of errors caused by pfsense. In other words, if there is no pfsense Error limits, everything works fine.

yon 0

The previous pfsense versions frr bgp will not interrupt bgp sessions due to the wiregaurd service status. I don’t know what code pfsense has changed.

stephenw10

Does pkg update always succeed or did we just happen to see it succeed previously?

Its very odd that pkg update succeeds and repoc succeeds but pfSense-upgrade fails. And that it fails with an error that implies it cannot find the file. pkg update clearly was able to find it.

You should be seeing FRR 9 if you're on the latest beta build.

yon 0

@stephenw10

pkg update is ok. pfSense-upgrade not normal work.

i have using frr9. Since frr8- frr9 requires that the fe80:: address must be configured. so wiregaurd need add this fe80:: address.

How do I now change Pfsense's limit on wiregaurd service status errors? And how to cancel frr bgp sessions to establish a connection based on the wireguard service status?

The main problem now is that wiregaurd has connected to the tunnel normally, and pfsense mistakenly believes that wiregaurd is not working properly and stops the service. At the same time, frr bgp sessions also stop working.

i am using the version

23.09-BETA (amd64)
built on Fri Oct 13 14:00:00 CST 2023
FreeBSD 14.0-CURRENT

stephenw10

Ok, as I stated previously, the service that pfSense is checking for is php_wg. So is that actually running? If it isn't is there an error logged when you try to start it?

yon 0

@stephenw10 said in some services show can't start:

Ok, as I stated previously, the service that pfSense is checking for is php_wg. So is that actually running? If it isn't is there an error logged when you try to start it?

how i check the php_wg run? When I use webgui, I can sometimes start the wiregaurd service. But after a while, it will automatically show that the wiregaurd service is stopped. But in fact, wiregaurd is running normally.

yon 0

@stephenw10

yon 0

@stephenw10

Oct 15 03:35:24	check_reload_status	507	Syncing firewall
Oct 15 03:35:24	php-fpm	78457	/pkg_edit.php: Configuration Change:
Oct 15 03:32:00	sshguard	44701	Now monitoring attacks.
Oct 15 03:32:00	sshguard	39046	Exiting on signal.
Oct 15 03:25:11	xinetd	6828	Reconfigured: new=0 old=10 dropped=0 (services)
Oct 15 03:25:11	xinetd	6828	readjusting service 19007-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19006-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19005-udp
Oct 15 03:25:11	xinetd	6828	readjusting service 19005-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19004-udp
Oct 15 03:25:11	xinetd	6828	readjusting service 19004-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19003-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19002-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19001-tcp
Oct 15 03:25:11	xinetd	6828	readjusting service 19000-tcp
Oct 15 03:25:11	xinetd	6828	Swapping defaults
Oct 15 03:25:11	xinetd	6828	Starting reconfiguration
Oct 15 03:24:53	php-fpm	91277	/rc.dyndns.update: phpDynDNS: Not updating wg A record because the public IP address cannot be determined.
Oct 15 03:24:38	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was ''
Oct 15 03:24:38	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Default gateway setting frwg0 as default.
Oct 15 03:24:37	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary.
Oct 15 03:24:36	check_reload_status	507	Syncing firewall
Oct 15 03:24:36	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard).
Oct 15 03:24:36	check_reload_status	507	Syncing firewall
Oct 15 03:24:36	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard).
Oct 15 03:24:35	check_reload_status	507	Syncing firewall
Oct 15 03:24:35	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard).
Oct 15 03:24:34	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard).
Oct 15 03:24:34	check_reload_status	507	Syncing firewall
Oct 15 03:24:34	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s).
Oct 15 03:24:34	check_reload_status	507	Syncing firewall
Oct 15 03:24:33	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s).
Oct 15 03:24:30	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Error starting gateway monitor for UKWG
Oct 15 03:24:30	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/usr/local/bin/dpinger -S -r 0 -i UKWG -B 10.18.1.2 -p /var/run/dpinger_UKWG~10.18.1.2~10.18.1.1.pid -u /var/run/dpinger_UKWG~10.18.1.2~10.18.1.1.sock -C "/etc/rc.gateway_alarm" -d 1 -s 500 -l 2000 -t 60000 -A 1000 -D 600 -L 80 10.18.1.1 >/dev/null' returned exit code '1', the output was ''
Oct 15 03:24:29	php-fpm	8018	/rc.filter_configure_sync: dpinger: No dpinger session running for gateway FRVPN_VPNV4
Oct 15 03:24:29	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/ifconfig tun_wg3 inet6 'fe80::32ed:b7ff:fe85:93d3' -alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Oct 15 03:24:28	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary.
Oct 15 03:24:27	check_reload_status	507	Syncing firewall
Oct 15 03:24:27	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard).
Oct 15 03:24:27	check_reload_status	507	Syncing firewall
Oct 15 03:24:27	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard).
Oct 15 03:24:26	check_reload_status	507	Syncing firewall
Oct 15 03:24:26	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard).
Oct 15 03:24:25	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard).
Oct 15 03:24:25	check_reload_status	507	Syncing firewall
Oct 15 03:24:25	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s).
Oct 15 03:24:25	check_reload_status	507	Syncing firewall
Oct 15 03:24:24	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s).
Oct 15 03:24:22	check_reload_status	507	Reloading filter
Oct 15 03:24:22	check_reload_status	507	Restarting OpenVPN tunnels/interfaces
Oct 15 03:24:22	check_reload_status	507	Restarting IPsec tunnels
Oct 15 03:24:22	check_reload_status	507	updating dyndns WAN_PPPOE
Oct 15 03:24:22	rc.gateway_alarm	74567	>>> Gateway alarm: WAN_PPPOE (Addr:10.1.8.1 Alarm:1 RTT:2.320ms RTTsd:.061ms Loss:33%)
Oct 15 03:24:21	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Error starting gateway monitor for OPT7GW
Oct 15 03:24:21	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/usr/local/bin/dpinger -S -r 0 -i OPT7GW -B 10.17.2.2 -p /var/run/dpinger_OPT7GW~10.17.2.2~10.17.2.1.pid -u /var/run/dpinger_OPT7GW~10.17.2.2~10.17.2.1.sock -C "/etc/rc.gateway_alarm" -d 1 -s 500 -l 2000 -t 60000 -A 1000 -D 600 -L 80 10.17.2.1 >/dev/null' returned exit code '1', the output was ''
Oct 15 03:24:21	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/ifconfig tun_wg5 inet6 'fe80::f291:32ff:fe07:db47' -alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Oct 15 03:24:19	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary.
Oct 15 03:24:19	check_reload_status	507	Syncing firewall
Oct 15 03:24:18	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard).
Oct 15 03:24:18	check_reload_status	507	Syncing firewall
Oct 15 03:24:18	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard).
Oct 15 03:24:17	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard).
Oct 15 03:24:17	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard).
Oct 15 03:24:17	check_reload_status	507	Syncing firewall
Oct 15 03:24:16	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s).
Oct 15 03:24:16	check_reload_status	507	Syncing firewall
Oct 15 03:24:16	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s).
Oct 15 03:24:13	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Error starting gateway monitor for DEwgGW
Oct 15 03:24:13	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/usr/local/bin/dpinger -S -r 0 -i DEwgGW -B 10.11.0.2 -p /var/run/dpinger_DEwgGW~10.11.0.2~10.11.0.1.pid -u /var/run/dpinger_DEwgGW~10.11.0.2~10.11.0.1.sock -C "/etc/rc.gateway_alarm" -d 1 -s 500 -l 2000 -t 60000 -A 1000 -D 600 -L 80 10.11.0.1 >/dev/null' returned exit code '1', the output was ''
Oct 15 03:24:13	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/ifconfig tun_wg4 inet6 'fe80::a9b3:3fff:febe:d75a' -alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Oct 15 03:24:11	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary.

yon 0

Force pfsense to start wiregaurd, but it will stop automatically after a while

stephenw10

I would use:

[23.09-BETA][admin@6100-3.stevew.lan]/root: ps aux | grep php_wg
root    24313   0.0  0.5  69316 44980  -  Ss   21:45     0:00.03 php_wg: WireGuard service (php_wg)
root    59822   0.0  0.0  12752  2364  0  S+   21:46     0:00.00 grep php_wg

But it looks like you have an actual errors there.

It's unable to add the linklocal IPv6 address:

Oct 15 03:24:13	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/ifconfig tun_wg4 inet6 'fe80::a9b3:3fff:febe:d75a' -alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'

So that likely explains the issues you were seeing with FRR.
But also:

Oct 15 03:24:38	php_wg	22802	/usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was ''

If you run /sbin/route -n6 get 'default' dircetly does it return a valid default route?

Is that the last thing logged by the php_wg process before it stops?

Steve