pfSense refuses to reboot

hunor

@stephenw10 Well, I can sanitize my config file and I can post it. Or when I have time I can replace the SSD in my system and "reinstall" it. It is my main router, so I need some time.

kravenul

I upgraded last night to [24.11-RELEASE] on my Netgate 2100, and I encounterred the exact same problem.

PHP ERROR: Type: 1, File: /etc/inc/util.inc, Line: 123, Message: Uncaught TypeError: is_process_running(): Argument #1 ($name) must be of type string, null given, called in /etc/inc/service-utils.inc on line 295 and defined in /etc/inc/util.inc:123
Stack trace:
#0 /etc/inc/service-utils.inc(295): is_process_running()
#1 /etc/inc/pkg-utils.inc(1277): is_service_running()
#2 /etc/inc/system.inc(2913): stop_packages()
#3 /etc/inc/system.inc(2869): system_reboot_cleanup()
#4 /usr/local/www/diag_reboot.php(102): system_reboot()
#5 {main}
thrown @ 2025-03-01 06:17:01

Is there anything I can do to find which package has a damaged config causing the php reboot script to crash?

Gertjan

@kravenul said in pfSense refuses to reboot:

and I encounterred the exact same problem

It didn't boot ?
You have this package installed :

and applied all 'Netgate' proposes patches ?

stephenw10

Also only affects non-admin users?

kravenul

@Gertjan I've confirmed System_Patches 2.2.19 is installed, but "No recommended patches for this version". I've also upgraded further to the beta version FreeBSD 15.0-CURRENT #0 plus-RELENG_25_03-n256448-5d69d8519d49 (25.03.b.20250204.0023) in an attempt and hopping to resolve the issue, but it persists. I'm using only official repository packages.

@stephenw10 I am sure the problem originated during the 24.03 to 24.11 upgrade. The diag_reboot.php menu item is the only function experiencing this failure. Executing sudo /etc/rc.initial.reboot in the shell yields the same crash error.

While I can reboot the SG-2100 via SSH or the GUI Command menu (Execute > reboot), a fresh install may be necessary. As this is my primary router, I'd prefer to avoid that if possible. Do you have any further troubleshooting suggestions? Thanks!"

stephenw10

Hmm, you should have System Patches pkg version 2.2.20_1. It has a long list of recommended patches.

But to be clear, if you're logged in as admin the webgui reboot works correctly?

kravenul

@stephenw10 I wish I could upgrade the System_Patches unfortunatelly my pfsense is not suggesting any upgrade. 2025-03-02 16_42_28-pfs.Packages.png

And no, the Reboot in the webgui doesn't work, is where I get that crash, but I can reboot if I execute a shell command as depicted below 2025-03-02 16_47_11-Clipboard.png

hunor

I couldn't figure out the cause; I had to reinstall everything.

The only "remaining" thing that I have with reboot is one problem. When I initiate a restart, sometimes it gets stuck during boot. It continues when I try to connect over serial and hit enter a few times.

I am not sure if that is some hardware/BIOS issue. Because when I have the serial connection on, it never happens. (I have a Topton N100 Mini PC)

kravenul

@hunor

I couldn't figure out the cause; I had to reinstall everything.

Regrettably even in my case using pfSense+ on an official Netgate SG-2100, I anticipate a fresh installation may be required. This is unexpected, as I have exclusively utilized the official, stable channel upgrades provided within the Update menu.

stephenw10

Hmm, you should see 2.2.20_1 in 24.11.

Do you see any available packages?

Try running pkg -d update at the command line and see if it shows any errors.

kravenul

@stephenw10 No available System_Patches in Package Manager. I have attached the command output for your review. It happened very fast, 5-7 seconds. One specific point has caught my attention, but I am not sure if it's important or it is an error:

Couldn't find host pfsense-plus-pkg-beta00.atx.netgate.com in the .netrc file; using defaults

However I believe there is a problem with the Letsencrypt certificate on the server. The curl is try fetching from https://pfsense-plus-pkg-beta.netgate.com/pfSense_plus-v25_03_aarch64-pfSense_plus_v25_03/data.pkg but the certificate is only for pfsense.org and other Subject Alternative Names except *.netgate.com

I am on version 25.03.b.20250204.0023 not 24.11

pkg -d update output

stephenw10

Ah, OK. In 25.03 there isn't yet a System Packages update since all the current updates are already present.

Hmm, I can't replicate this here.

What do you see from: pkg info -x pfsense ?

kravenul

@stephenw10 said in pfSense refuses to reboot:

What do you see from: pkg info -x pfsense ?

Here's the output:
pfSense-25.03.b.20250204.0023.1500029
pfSense-Status_Monitoring-php83-1.8_8
pfSense-base-25.03.b.20250204.0023
pfSense-boot-25.03.b.20250204.0023
pfSense-composer-deps-0.1
pfSense-default-config-serial-25.03.b.20250204.0023
pfSense-gnid-0.20
pfSense-kernel-pfSense-25.03.b.20250204.0023
pfSense-mim-25.03.b.20250204.0023
pfSense-pkg-Avahi-2.2_6
pfSense-pkg-Backup-0.6.3
pfSense-pkg-Cron-0.3.8_5
pfSense-pkg-RRD_Summary-2.2
pfSense-pkg-Service_Watchdog-1.8.7_4
pfSense-pkg-Status_Traffic_Totals-2.3.2_7
pfSense-pkg-System_Patches-2.2.19
pfSense-pkg-Telegraf-0.9_8
pfSense-pkg-WireGuard-0.2.9_3
pfSense-pkg-acme-0.9_1
pfSense-pkg-arping-1.2.2_6
pfSense-pkg-arpwatch-0.2.3
pfSense-pkg-haproxy-0.63_10
pfSense-pkg-iperf-3.0.5
pfSense-pkg-mailreport-3.6.4_5
pfSense-pkg-nmap-1.4.4_8
pfSense-pkg-ntopng-5.6.0_1
pfSense-pkg-openvpn-client-export-1.9.5
pfSense-pkg-openvpn-client-import-1.2_3
pfSense-pkg-pfBlockerNG-devel-3.2.1_22
pfSense-pkg-stunnel-5.50_13
pfSense-pkg-sudo-0.3_10
pfSense-pkg-suricata-7.0.8_1
pfSense-pkg-tftpd-0.1.3_6
pfSense-repo-25.03.b.20250204.0023
pfSense-repoc-20250131
pfSense-u-boot-1100-20220428
pfSense-u-boot-2100-20210930_1
pfSense-u-boot-env-20230123
pfSense-upgrade-1.2.31
php83-pfSense-module-0.101

Thank you!

stephenw10

Hmm, I wonder if you have some service defined in the config that isn't actually installed any longer.

What do you see in Status > Services? Anything showing an error there?

Gertjan

@stephenw10 said in pfSense refuses to reboot:

Ah, OK. In 25.03 there isn't yet a System Packages update since all the current updates are already present.

I'm on the latest

25.03-BETA (amd64)
built on Tue Feb 4 1:23:00 CET 2025

and the packet manager proposes me to downgrade the System_Patches package :

The daily mail notification (a non official script, you already know about ):

Notifications in this message: 1
================================

13:15:07 The following updates are available and can be installed using System > Package Manager:

System_Patches: 2.2.20_1 ==> 2.2.19 (downgrade)

The 'official' list with patches is empty of course, as they are, imho, all now part of 25.03 Beta.

I'll leave it at 2.2.20_1 for now.

stephenw10

Yes, I'll see if we can bump that to at least prevent the confusion there. But it can't be the cause the issue here.

kravenul

@stephenw10 No Errors under Services, everything seems to work, except the "Diagnostic" > "Reboot"

On top of that, since the upgrade to 24.11, then to the latest 25.03 beta, my SG-2100 became incredibly slow with a dramatic increase in load average (~8.0~10.0) and the extreme slowness of the WebGUI (10-15 seconds per page) indicate a severe resource contention

I understand the fact that it may have been propagated from the past upgrades, with a corrupted file, or simply a corrupted file system, which makes the issue very hard to replicate and troubleshoot.

Given the combination of issues and the difficulty in pinpointing the cause, my decision to wipe and reinstall is just a pragmatic one. Sometimes, a clean slate is the most efficient "solution"

Thank you all for you help!

Gertjan

@kravenul

First, check why it is slow.
You do this be lookup up what it is doing.
It's one click away : Diagnostics > System Activity

Example :

PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
 11 root        187 ki31     0B    32K RUN      0 135.7H 100.00% [idle{idle: cpu0}]
 11 root        187 ki31     0B    32K CPU1     1 132.5H  90.58% [idle{idle: cpu1}]
85893 root         68    0   150M    64M piperd   1   0:35   8.15% php-fpm: pool nginx (php-fpm){php-fpm}
84969 root         68    0   117M    60M accept   1   0:19   1.56% php-fpm: pool nginx (php-fpm)
65579 unbound      20    0   182M   158M kqread   0   9:16   0.39% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
65579 unbound      20    0   182M   158M kqread   1  11:33   0.29% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
....

The top resource intensive process are at the top : mine are "being idle" although I'm still routing 300+ Mbytes per sec right now.

Btw :

The dashboard page, deepening on what widgets you have activated, can be somewhat slow as you said yourself : a lot of resources are needed to update that page every x seconds.
Solution : close the page or look at another, more useful page like the log pages.

@kravenul said in pfSense refuses to reboot:

or simply a corrupted file system

It takes a minute to be sure about that, and deal with it. Go here and click play.