Hard reset & console reset not working any other options?

[beardedguy]

My 2100 Netgate stopped working, I tried a hard reset, lights never turn red, slow blue flashing right triangle light after 30 sec.

tried a console reset which comes up with the following error, any thoughts on what i can do about this?

Enter an option: 4

Fatal error: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /etc/rc.initial.setlanip(45): require_once('/etc/inc/config...')
#12 {main}
thrown in /etc/inc/notices.inc on line 135
PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /etc/rc.initial.setlanip(45): require_once('/etc/inc/config...')
#12 {main}
thrown
Fatal error: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(1168): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#2 [internal function]: pfSense_clear_globals()
#3 {main}
thrown in /etc/inc/notices.inc on line 135

Fatal error: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /etc/rc.banner(27): require_once('/etc/inc/config...')
#12 {main}
thrown in /etc/inc/notices.inc on line 135
PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /etc/rc.banner(27): require_once('/etc/inc/config...')
#12 {main}
thrown
Fatal error: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(1168): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#2 [internal function]: pfSense_clear_globals()
#3 {main}
thrown in /etc/inc/notices.inc on line 135

0. Logout (SSH only) 9) pfTop
1. Assign Interfaces 10) Filter Logs
2. Set interface(s) IP address 11) Restart webConfigurator
3. Reset webConfigurator password 12) PHP shell + Netgate pfSense Plus tools
4. Reset to factory defaults 13) Update from console
5. Reboot system 14) Enable Secure Shell (sshd)
6. Halt system 15) Restore recent configuration
7. Ping host 16) Restart PHP-FPM
8. Shell

Enter an option:

[stephenw10]

Which pfSense version are you running?

Were you installing packages or trying to upgrade when this happened?

Steve

[Gertjan]

@beardedguy said in Hard reset & console reset not working any other options?:

#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')

This is your pfSense telling you that it can't write out the main config file.
The file that contains ALL pfSense settings. A bit like "can't write the registry" on a Windows PC : very bad news.
Hard resetting should be one of the latest things you should try right now.
Using the console, or better, SSH (no, even better : use SFTP == some sort of FTP over SSH) to retrieve immediately a most recent pfSense config file, you'll find the last 100 of them in /cf/conf/backup/

And while you have the console access open (or SSH), now figure out what's up.
Most probable reason : no more disk space left ? And if low to very low : who has eaten all this pace ? Are you using any pfSense packages known to do this ?
Another reason : the drive is flagged as 'dirty' and should be 'repaired' : youtube netgate file system. (on a dirty drive pfSEnse can't write - and not being able to write is the error showing up).

[beardedguy]

@stephenw10
no updating, did have a power failure during day when i was out.

[beardedguy]

@Gertjan
I have copies for the config files. Hard resetting did not work. going to try to following youtube video you provided.

[beardedguy]

@beardedguy
I have the latest firmware, not sure of version, was downloaded within system and installed 3 weeks ago apx.

[stephenw10]

If you are running ZFS you can roll back to the auto-generated BE created at upgrade and then upgrade again.

[beardedguy]

@stephenw10
is there a process i can follow? thx.

additional info...

fsck -y /

** /dev/ufsid/6172db556b40a343 (NO WRITE)
** SU+J Recovering /dev/ufsid/6172db556b40a343

USE JOURNAL? no

Skipping journal, falling through to full fsck
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
UNREF FILE I=92974 OWNER=root MODE=100666
SIZE=0 MTIME=Jul 17 09:06 2024

CLEAR? no

** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? no

SUMMARY INFORMATION BAD
SALVAGE? no

BLK(S) MISSING IN BIT MAPS
SALVAGE? no

59367 files, 1015804 used, 807162 free (8402 frags, 99845 blocks, 0.5% fragmentation)

[stephenw10]

@beardedguy said in Hard reset & console reset not working any other options?:

** /dev/ufsid/6172db556b40a343 (NO WRITE)

That's not expected. Does it show that every time you run fsck?

That ufsid implies you're running UFS not ZFS so there are no snapshots.

[beardedguy]

@stephenw10
it shows that every time, i run fsck. expect it is UFS.

[stephenw10]

Hmm, how old is that 2100 is it a max? It's possible the emmc has gone read only if it's booting from that.

Try booting normally then drop to the shell and create a file. If there are no errors, reboot and see if it's still there.

Ultimately you can try reinstalling from the recovery image.

[beardedguy]

@stephenw10
the 2100 is apx 1.5 yrs old and is not a max.
I'm not familiar with the shell commands, could you let me know which commends to enter or provide a link to shell commands? unable to find what I'm looking for. sry

[SteveITS]

@beardedguy Something like:

ls -l /myfile.txt
(should not already be there)

echo x > /myfile.txt

ls -l /myfile.txt
(should be there now)

[reboot]

ls -l /myfile.txt
(should still be there)

(delete with "rm /myfiletxt")

[beardedguy]

@SteveITS

Enter an option: ^A^ZYou have stopped jobs.
[1] + Suspended /etc/rc.initial

ls -l /myfile.txt

-rw-r--r-- 1 root wheel 2 Jul 17 10:39 /myfile.txt

[stephenw10]

Use menu option 8 to hit the shell. That should still work.

Then run: touch test.txt

If no error is shown check it's actually there and the date is correct with ls -ls

Reboot and check it's still there.

[beardedguy]

@stephenw10
[23.09.1-RELEASE][root@]/root: touch test.txt
[23.09.1-RELEASE][root@]/root: ls -ls
total 12352
4 -rw-r--r-- 2 root wheel 1011 Dec 6 2023 .cshrc
0 -rw-r--r-- 1 root wheel 0 Jul 16 14:34 .hushlogin
4 -rw-r--r-- 1 root wheel 68 Dec 6 2023 .k5login
4 -rw-r--r-- 1 root wheel 316 Dec 6 2023 .login
4 -rw-r--r-- 2 root wheel 1199 Jul 16 14:34 .profile
4 -rw------- 1 root wheel 1024 Jun 13 18:18 .rnd
4 -rw------- 1 root wheel 503 Jul 17 10:49 .sh_history
4 -rw-r--r-- 1 root wheel 2143 Jul 16 14:34 .shrc
4 -rw-r--r-- 1 root wheel 3410 Jul 16 14:34 .tcshrc
1408 -rw-r--r-- 1 root wheel 1381843 Mar 5 2023 packetcapture.cap
0 -rw-r--r-- 1 root wheel 0 Mar 5 2023 packetcapture.start
10912 -rw------- 1 root wheel 11128832 Dec 19 2022 pkg-static.core
0 -rw-r--r-- 1 root wheel 0 Jul 17 10:52 test.txt
[23.09.1-RELEASE][root@]/root:

reboot

Enter an option: ^A^ZYou have stopped jobs.
[1] + Suspended /etc/rc.initial

ls -l /test.txt

ls: /test.txt: No such file or directory

/myfile.txt

-sh: /myfile.txt: Permission denied

[SteveITS]

@beardedguy
Your commands show you created text.txt while you were in /root/ directory but looked for it in the top level: "ls -l /test.txt"...try "ls -l /root/text.txt" instead.

For reference, "ls" shows the directory LiSting, "-l" is "long" format meaning it shows file sizes and uses columns.

"/myfile.txt" by itself would try to run that as a command which is why you got the error.

[beardedguy]

@SteveITS good catch . . thanks.

[23.09.1-RELEASE][root@]/root: touch test.txt
[23.09.1-RELEASE][root@]/root: ls -ls
total 12352
4 -rw-r--r-- 2 root wheel 1011 Dec 6 2023 .cshrc
0 -rw-r--r-- 1 root wheel 0 Jul 16 14:34 .hushlogin
4 -rw-r--r-- 1 root wheel 68 Dec 6 2023 .k5login
4 -rw-r--r-- 1 root wheel 316 Dec 6 2023 .login
4 -rw-r--r-- 2 root wheel 1199 Jul 16 14:34 .profile
4 -rw------- 1 root wheel 1024 Jun 13 18:18 .rnd
4 -rw------- 1 root wheel 545 Jul 17 10:55 .sh_history
4 -rw-r--r-- 1 root wheel 2143 Jul 16 14:34 .shrc
4 -rw-r--r-- 1 root wheel 3410 Jul 16 14:34 .tcshrc
1408 -rw-r--r-- 1 root wheel 1381843 Mar 5 2023 packetcapture.cap
0 -rw-r--r-- 1 root wheel 0 Mar 5 2023 packetcapture.start
10912 -rw------- 1 root wheel 11128832 Dec 19 2022 pkg-static.core
0 -rw-r--r-- 1 root wheel 0 Jul 17 11:08 test.txt

reboot

Enter an option: ^A^ZYou have stopped jobs.
[1] + Suspended /etc/rc.initial

ls -l /root/text.txt

ls: /root/text.txt: No such file or directory

ls -l /root/test.txt

-rw-r--r-- 1 root wheel 0 Jul 17 11:08 /root/test.txt

[stephenw10]

Hmm, then I would try to recover to 24.03 dircetly and see if it shows errors installing. You should be able to recover the config at the same time.

[beardedguy]

@stephenw10
thanks, is there a process I can follow?