Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Random crash report in notices.inc and wg.inc

    Scheduled Pinned Locked Moved General pfSense Questions
    30 Posts 3 Posters 751 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      whinis @stephenw10
      last edited by

      I did see the exact same error with this config on bare metal.
      Here is the full php error log for the latest crashes it seems

      Crash report begins.  Anonymous machine information:

amd64
14.0-CURRENT
FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec  6 20:45:47 UTC 2023     root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/F

Crash report details:

PHP Errors:
[02-May-2025 19:45:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /usr/local/www/xmlrpc.php(32): require_once('/etc/inc/config...')
#12 {main}
  thrown in /etc/inc/notices.inc on line 135
[02-May-2025 19:45:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(1154): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#2 [internal function]: pfSense_clear_globals()
#3 {main}
  thrown in /etc/inc/notices.inc on line 135
[02-May-2025 21:10:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /usr/local/www/xmlrpc.php(32): require_once('/etc/inc/config...')
#12 {main}
  thrown in /etc/inc/notices.inc on line 135
[02-May-2025 21:10:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(1154): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#2 [internal function]: pfSense_clear_globals()
#3 {main}
  thrown in /etc/inc/notices.inc on line 135
[03-May-2025 04:03:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
#2 /etc/inc/config.gui.inc(53): parse_config()
#3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
#4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
#5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
#11 /usr/local/www/xmlrpc.php(32): require_once('/etc/inc/config...')
#12 {main}
  thrown in /etc/inc/notices.inc on line 135
[03-May-2025 04:03:00 US/Eastern] PHP Fatal error:  Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
Stack trace:
#0 /etc/inc/notices.inc(135): fopen('', 'w')
#1 /etc/inc/config.lib.inc(1154): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#2 [internal function]: pfSense_clear_globals()
#3 {main}
  thrown in /etc/inc/notices.inc on line 135



No FreeBSD crash data found.
			```
      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Is there anything in the system log just before that though? Maybe something common to all the crashes? For some reason there it's unable to open the filesystem which is triggering those php errors but they are a symptom of some other issue.

        W 1 Reply Last reply Reply Quote 0
        • W
          whinis @stephenw10
          last edited by

          Nothing that jumps out. From the system log there is only one pattern I see

           /getstats.php: No config.xml found, attempting last known config restore.
May  2 13:00:00 pfsense php-fpm[68498]: /getstats.php: PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
May  2 13:00:00 pfsense php-fpm[68498]: Stack trace:
May  2 13:00:00 pfsense php-fpm[68498]: #0 /etc/inc/notices.inc(135): fopen('', 'w')
May  2 13:00:00 pfsense php-fpm[68498]: #1 /etc/inc/config.lib.inc(95): file_notice('config.xml', 'No config.xml f...', 'pfSenseConfigur...', '')
May  2 13:00:00 pfsense php-fpm[68498]: #2 /etc/inc/config.gui.inc(53): parse_config()
May  2 13:00:00 pfsense php-fpm[68498]: #3 /etc/inc/auth.inc(34): require_once('/etc/inc/config...')
May  2 13:00:00 pfsense php-fpm[68498]: #4 /etc/inc/openvpn.inc(36): require_once('/etc/inc/auth.i...')
May  2 13:00:00 pfsense php-fpm[68498]: #5 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
May  2 13:00:00 pfsense php-fpm[68498]: #6 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
May  2 13:00:00 pfsense php-fpm[68498]: #7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
May  2 13:00:00 pfsense php-fpm[68498]: #8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
May  2 13:00:00 pfsense php-fpm[68498]: #9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
May  2 13:00:00 pfsense php-fpm[68498]: #10 /etc/inc/config.inc(37): require_once('/etc/inc/notice...')
May  2 13:00:00 pfsense php-fpm[68498]: #11 /usr/local/www/includes/functions.inc.php(27): require_once('/etc/inc/config...')
May  2 13:00:00 pfsense php-fpm[68498]: #12 /usr/local/www/getstats.php(38): include_once('/usr/local/www/...')
May  2 13:00:00 pfsense php-fpm[68498]: #13 {main}
May  2 13:00:00 pfsense php-fpm[68498]:   thrown

          Some thing attempts to load config.xml, its not found and then the stack trace.

          May  2 16:18:00 pfsense php-fpm[29876]: /xmlrpc.php: No config.xml found, attempting last known config restore.
May  2 16:18:00 pfsense php-fpm[29876]: /xmlrpc.php: PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
May  2 16:18:00 pfsense php-fpm[29876]: Stack trace:
May  2 16:18:00 pfsense php-fpm[29876]: #0 /etc/inc/notices.inc(135): fopen('', 'w')

          May  2 19:45:00 pfsense php-fpm[53488]: /xmlrpc.php: No config.xml found, attempting last known config restore.
May  2 19:45:00 pfsense php-fpm[53488]: /xmlrpc.php: PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
May  2 19:45:00 pfsense php-fpm[53488]: Stack trace:

          Here is one with several hours and no obvious error

          May  3 00:47:22 pfsense kernel: arp: 192.168.0.64 moved from xx:xx:xx:xx:xx:xx to xx:xx:xx:xx:xx:xx on bridge0
May  3 01:47:52 pfsense kernel: arp: 192.168.0.64 moved from xx:xx:xx:xx:xx:xx to xx:xx:xx:xx:xx:xx on bridge0
May  3 01:54:00 pfsense sshguard[91998]: Exiting on signal.
May  3 01:54:00 pfsense sshguard[50666]: Now monitoring attacks.
May  3 02:15:00 pfsense sshguard[50666]: Exiting on signal.
May  3 02:15:00 pfsense sshguard[79301]: Now monitoring attacks.
May  3 02:38:00 pfsense sshguard[79301]: Exiting on signal.
May  3 02:38:00 pfsense sshguard[4925]: Now monitoring attacks.
May  3 02:48:22 pfsense kernel: arp: 192.168.0.64 moved from xx:xx:xx:xx:xx:xx to xx:xx:xx:xx:xx:xx on bridge0
May  3 02:59:00 pfsense sshguard[4925]: Exiting on signal.
May  3 02:59:00 pfsense sshguard[45638]: Now monitoring attacks.
May  3 03:01:00 pfsense php-cgi[52259]: rc.periodic: New alert found: The following CA/Certificate entries are expiring:
May  3 03:01:00 pfsense php-cgi[52259]: Certificate: webConfigurator default (62294113a9989) (62294113a9989): Expiring soon, in 21 days
May  3 03:03:00 pfsense sshguard[45638]: Exiting on signal.
May  3 03:03:00 pfsense sshguard[54381]: Now monitoring attacks.
May  3 03:48:53 pfsense kernel: arp: 192.168.0.64 moved from xx:xx:xx:xx:xx:xx to xx:xx:xx:xx:xx:xx on bridge0
May  3 04:03:00 pfsense php-fpm[53488]: /xmlrpc.php: No config.xml found, attempting last known config restore.
May  3 04:03:00 pfsense php-fpm[53488]: /xmlrpc.php: PHP ERROR: Type: 1, File: /etc/inc/notices.inc, Line: 135, Message: Uncaught ValueError: Path cannot be empty in /etc/inc/notices.inc:135
May  3 04:03:00 pfsense php-fpm[53488]: Stack trace:
May  3 04:03:00 pfsense php-fpm[53488]: #0 /etc/inc/notices.inc(135): fopen('', 'w')
          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Hmm, that's very odd. What hypervisor are you using?

            W GPz1100G 2 Replies Last reply Reply Quote 0
            • W
              whinis @stephenw10
              last edited by

              Proxmox

              1 Reply Last reply Reply Quote 0
              • GPz1100G
                GPz1100 @stephenw10
                last edited by

                @stephenw10 If some part of the config (config.xml) was causing this issue. How would one even begin to troubleshoot that?

                Restore portions of it to a new install until crash is triggered or is there a better way?

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Nothing should ever result in the config file being inaccessible. It looks like some filesystem or drive issue though because in addition to not being able to open the config file the PHP error is being unable to open the notices file to post an alert for it. 🤔

                  W 1 Reply Last reply Reply Quote 0
                  • W
                    whinis @stephenw10
                    last edited by

                    Just to be proactive, the SSD its on is at 30% wearout, I moved it to another ssd in the VM and still same issue. I checked the logs on the VM host (proxmox) and the sys logs of pfsense and no obvious errors in either.

                    W 1 Reply Last reply Reply Quote 0
                    • W
                      whinis @whinis
                      last edited by

                      And here is the error that occasionally occurs that causes the config to be reloaded

                      May  5 21:12:00 pfsense php_wg[93764]: /usr/local/pkg/wireguard/includes/wg_service.inc: XML error: no pfsense or pfsense object found!
May  5 21:12:00 pfsense php_wg[93764]: 
May  5 21:12:00 pfsense php_wg[93764]: /usr/local/pkg/wireguard/includes/wg_service.inc: pfSense is restoring the configuration /cf/conf/backup/config-1746449192.xml
May  5 21:12:00 pfsense php_wg[93764]: /usr/local/pkg/wireguard/includes/wg_service.inc: New alert found: pfSense is restoring the configuration /cf/conf/backup/config-1746449192.xml
May  5 21:12:00 pfsense php_wg[93764]: /usr/local/pkg/wireguard/includes/wg_service.inc: PHP ERROR: Type: 1, File: /usr/local/pkg/wireguard/includes/wg_globals.inc, Line: 197, Message: Uncaught Error: Cannot use a scalar value as an array in /usr/local/pkg/wireguard/includes/wg_globals.inc:197
May  5 21:12:00 pfsense php_wg[93764]: Stack trace:
May  5 21:12:00 pfsense php_wg[93764]: #0 /usr/local/pkg/wireguard/includes/wg_globals.inc(238): wg_init_config_arr(-1, Array)
May  5 21:12:00 pfsense php_wg[93764]: #1 /usr/local/pkg/wireguard/includes/wg.inc(345): wg_globals()
May  5 21:12:00 pfsense php_wg[93764]: #2 /usr/local/pkg/wireguard/includes/wg_service.inc(360): wg_resolve_endpoints()
May  5 21:12:00 pfsense php_wg[93764]: #3 /usr/local/pkg/wireguard/includes/wg_service.inc(294): wg_service_daemon()
May  5 21:12:00 pfsense php_wg[93764]: #4 /usr/local/pkg/wireguard/includes/wg_service.inc(60): wg_service_cli_start(false)
May  5 21:12:00 pfsense php_wg[93764]: #5 {main}
May  5 21:12:00 pfsense php_wg[93764]:   thrown
May  5 21:12:00 pfsense php_wg[93764]: /usr/local/pkg/wireguard/includes/wg_service.inc: New alert found: PHP ERROR: Type: 1, File: /usr/local/pkg/wireguard/includes/wg_globals.inc, Line: 197, Message: Uncaught Error: Cannot use a scalar value as an array in /usr/local/pkg/wireguard/includes/wg_globals.inc:197
May  5 21:12:00 pfsense php_wg[93764]: Stack trace:
May  5 21:12:00 pfsense php_wg[93764]: #0 /usr/local/pkg/wireguard/includes/wg_globals.inc(238): wg_init_config_arr(-1, Array)
May  5 21:12:00 pfsense php_wg[93764]: #1 /usr/local/pkg/wireguard/includes/wg.inc(345): wg_globals()
May  5 21:12:00 pfsense php_wg[93764]: #2 /usr/local/pkg/wireguard/includes/wg_service.inc(360): wg_resolve_endpoints()
May  5 21:12:00 pfsense php_wg[93764]: #3 /usr/local/pkg/wireguard/includes/wg_service.inc(294): wg_service_daemon()
May  5 21:12:00 pfsense php_wg[93764]: #4 /usr/local/pkg/wireguard/includes/wg_service.inc(60): wg_service_cli_start(false)
May  5 21:12:00 pfsense php_wg[93764]: #5 {main}
May  5 21:12:00 pfsense php_wg[93764]:   thrown
                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Hmm, you've only seen this in 2.7.2?

                        W 1 Reply Last reply Reply Quote 0
                        • W
                          whinis @stephenw10
                          last edited by

                          As far as I know. This config I think used to be 23. Something and then I rebuilt it once I stopped using commercial for my home.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Are you able to test 2.8-Beta?

                            W 1 Reply Last reply Reply Quote 0
                            • W
                              whinis @stephenw10
                              last edited by

                              I can if I have a guide of how to update to it

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                You just have to select it in the repo branch in System > Update.

                                W 1 Reply Last reply Reply Quote 0
                                • W
                                  whinis @stephenw10
                                  last edited by

                                  Cool, ill try this tonight after the work day ends. Joy of working from home.

                                  W 1 Reply Last reply Reply Quote 1
                                  • W
                                    whinis @whinis
                                    last edited by

                                    So, we are now a few days into the 2.8-Beta version and no more crashes have happened. Hopefully this is somewhat helpful for you but unsure if it is.

                                    1 Reply Last reply Reply Quote 1
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Well if anything changed there it's probably something low level that the OS updates could have addressed.

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.