Device shutting down every night
-
Hello,
we have an APU pcengines device shutting down everynight and coming back on boot after plug and unplug the power. I'm wondering if it's a power supply issue or system issue. i have managed to get some logs during that time.
Any comments appreciated !!!
Jul 16 23:35:28 kernel MCA: Bank 2, Status 0xd541c0d00010011a
Jul 16 23:35:28 kernel MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000
Jul 16 23:35:28 kernel MCA: Vendor "AuthenticAMD", ID 0x730f01, APIC ID 0
Jul 16 23:35:28 kernel MCA: CPU 0 COR EN OVER
Jul 16 23:35:28 kernel GCACHE L2 RD error
Jul 16 23:35:28 kernel MCA: Address 0x14746070
Jul 16 23:40:29 kernel MCA: Bank 2, Status 0xd541c0d30010011a
Jul 16 23:40:29 kernel MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000
Jul 16 23:40:29 kernel MCA: Vendor "AuthenticAMD", ID 0x730f01, APIC ID 0
Jul 16 23:40:29 kernel MCA: CPU 0 COR EN OVER GCACHE L2 RD error
Jul 16 23:40:29 kernel MCA: Address 0x18e46070
Jul 16 23:45:29 kernel MCA: Bank 2, Status 0xd541c0d30010011a
Jul 16 23:45:29 kernel MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000
Jul 16 23:45:29 kernel MCA: Vendor "AuthenticAMD", ID 0x730f01, APIC ID 0
Jul 16 23:45:29 kernel MCA: CPU 0 COR EN OVER GCACHE L2 RD error
Jul 16 23:45:29 kernel MCA: Address 0x14746070
Jul 17 10:07:23 syslogd kernel boot file is /boot/kernel/kernel
Jul 17 10:07:23 kernel ---<<BOOT>>---
Jul 17 10:07:23 kernel Copyright (c) 1992-2023 The FreeBSD Project.
Jul 17 10:07:23 kernel Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Jul 17 10:07:23 kernel The Regents of the University of California. All rights reserved.
Jul 17 10:07:23 kernel FreeBSD is a registered trademark of The FreeBSD Foundation.
Jul 17 10:07:23 kernel FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec 6 20:45:47 UTC 2023
Jul 17 10:07:23 kernel root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/FreeBSD-src-RELENG_2_7_2/amd64.amd64/sys/pfSense amd64
Jul 17 10:07:23 kernel FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152)
Jul 17 10:07:23 kernel VT(vga): resolution 640x480
Jul 17 10:07:23 kernel CPU: AMD GX-412TC SOC (998.17-MHz K8-class CPU)
Jul 17 10:07:23 kernel Origin="AuthenticAMD" Id=0x730f01 Family=0x16 Model=0x30 Stepping=1 -
That looks like it was (already) rebooting around 23:40:29 ... You should go back in the logs to see when this rebooting started to happen.
And then everything went silent at this moment "Jul 16 23:45:29 kernel MCA: Address 0x14746070".I already feel sorry for the state of the "hard disk" in your system. The file sytem can really take a shoot at non user initiated reboots moments ( which eventually leads to a less stable device : more reboot / crashes etc).
You better check op with the Netgate Youtube video : "Check and Repair pfSense Filesystem".Then the system was started at July 17, 10:07 in the morning.
-
@Gertjan if from 23:35 start rebooting and this happen again on 23:40 (as you stated) and again 23:45 , all those logs seem to me failed boot attempts because of power. (that's why we have silent logs in between).
what you say about that ?
-
MCA errors like that at almost exclusively a hardware issue. Especially if it just started happening and not after an upgrade or after enabling something new (which might access new hardware).
Steve
-
@stephenw10 Hello Steph,
the only update performed is pfsense 2.6 to 2.7 and then 2.7.2 (because the device came with 2.6 from the supplier).
-
And that wasn't immediately before the MCA errors started?
-
@Bambos said in Device shutting down every night:
because of power
If power is an issue, get an UPS.
-
@stephenw10 no, the initial setup was 12 days ago.
-
@Gertjan there is a UPS, the power disconnection might be after the UPS , on the socket of the APU device power supply 12V adapter.
-
Ah ... maybe it's time to lock the equipment up into a sealed box, with padlock and all of that
pfSense, like any other device with a big OS, file systems, etc, doesn't like it at all (understatement) when the power gets removed like that.
-
Check the BIOS/Coreboot version. It's probably waaay out of date and that's about the only thing you can do.