SG-4860 crashing daily
-
@stephenw10 I found some more crash logs that I had sent to myself over Telegram. Hopefully these might shed some light.
-
Mmm, those are all different. That is looking more like a memory fault unfortunately.
Are you able to try a clean install of 22.05?
Steve
-
@stephenw10 This is currently the production firewall for this location. I purchased a XG-1537 last year and a stack of new switches to install but haven't scheduled a time to replace it all.
I'll try to reload the 4860 after everything stabilizes.
Last nights crash dump.
textdump.tar.0 -
Mmm, another similar crash but different panic. Again it doesn't point to any specific thing and looks increasingly like a hardware issue unfortunately.
Steve
-
@stephenw10 The device hadn't crashed in a few days, but this morning it has a PHP crash log as well.
[12-Aug-2022 00:42:00 UTC] PHP Warning: Static function mbereg_search() cannot be abstract in Unknown on line 0 -
Hmm, that looks different, more like it just ran out of memory.
That also ties in with this:
<6>pid 71216 (unbound), jid 0, uid 59: exited on signal 11If you check the monitoring graphs in Status > Monitoring do you see memory usage increasing with time?
-
@stephenw10 I checked the memory graph for a 2 day period with 5 min resolution and didn't see the free memory decrease except during the crashes.
I'll keep a watch for anything new.
-
Mmm, I agree, it doesn't look like it's exhausting the memory directly.
-
@stephenw10 I believe I have narrowed the issue down to the tailscale package. I noticed when I came back from vacation that the firewall had been up over 8 days w/o a crash.
Checking the logs showed that either PHP or PHP-CGI was exiting on signal 11 with a core dump, and the services section showed that tailscale wasn't running either.
On a hunch I started the tailscale service yesterday morning to see if a crash would happen. Sure enough, last night it crashed again.
Attached is the latest dump. textdump.tar.0
-
So you had disabled tailscale while you were away? Or it had stopped by itself and then crashed after you restarted it?
Steve
-
@stephenw10 tailscale had crashed apparently, but the connections it made we're still running so I didn't notice the service itself was down.
I restarted the service yesterday morning to see if it was the cause of the crashes, then this morning when I logged in, I saw the crash report.
-
Mmm, not familiar to me. Let me see if any one else has seen it....
