Troubleshooting repeated SG-3100 lockups

[tuser11]

We have a SG-3100 that keeps locking up. It's happening every few weeks, 4 times now in an environment where we weren't having issues more than maybe once a year. During the lockup, I can't get data via usb console. I have remote syslog data up until the actual event. I can't tell from the logs what happened. Here's what I've gathered:

• Running 23.05.1-RELEASE (arm)
• Avg temp 79c
• Avg ram 48%
• Disk 24%
• Load average 6.12, 4.60, 4.37
• Power adapter changed last month
• There were a bunch of BLOCK events logged before it stopped logging. Maybe 1-2 every couple of seconds.
• unbound was restarted almost a minute before it went offline. But based on logs, unbound seems to always be restarting. Sometimes more than once in a single minute.
• No warning or error messages were logged before it went offline.

Any ideas on how to resolve this?

[SteveITS]

@tuser11 said in Troubleshooting repeated SG-3100 lockups:

unbound seems to always be restarting. Sometimes more than once in a single minute

If the option to register DHCP leases in DNS is checked, Unbound will restart at every lease renewal. Doesn't cause lockups though, just brief DNS slowness/outages.

"Block" as in disk, or firewall logs? You might check https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html.

[tuser11]

@SteveITS sorry, BLOCK as in IP blocks.

Can't reach gateway on any interface even with putting a static IP on my laptop. Remote syslog no longer receiving messages from firewall. USB console connection just shows a blank screen and key inputs are unresponsive. Cold reboot is only thing I was able to make it work again.

The problem started with a old SG-3100 that was in production for years. We moved to a cold backup after problems started after updating to v23.01. Current setup is running off a cold spare that has been on shelf offline for years and has only been in production since April 2023 and receiving all updates since then.

[stephenw10]

Yup, try disabling DHCP leases in DNS as a test. That's a high loading and that's running quite hot as a result.

[rcoleman-netgate]

@tuser11 Let's keep this to a single thread, please.

Locking this one down. Please use SG3100 keeps locking up after latest update