Random reboots

wgstarks

Is this the right graph? I don't see anything unusual here.

stephenw10

Indeed, I don't see anything spiking there but it did stop logging data entirely.
I would hook up a console to log any output there if you can. When there is nothing logged at all like that it does start to point towards a hardware issue though. That's an SG-3100 I assume?

Steve

wgstarks

@stephenw10
I can hookup console no problem. Not sure how to setup the logging though on my Mac.

I’m investigating buying replacement ram (rather than a long shutdown to run memtest) to test that. Just need to figure out what to buy.

If that doesn’t work I may just contact Netgate regarding warrantied replacement.

stephenw10

The RAM is non-replacable in the SG-3100.
If you have not re-installed it clean I would certainly try that first:
https://www.netgate.com/docs/pfsense/solutions/sg-3100/reinstall-pfsense.html

Logging the console output in OSX is not something I've ever tried to do though I don't imagine it's difficult with the right terminal client. I always use putty in Linux or Windows(if forced).

Steve

wgstarks

@stephenw10 said in Random reboots:

If you have not re-installed it clean I would certainly try that first:

Hasn't been more than a couple of months since the last clean install but worth a shot. I'll also work out the console logging to see if that turns up anything useful.

wgstarks

@stephenw10 said in Random reboots:

The RAM is non-replacable in the SG-3100.
If you have not re-installed it clean I would certainly try that first:
https://www.netgate.com/docs/pfsense/solutions/sg-3100/reinstall-pfsense.html

Logging the console output in OSX is not something I've ever tried to do though I don't imagine it's difficult with the right terminal client. I always use putty in Linux or Windows(if forced).

Steve

I've got console logging started, but never tried this before. Should I just be logging the shell window?

stephenw10

Yes, just log all the output there. If there is some drive error etc it will show on the console.

Steve

Derelict

Yeah Terminal by default will just keep a scrollback buffer.

But I use C-Kermit for serial console. Not sure if screen etc changes things.

If you're using screen, Ctrl-A H might be a friend.

wgstarks

@derelict
Actually, I already had the serial app installed. It will save everything to a text file.

wgstarks

No luck with the console messages. Had a reboot this morning. Nothing shown on the console other than the bootup.

Guess I'll give the clean install a try next?

wgstarks

Is it possible that a latency alarm could trigger this behavior?

Looking through all my logs I see a latency alarm occurred about the same time as the reboot.

Sep 14 02:59:58	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 30% dest_addr 8.8.8.8 bind_addr 24.211.135.107 identifier "WAN_DHCP "

Sep 14 02:59:51	syslogd		kernel boot file is /boot/kernel/kernel

Time stamp shows the reboot happened first though, so maybe the latency is being caused by the boot process???

Edit: Nevermind. Looks like I read the log entry wrong. Now that I’ve had a little sleep I see that it’s just showing the current settings.

stephenw10

Yes that is logged when apinger is started with whatever settings are configured.

Nothing in the logs, no error reports and nothing on the console looks like either a hardware issue or a power problem.

I assume nothing else is rebooting at that time though.

Steve

wgstarks

No. Nothing else rebooting. I doubt it’s an external power problem. The unit is powered via a rack mounted APC ups, so power should be relatively clean.

stephenw10

Over heating then maybe, the PSU brick or the CPU.

Steve

wgstarks

I believe this is the cpu temp. It seems to always run a little warm to me (between 64 and 70C). Not sure what a normal temp would be on the 3100.

No idea about power supply temps.

wgstarks

@stephenw10 said in Random reboots:

If you have not re-installed it clean I would certainly try that first:
https://www.netgate.com/docs/pfsense/solutions/sg-3100/reinstall-pfsense.html

Finally got a chance to do the re-install. Install failed at first. The USB port is extremely hard to insert a flash drive into and I didn't have it fully inserted. After a good hard push and another reboot I think it went ok. We'll see what happens.

stephenw10

That temperature looks normal for the 3100. Nothing I haven't seen here.

Steve

wgstarks

@stephenw10 said in Random reboots:

That temperature looks normal for the 3100. Nothing I haven't seen here.

Steve

Thanks. I might change my set points for temp alerts so that it’s not in the yellow as much.

dbinoj

I'm also having the same issue in a 3100. Random restarts. I'm suspecting temperature. It constantly runs at 70+. I've seen 77-78 at a point. Couldn't get to log historical data of temperature though.

wgstarks

@dbinoj said in Random reboots:

Couldn't get to log historical data of temperature though.

Yeah, I'd lover to know how to do this as well. The clean install seems to have fixed my issue though [fingers crossed].