SG-1100 Never Completes Boot--Stuck on Starting DNS Resolver

scolby33

Hello,

Right before the pandemic, I set up a new SG-1100 as the router at my apartment. Then, I went away for a weekend. That weekend became three months. At some point during this time, my roommate told me the WiFi was misbehaving, and I told her just to flip the power strip off and on again. The SG-1100 never came back up, with the green circle LED on solid and the black diamond LED blinking fast.

Now that I've returned, I got down to debugging this issue, and collected the attached log from the serial console, starting from pressing and releasing the reset button.

I'm no expert in this boot process, but there's two parts that stand out to me. The first:

Apr 13 15:01:53  syslogd: /var/log/ppp.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/poes.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/l2tps.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/ipsec.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/openvpn.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/dhcpd.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/relayd.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/filter.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/vpn.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/portalauth.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/dhcpd.log: Operation not supported by device
Apr 13 15:01:53  syslogd: /var/log/system.log: Operation not supported by device

I've seen similar errors elsewhere on the forum and I'm not sure if this is related to the problem or a red herring.

The second:

...
Starting PFLOG...done.
Setting up gateway monitors...done.
Setting up static routes...route: writing to routing socket: Network is unreachable
route: route has not been found
done.
Setting up DNSs...
Starting DNS Resolver...

...and then it hangs. I left it open while I was working today, and there was no progress after approximately 6 hours.

My hypothesis, although I have no evidence to back this up, is that my roommate caught the device at a bad time when pfBlocker was writing out an update to its blocklists, and one of these files got corrupted, preventing the resolver from starting properly.

What should I do about this? I imagine something involving one of those two "press any key to stop boot" prompts followed by either some commands to fix or investigate the broken DNS Resolver service, or overwriting fresh firmware and starting fresh.

I'm looking forward to any advice!

Rico

Starting fresh is pretty easy: https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html

-Rico

Gertjan

@scolby33 said in SG-1100 Never Completes Boot--Stuck on Starting DNS Resolver:

and I told her just to flip the power strip off and on again. The SG-1100 never came back up, with the green circle LED on solid and the black diamond LED blinking fast.

That's as loading up a pistol with the 'file system dirty' price and play the Russ roulette game with it.
The behaviour of any PC isn't very different.

Because these routers have filer systems o disks that are writeable, a video was even made to explain what needs to be done
How to Run a pfSense Software File System Check

What needed to be done is : restart the wifi device ? (SG-1100 has no wifi)
If no ok : using another device using wifi ?
If no ok : use another device using cable (RJ45).
If no ok - attribute a static IP mask to your device - and ping the 'router' == pfSense - is it answering ? Accessing the GUI works ?
If no ok : use the console access .... (== using a special must-have cable) and check, like can you ping outside == to 8.8.8.8 for example
If not ok, => NO WAN access ... call the ISP.
If not ok, (console access doesn't work) well, ok, your router has crashed and became unreachable, and a power reset is needed. You might be needing the video mentioned before.
It's also advisable to have a config back - in case of.

The list wouldn't be complete with this one : you are not the only one who is controlling the power switch.
You re re gain most of the control by using an UPS, such a device is not a luxury.

Btw : I'm very aware that some if not all these steps, even after 50 years of existence, and the fact billion are using all these steps without actually knowing it, it still looks like rocket science. This will change slowly. As people do (have to) check their tire pressure, breaking and directional lights , doors closed, outside mirrors ok, gas in the tank, hand break removed, etc etc before you even start the car (what ? you say they don't ?).

edit : my check list isn't just for pfSense, but any device that connects to the net that has no build in screen / keyboard.
It could be your fridge, AP, house heater, Air conditioner, coffee machine, web cam, etc etc etc - as people tend tio buy these things by wagons at the time, my list becomes indispensable.

edit2 : if the device has no file system on writeable media, but is more or less ROM based, the the file system couldn't crash. Example : Ours ISP give us routers with minimal settings needed to work, the exception would be : the user and password codes. The rest is hardcoded by default, which means : less issues when the power was removed.

scolby33

@Gertjan I absolutely agree with you! Although it wasn't clear, I elided some earlier troubleshooting steps from my story. I was stuck helping a non-technical person via phone without any remote access--I had only intended to be away for the weekend and hadn't gotten around to setting up remote access and dynamic DNS yet. The power strip (in fact, it is a UPS) I had her toggle was connected to the modem, SG-1100, and the WAP, and this was my last-ditch effort to fix something. As you say, I lost my bet.

@Rico thanks for the pointer to the right place in the docs! I'll open a ticket with Netgate ASAP and get started.