Strange behaviour after power failure
-
Yesterday, we had a power failure in our area. Normally this shouldn't be a problem, as we have a UPS to secure our servers, but fate decided otherwise, the batteries in the UPS where too far gone...
So on to the actual problem:
First of all, pfsense is installed vitually, on ESXi. Not ideal, I know. When the power returned, the server started booting up again, also the rest of the VM's.
After this bootup process, I tried to login to our Pfsense dashboard, but nothing seemed to be there. Pfsense was started completely, but I didn't seem to get an outside IP (WAN)
I tried pinging the internal IP of PFsense, also nothing ... After searching for hours, I had to reassign the interfaces to get everything back in working order... (command line access, then option 1) But the strange thing, these interfaces still where correctly assigned before I did this.
Is this normal? I have restarted the ESXi server before, without any problems.
Any insight welcome off course!
-
Virtual, or not, pfSense as a software doesn't write write everything right away to the disk.
The OS, FreeBSD, can also introduce a delay.
And because it's a VM, the host OS, who actually accesses the virtual file system, also can delay.
So, even more changes the file system is corrupted.The result is that some config files can become corrupt or invalid.
When you suspect an issue, and a power loss is an issue, upon starting of a VM, you open the VM console right away, forget about high level GUI access. All booting messages, shown in the console that are out of the ordinary become important now. The good new : the console access is one click away.
Just to be sure, Netgate made a video about the subject : How to Run a pfSense Software File System Check.
"Restarting" is nothing more as a clean system shut-down. This guarantees a good system startup.
A power loss looks like a part of a restart start-up process, bu it's more like throwing an iron bar into a running motor to have it stopped.
When using an VM, even more reason to use and validate that the UPS is in a good state, typically they last for 2 to3 years. -
How were/are the interfaces configured? All static?
Any errors in the boot or system logs?
Steve
-
@stephenw10 said in Strange behaviour after power failure:
How were/are the interfaces configured? All static?
Any errors in the boot or system logs?
Steve
WAN is set as dhcp (isp requirement) LAN/DMZ are static. OS Boot log doesn't make me any wiser anymore, as it seems to be allready overwritten with a normal boot.
Like @Gertjan replied, a bad system shutdown could do some strange things. Replacement batteries are allready arrived so I hope this was a one-off experience ....
