PfSense Crashed
-
Another failure mode is the text "ipsec_filter: m_pullup failed" on the console and then a hard hang, no crash or reboot. There seems to be something seriously wrong with this kernel.
/wj
-
I'm getting the same failure as well, but there is not auto reboot. The system just hangs after a few hours and I have to manually reset it using the hw reset switch
-
I'm guessing at this point we can be pretty confident something just went awfully, horribly wrong :) Thanks pfSense team for all your hard work, it's times like these that we appreciate how much goes into making a project like this happen!
-
Same here… from an Alix board, looking at the console, the system is locked in boot-looping, restarting at some ramdom points... :S
-
Ok, I post just to help someone in the case. The easiest way I found to recover pfSense to a working state was restoring a full backup from an image I made some time ago.
Just connect to the console, control+break the system while it is initializing, and run:/etc/rc.restore_full_backup /root/pfSense-full-backup-20130219-2240.tgz
(just change pfSense-full-backup-20130219-2240.tgz with the filename of the full backup).Then, just wait for the process to complete and when it's done run: reboot
Finally, you can restore the latest configuration from the webadmin interface.
Ciao,
Michele -
crashes here also and i guess no new snaps also being generated as builder seems down
-
I was also stuck in a panic loop on the latest snapshot. Disconnecting from external networks prevented random rebooting as the panics were on network events. I am rolling back at present to pfSense-Full-Update-2.1-BETA1-i386-20130420-1706 hoping it's stable.
A crash dump was successfully saved and has been submitted to the developers.
-
For mine, I had same panics on network activity. It would not stay up long enough to download a previous firmware version and apply it before it crashed.
It would also not recover from what I thought I had as a full backup.
I finally managed it by a) downloading the firmware to a webserver nearby. I also had to boot the pfsense box with option 2 disabling ACPI.
I then specified to upgrade firmware by URL and entered the nearby server. Back online now.
There may have been an easier way of doing it but thats what worked for me.
pfsense still rocks!
-
-
found myself today with Alix crashing after the upgrade
for those who run 'embedded', just switch to the second slice, which should hold the previous build installed.
and yeah… pfSense rocks ;D
-
Yeah… it rocks... most of the time. Yesterday's build though sucks rocks.
What happened? Must have been something pretty fundamental, but nothing significant shows up in commit history!
-
Yeah… it rocks... most of the time. Yesterday's build though sucks rocks.
What happened? Must have been something pretty fundamental, but nothing significant shows up in commit history!
While I certainly share your frustration as many of us are running 2.1 nightlies on production systems for driver reasons, I think saying it "sucks rocks" is unfair. These are nightlies we're running. They're not even alphas (yes, I know they're labeled "beta" but they're not betas in the traditional sense of a beta that's been slightly tested and expected to mostly work - they're automatic nightly builds). Nightlies can be broken in very fundamental ways, it's the nature of software development.
The pfSense team is doing great work, and I'm especially appreciative of the members of the team who are friendly and helpful on these forums to all of us running pfSense. When 2.1 is out, or even an actual beta/RC, then we'll have builds we can use and not run the risk of them fundamentally trashing the whole system. Until then, they're nightlies, for better or for worse. I'm glad the team's made them available for widespread community testing.
-
Ok, I post just to help someone in the case. The easiest way I found to recover pfSense to a working state was restoring a full backup from an image I made some time ago.
Just connect to the console, control+break the system while it is initializing, and run:/etc/rc.restore_full_backup /root/pfSense-full-backup-20130219-2240.tgz
(just change pfSense-full-backup-20130219-2240.tgz with the filename of the full backup).Then, just wait for the process to complete and when it's done run: reboot
Since I always check the "make full backup" box when upgrading, that was an almost painless process to get things back working.
Almost painless, because I didn't know how, until I read this here, but of course, my net was down, so…...and also because my box needs to be opened up and a ribbon connector must be plugged onto the MB for me to hook up screen and keyboard, because that's just an "for install only" afterthought. Time to make a mod to the case, maybe.
In any case, things are up and running again, as this post shows. Will someone deposit a message here when a new build is available that works?
Given the hoops I have to jump through when things are busted in this particular way, I'm not keen on installing random builds until I know this particular issue is fixed. -
While I certainly share your frustration as many of us are running 2.1 nightlies on production systems for driver reasons, I think saying it "sucks rocks" is unfair. These are nightlies we're running. They're not even alphas (yes, I know they're labeled "beta" but they're not betas in the traditional sense of a beta that's been slightly tested and expected to mostly work - they're automatic nightly builds). Nightlies can be broken in very fundamental ways, it's the nature of software development.
The pfSense team is doing great work, and I'm especially appreciative of the members of the team who are friendly and helpful on these forums to all of us running pfSense. When 2.1 is out, or even an actual beta/RC, then we'll have builds we can use and not run the risk of them fundamentally trashing the whole system. Until then, they're nightlies, for better or for worse. I'm glad the team's made them available for widespread community testing.
Lighten up guys. My comment was only an indication of surprise at the catastrophic outcome in applying this last nightly 'alpha', 'beta' or whatever. I had been lulled into complacence by the general quality of these releases and was taken off guard by this one.
One thing that would have facilitated recovery is a boot option allowing the selection and re-installation of one of the previously created backup tarballs. Sure its easily done manually, but it's difficult to look up how when you've lost all Internet connectivity.
-
yes, let's relax a bit everybody… we all know that all the pfSense guys are doing an AWESOME job, just let's us all remember that version 2.1 is still in the developing phase. For example, I run it at home, but in the office I run the stable 2.0.X version, and it's rock solid.
Since version 2.1 is still in developing, this can happen, we all have to remember it on each update, and have a backup plan, like trying to restore a previous config or full backup when things are working (because when things are not working could be too late) or have a stable backup disk/flash/microdrive to use in case of emergency or have two or more firewalls to update in different times, and so on.
Btw, let's wait for the pfSense staff to produce a stable snapshot with all the latest updates and fixes.
Michele
-
Since version 2.1 is still in developing, this can happen, we all have to remember it on each update, and have a backup plan, like trying to restore a previous config or full backup when things are working (because when things are not working could be too late) or have a stable backup disk/flash/microdrive to use in case of emergency or have two or more firewalls to update in different times, and so on.
No I totally agree that's all I meant, in a very light-hearted way :D
-
Since version 2.1 is still in developing, this can happen, we all have to remember it on each update, and have a backup plan, like trying to restore a previous config or full backup when things are working (because when things are not working could be too late) or have a stable backup disk/flash/microdrive to use in case of emergency or have two or more firewalls to update in different times, and so on.
Fully understood, but also one of the reasons why it would be great if the non-embedded version of pfSense could also have two slices, or a recovery partition that one can boot into to restore a previous backup, maybe even automatically if a system crashes more than X times within timespan t.
Otherwise, particularly if a unit is in a remote location, it can be a true PITA to even restore a backup that does exist.
-
I just upgraded 3 instances to the Thursday build, and I've started experiencing this as well.
Theyre all configured differently, and the only things in common are
- OpenVPN client export
- iPerf
- They have OpenVPN configured (but disabled on one of them)
It seems that messing with the firmware upgrade page is a sure-fire way to trigger a crash. One of them seems stable as long as I dont log into it.
The problem only started happening after a few hours of working on (2 of) the boxes, then all of a sudden all 3 started acting unstable. I dont think its just the webGUI, I had a number of crashes while launching daemons from SSH as well. Right now i have a ping -t running against one and I have ceased attempting to log into the GUI, and it seems stable.
Not sure how helpful that is.
-
Um, me again…
I think anybody chosing to run the DEVELOPMENT version should be doing so in the strict knowledge and understanding that there could be issues. I do.
Ensure your backup/DR/contingency plan is robust, thats basic. Carry out own testing before releasing to a production or working rig, that's basic as well.
I dont think its right however "light-hearted" it seems to criticise the project for a broken DEVELOPMENT version. But there we go. Thats my opinion.
Constructively - would the developers perhaps consider PULLING the broken release to negate more people being affected?
-
In my case it doesn't reboot: it simply freezes and refuses to accept any input. Strangely, it was working fine yesterday. Not that it matters much: I've been using it on my secondary (backup) firewall to provide IPv6 connectivity. My primary one is still on 2.0.X.
It probably would be an idea to pull the broken release.
