Frequent Crashing (Page Fault) After Upgrade to 2.8.0 From Latest 2.7
-
So unfortunately, I have been unable to get this to fail......so I guess it's not as frequent as it was. I am out of time with this, so I am gonna have to deploy these.
At this point I am considering getting the supported version and migrate to PFSense Plus. Are there any stability benefits with doing this? Any chance this issue might be fully resolved by simply migrating to the PFSense Plus code? Is the migration process fairly reliable? Anything I need be mindful of? And do I need a support license for each device in an HA pair? Thanks for all the help with this. Sorry I couldn't get this to happen again.
-
2.8.1 and 25.07.1 are built on the same base so I would expect this to happen identically in Plus. There are 25.11 dev snapshots available in Plus you could test but they are intended for development only.
Yes you would need a Plus subscription for both nodes in the HA pair.
-
So, The backup FW I have the debug settings set on 'crashed' yesterday finally but again did not create any sort of crash dump, at least not one seen in the webGUI or no alert like it normally does stating that a crash has occurred and dump file has been created. I heard the box's fans spin up like they do when it restarts and checked after it booted the uptime on the home page of the device:
So not sure.
Additionally, I don't want to shift gears too much here but with regards to the idea of moving to PFSense Plus, I have a few queries I am trying to understand with regards to how this works.
- Can I keep the CE edition setup I have now and just buy support for that version or do I need to actually migrate to using the Plus version of the software to get support. Can I just buy support and just not migrate?
- Are there any configuration settings I risk losing by migrating. Right now, I am using OpenVPN, FreeRadius, FRR OSPF, OpenVPN Export utility, PFBlockerNG and obvious HA setup. Assuming if I have to migrate, the configuration would all come over OK between CE and Plus.
- Once migrated I would not be able to use my current config backups to restore to the migrated Plus version if needed.
Really like where my config is for this deployment next week so a little skittish about making this sort of change now so close to deployment date. I have a ton of users lined up to begin using OpenVPN so really need this to go smooth. Support availability would be great for this as I am an obvious noob but breaking this now would be rough to get back from.
Thanks for any help here. I reached out to sales for all this but have not heard back yet.
-
@rfranzke
1Yes, you can just register ce and it becomes plus
2.You will loose nothing. Just more options e.g in openvpn
3. You can use ce backups if neededThis wont solve your reboot issue, but it doesnt seem to crash either.
Could it be a faulty pdu? -
You can import a CE config into Plus and won't lose anything. Your current config backups would still be valid.
But I'm not sure support would be able to help much more with an issue like this. They would be doing the same things we are already doing.
Does it still create a core as expected if you force a panic with the sysctl?
Spontaneously rebooting like that without generating a crash report is usually a indication of a hardware issue. Though it can be some hardware issue that's triggered by a software change.
-
Yeah the support idea is more for my own inexperience with PFSense not so much for this particular issue. I have to deploy these next week and wanted some back up in case what I have done doesn't work for some reason. Its not an indictment of what we have done for this particular issue. Just not convinced I know what I am doing with this product to go full on deployment without some help. I think I have a handle on what I am doing with it but if I get this in place and some element does not work I'll need to get it sorted quickly. I'm not sure what to think on this crashing issue. It could be a hardware issue I guess, and this device is coincidentally restarting for other reasons just after we made changes to the panic capture settings. Just seems suspect that the dumps have disappeared after our changes and does not change the fact that it WAS creating dumps each time this thing would restart before we made changes. I am afraid maybe I didn't do something correct here somehow to break the dumps.
-
Also I did confirm I can force a dump file to be created using sysctl sysctl debug.kdb.panic=1. Seems like everything is in place to do this, it just doesn't. I agree that some faulty power issue somewhere would make some sense here, but we were definitely getting dump files before the changes and both boxes are doing this. I'll point out they are on the same PDU so yeah maybe it's to do with that. But we were getting dumps. Would a power issue have caused the dump file errors we were seeing before? If it was a power problem, then how did we ever get dumps. I'm hoping when I get these in place where they are going to live, things are more stable than where they are now.
-
@rfranzke I believe you see a combination of issues and this is driving us crazy.
It could be a pdu issue, AND something else too.Its very highly unlikely not to catch anything on dump and being a system crash.
Two things you can do.Put something on the pdu that can log power outages. This will shed some light on that front. A ups with power logging capabilities is something that can be found in an it environment. There are other options too.
Also try moving cables, sometimes its just a faulty cable that arcs and causes a reboot (or maybe a fire too)Second, put a serial console somewhere and log console output.
You might be able to catch something before system disappears. -
@netblues said in Frequent Crashing (Page Fault) After Upgrade to 2.8.0 From Latest 2.7:
Second, put a serial console somewhere and log console output.
Yup, that. In cases where nothing is logged or dumped the console can be the only place anything is seen.
If it panics you should still get the panic string in the logs even if no crash report is created though. Unless it cannot log for some reason.
-
So, for anybody keeping score, I finally got this deployed to production last weekend. So far this couldn't have gone smoother. Aside from a few users messing up OTP with VPN logins everything seems to have worked fine on PFSense's new home. HA works, FW rules work, NAT all seems to work. PFBlocker is doing its thing, OpenVPN seems as good if not better than our old AnyConnect setup from Cisco. Very impressed with the solution I have here after a week. Servers are not even breathing hard and handling our traffic fine. Really happy to get this behind me and to see PFSense work so well for us.
As for any crashes, so far there have been none. I'm worried this is something to do with the environment I was building this in. Everything is set to capture another crash if it happens but for now, I am just in wait and see mode.
Thanks everyone for their input. Really appreciate all the guidance. Hopefully all this still yields something useful. Will let you know.
