pkg-static update still using 100% cpu! Unacceptable!
-
@bmeeks Hi Bill. I gave up and tried it on a windows 7 pc and that worked. I've managed to reflash pfsense onto my SG-3100 and restore my config. Looks to still have the same problem. 1 cpu core stuck perpetually at 100%, and has not reinstalled the packages as it said it was doing at boot up.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
I don't know how to restore without restoring the package data
Open up your config.xml backup in a text editor. Look for the section titled <installedpackages> and delete that entire section then save the file under a different name. Restore from that new file.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Hi Bill. I gave up and tried it on a windows 7 pc and that worked. I've managed to reflash pfsense onto my SG-3100 and restore my config. Looks to still have the same problem. 1 cpu core stuck perpetually at 100%, and has not reinstalled the packages as it said it was doing at boot up.
One thing that can cause this is for the box to not have Internet access during the package installation stage. It will try forever to contact the pkg repository. Are you sure the box has a good Internet connection and that DNS is working?
Can you log in to the web GUI and then go to DIAGNOSTICS > DNS LOOKUP and try to look up a common web site by name such as google.com or cnn.com. See if you get back valid IP addresses.
@KOM has given you a method to manually edit your
config.xml
file to have the firewall skip attempting to auto-reinstall packages. You can try that as well, but make that change on a copy of your backup file and not to the original! -
@bmeeks Yes I've got internet access. DNS test is working fine. I can get on all my normal sites. And I'm responding to you.
So it seems that even a fresh, factory install cannot/will not install suricata. It just sits hung at 100% cpu.
What in the world is happening here? I'm ready to stomp on this thing.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Yes I've got internet access. DNS test is working fine. I can get on all my normal sites. And I'm responding to you.
So it seems that even a fresh, factory install cannot/will not install suricata. It just sits hung at 100% cpu.
What in the world is happening here? I'm ready to stomp on this thing.
Look in the system log and see where it is stalling with the installation. I would check if pfBlocker is perhaps blocking an IP address that Suricata wants to access. That has happened before since some pfBlocker lists target sections of AWS, and the Snort rules (if you are using them in Suricata) are hosted on AWS infrastructure.
The
pkg
utility will install the binary and GUI package code and then call a post-install PHP script within the Suricata package. That script detects your previous installation's configuration inconfig.xml
and starts restoring it. One step in that process is downloading the configured rules. -
@bmeeks pfblocker is not yet installed. And again, even a fresh factory image cannot install suricata. I will check the logs.
-
@bmeeks I've tried everything multiple times. I don't know what else to do. This box has turned itself into a doorstop. I give up.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried everything multiple times. I don't know what else to do. This box has turned itself into a doorstop. I give up.
You can easily remove the Suricata package configuration section. Just make a copy of the
config.xml
file and then open the copy in a text editor. Find the section that says <installedpackages> and remove all the suricata from that section. You will find several XML elements with Suricata info. There will be a <menu></menu> entry, a <service></service> entry, and then finally a <suricata></suricata> entry. Remove all of those tags and Suricata-related info enclosed by them. Save the newly modified file on the firewall and try rebooting again. -
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
-
@bmeeks I'm afraid pfsense is just not a production ready software suite. It's just not. It's glitchy, and full of bugs. And I've just ran face first into two big ones. #1 The fact that the package updater pegs the cpu at 100% if it doesn't get the responses it expects. And #2 The package update service is wholly unreliable, if it even works at all.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
Your replies to me and @KOM have been a little confusing. I thought you said the package reinstall was hanging during the initial reboot after first installing an image. Is that the case? Or does the box boot up fine and then you are attempting to install the packages onto a clean image (one where you did NOT import an existing
config.xml
)?If the latter, then your machine has a gremlin in it for sure. If you are restoring a config that had your list of installed packages in it, then try to restore a config with all the packages removed from the
config.xml
file. -
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
We're trying to help you. There are thousands and thousands of successful pfSense installs around the world. The vast majority of them in production situations. This problem appears isolated to your setup.
If pfSense truly had the issues you describe as a common situation, this board would be overrun with complaints and posts. There are none (or very, very few considering the number of pfSense installs around the world). Ranting and raving won't fix your problem. Maybe you need to stop for today, take a long rest, and try again tomorrow?
-
Dude, you admit in the first post that you messed something thing up and you have to resist 'cussing us up one side and down the other' You realize that you are asking for help from other users, right? This is not official support. You sound like a whiny twist- you can't get the usb driver to work on your out of support, over ten year old server, etc, etc. What's totally unacceptable is your attitude toward the people offering you help. Calm down and grow up.
-
@bmeeks It hangs during the initial reboot, and it hangs everywhere else too. It simply will not install suricata even after a clean re-image just after I enter the data needed to get the internet working. I can't even get it to install openvpn client export now. Just hangs and pegs the CPU. And nothing of any value is getting written into the system log either.
I know you're trying to help me and I greatly appreciate it. It just seems like I/we have ran out of ammo. I can't think of anything left to try. I've been working on this straight since midnight last night.
Yes Bill. CLEAN BRAND NEW RE-IMAGE. With nothing done except going through the wizard to get the internet working. Still will not work.
-
@dotdash I changed a setting that I couldn't figure out how to get it back how it was. I had backup's made assuming that it would be a simple restore and 10 minutes I'd be back to where I was before I started. I was so very wrong. So yes, I screwed up in assuming that a backup could actually be properly restored.
-
I've never seen that behavior, but I don't have experience with the arm version. I would re-image so I knew I was starting from scratch. Then I would NOT restore the config, but manually get connected to the internet, then verify I could ping from the box. Then I would try loading a package and see if it worked or returned errors. Knowing the error message might help to track down what's happening.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks It hangs during the initial reboot, and it hangs everywhere else too. It simply will not install suricata even after a clean re-image just after I enter the data needed to get the internet working. I can't even get it to install openvpn client export now. Just hangs and pegs the CPU. And nothing of any value is getting written into the system log either.
I know you're trying to help me and I greatly appreciate it. It just seems like I/we have ran out of ammo. I can't think of anything left to try. I've been working on this straight since midnight last night.
Yes Bill. CLEAN BRAND NEW RE-IMAGE. With nothing done except going through the wizard to get the internet working. Still will not work.
I would check to be sure that the /var/db/pkg directory is empty and then try the image restore again. It really looks like your
pkg
database files are trashed. I don't know if simply reinstalling a factory image will actually clear that directory out and start over or not.To be sure, here is what I would do. Get to a shell prompt on the firewall and run this command --
rm -rf /var/db/pkg
After running this command,
pkg
will definitely be hosed up. Refer to this pfSense documentation page and perform the steps there to recreate a newpkg
database structure: https://docs.netgate.com/pfsense/en/latest/packages/fixing-a-broken-pkg-database.html.You might also want to force a filesystem check at boot. Do that by following the instructions here: https://docs.netgate.com/pfsense/en/latest/hardware/forcing-a-filesystem-check.html.
-
Sounds to me like you might be seeing failing storage. You might try adding an M.2 and seeing if your issues are resolved.
https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/m-2-sata-installation.html
@RedDelPaPa So yes, I screwed up in assuming that a backup could actually be properly restored.
It can be. Perhaps not for you in your specific set of circumstances based on all of the facts I see, like a possibly failing storage.
If you have successfully recovered using the recovery image and are still experiencing problems, installing to new storage is probably your best path forward.
-
@dotdash I have tried exactly what you stated. I wanted to know if my config was really the problem. So I tried loading the packages I need right after booting from a fresh re-image and walking through the wizard and getting my internet connection working. It too was a no go.
-
@bmeeks Ok, here is what I have in there now after getting 3 of my 5 desired packages installed:
I will trying your next suggestions shortly.