pkg-static update still using 100% cpu! Unacceptable!
-
@jimp I can't get the com port driver installed on my windows server 2008 box. Tried to open a putty session and get only a blank screen. I need help!
-
The USB cable is (effectively) a serial console cable for these devices. Directions for the driver are all on the doc site if you need them. If you get hung up, send a message back to the support crew, they'll help you through the process.
-
@jimp I know that, i can't get the appropriate driver installed for my com port so i can putty into the pfsense box. I need help! The support guys haven't offered me any more help beyond sending me the image link.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@jimp I know that, i can't get the appropriate driver installed for my com port so i can putty into the pfsense box. I need help! The support guys haven't offered me any more help beyond sending me the image link.
The USB-to-serial driver should install and work on any current Windows desktop client. Do you not have a Windows PC or laptop you could use?
Never tried to use the USB-to-serial driver on Windows Server, especially something a bit older such as 2008.
-
@bmeeks Hi Bill. I gave up and tried it on a windows 7 pc and that worked. I've managed to reflash pfsense onto my SG-3100 and restore my config. Looks to still have the same problem. 1 cpu core stuck perpetually at 100%, and has not reinstalled the packages as it said it was doing at boot up.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
I don't know how to restore without restoring the package data
Open up your config.xml backup in a text editor. Look for the section titled <installedpackages> and delete that entire section then save the file under a different name. Restore from that new file.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Hi Bill. I gave up and tried it on a windows 7 pc and that worked. I've managed to reflash pfsense onto my SG-3100 and restore my config. Looks to still have the same problem. 1 cpu core stuck perpetually at 100%, and has not reinstalled the packages as it said it was doing at boot up.
One thing that can cause this is for the box to not have Internet access during the package installation stage. It will try forever to contact the pkg repository. Are you sure the box has a good Internet connection and that DNS is working?
Can you log in to the web GUI and then go to DIAGNOSTICS > DNS LOOKUP and try to look up a common web site by name such as google.com or cnn.com. See if you get back valid IP addresses.
@KOM has given you a method to manually edit your
config.xml
file to have the firewall skip attempting to auto-reinstall packages. You can try that as well, but make that change on a copy of your backup file and not to the original! -
@bmeeks Yes I've got internet access. DNS test is working fine. I can get on all my normal sites. And I'm responding to you.
So it seems that even a fresh, factory install cannot/will not install suricata. It just sits hung at 100% cpu.
What in the world is happening here? I'm ready to stomp on this thing.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Yes I've got internet access. DNS test is working fine. I can get on all my normal sites. And I'm responding to you.
So it seems that even a fresh, factory install cannot/will not install suricata. It just sits hung at 100% cpu.
What in the world is happening here? I'm ready to stomp on this thing.
Look in the system log and see where it is stalling with the installation. I would check if pfBlocker is perhaps blocking an IP address that Suricata wants to access. That has happened before since some pfBlocker lists target sections of AWS, and the Snort rules (if you are using them in Suricata) are hosted on AWS infrastructure.
The
pkg
utility will install the binary and GUI package code and then call a post-install PHP script within the Suricata package. That script detects your previous installation's configuration inconfig.xml
and starts restoring it. One step in that process is downloading the configured rules. -
@bmeeks pfblocker is not yet installed. And again, even a fresh factory image cannot install suricata. I will check the logs.
-
@bmeeks I've tried everything multiple times. I don't know what else to do. This box has turned itself into a doorstop. I give up.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried everything multiple times. I don't know what else to do. This box has turned itself into a doorstop. I give up.
You can easily remove the Suricata package configuration section. Just make a copy of the
config.xml
file and then open the copy in a text editor. Find the section that says <installedpackages> and remove all the suricata from that section. You will find several XML elements with Suricata info. There will be a <menu></menu> entry, a <service></service> entry, and then finally a <suricata></suricata> entry. Remove all of those tags and Suricata-related info enclosed by them. Save the newly modified file on the firewall and try rebooting again. -
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
-
@bmeeks I'm afraid pfsense is just not a production ready software suite. It's just not. It's glitchy, and full of bugs. And I've just ran face first into two big ones. #1 The fact that the package updater pegs the cpu at 100% if it doesn't get the responses it expects. And #2 The package update service is wholly unreliable, if it even works at all.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
Your replies to me and @KOM have been a little confusing. I thought you said the package reinstall was hanging during the initial reboot after first installing an image. Is that the case? Or does the box boot up fine and then you are attempting to install the packages onto a clean image (one where you did NOT import an existing
config.xml
)?If the latter, then your machine has a gremlin in it for sure. If you are restoring a config that had your list of installed packages in it, then try to restore a config with all the packages removed from the
config.xml
file. -
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I've tried all that, Bill. It still won't work. I've tried installing the packages I need on a fresh image just after I entered all of my IP, DNS, and WAN data to get the Internet working. I would think that if it doesn't work then, it surely isn't gonna work at any other step either.
We're trying to help you. There are thousands and thousands of successful pfSense installs around the world. The vast majority of them in production situations. This problem appears isolated to your setup.
If pfSense truly had the issues you describe as a common situation, this board would be overrun with complaints and posts. There are none (or very, very few considering the number of pfSense installs around the world). Ranting and raving won't fix your problem. Maybe you need to stop for today, take a long rest, and try again tomorrow?
-
Dude, you admit in the first post that you messed something thing up and you have to resist 'cussing us up one side and down the other' You realize that you are asking for help from other users, right? This is not official support. You sound like a whiny twist- you can't get the usb driver to work on your out of support, over ten year old server, etc, etc. What's totally unacceptable is your attitude toward the people offering you help. Calm down and grow up.
-
@bmeeks It hangs during the initial reboot, and it hangs everywhere else too. It simply will not install suricata even after a clean re-image just after I enter the data needed to get the internet working. I can't even get it to install openvpn client export now. Just hangs and pegs the CPU. And nothing of any value is getting written into the system log either.
I know you're trying to help me and I greatly appreciate it. It just seems like I/we have ran out of ammo. I can't think of anything left to try. I've been working on this straight since midnight last night.
Yes Bill. CLEAN BRAND NEW RE-IMAGE. With nothing done except going through the wizard to get the internet working. Still will not work.
-
@dotdash I changed a setting that I couldn't figure out how to get it back how it was. I had backup's made assuming that it would be a simple restore and 10 minutes I'd be back to where I was before I started. I was so very wrong. So yes, I screwed up in assuming that a backup could actually be properly restored.
-
I've never seen that behavior, but I don't have experience with the arm version. I would re-image so I knew I was starting from scratch. Then I would NOT restore the config, but manually get connected to the internet, then verify I could ping from the box. Then I would try loading a package and see if it worked or returned errors. Knowing the error message might help to track down what's happening.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks It hangs during the initial reboot, and it hangs everywhere else too. It simply will not install suricata even after a clean re-image just after I enter the data needed to get the internet working. I can't even get it to install openvpn client export now. Just hangs and pegs the CPU. And nothing of any value is getting written into the system log either.
I know you're trying to help me and I greatly appreciate it. It just seems like I/we have ran out of ammo. I can't think of anything left to try. I've been working on this straight since midnight last night.
Yes Bill. CLEAN BRAND NEW RE-IMAGE. With nothing done except going through the wizard to get the internet working. Still will not work.
I would check to be sure that the /var/db/pkg directory is empty and then try the image restore again. It really looks like your
pkg
database files are trashed. I don't know if simply reinstalling a factory image will actually clear that directory out and start over or not.To be sure, here is what I would do. Get to a shell prompt on the firewall and run this command --
rm -rf /var/db/pkg
After running this command,
pkg
will definitely be hosed up. Refer to this pfSense documentation page and perform the steps there to recreate a newpkg
database structure: https://docs.netgate.com/pfsense/en/latest/packages/fixing-a-broken-pkg-database.html.You might also want to force a filesystem check at boot. Do that by following the instructions here: https://docs.netgate.com/pfsense/en/latest/hardware/forcing-a-filesystem-check.html.
-
Sounds to me like you might be seeing failing storage. You might try adding an M.2 and seeing if your issues are resolved.
https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/m-2-sata-installation.html
@RedDelPaPa So yes, I screwed up in assuming that a backup could actually be properly restored.
It can be. Perhaps not for you in your specific set of circumstances based on all of the facts I see, like a possibly failing storage.
If you have successfully recovered using the recovery image and are still experiencing problems, installing to new storage is probably your best path forward.
-
@dotdash I have tried exactly what you stated. I wanted to know if my config was really the problem. So I tried loading the packages I need right after booting from a fresh re-image and walking through the wizard and getting my internet connection working. It too was a no go.
-
@bmeeks Ok, here is what I have in there now after getting 3 of my 5 desired packages installed:
I will trying your next suggestions shortly.
-
@Derelict Interesting. What is used in an SG-3100 for the native storage? Is it flash memory or a regular magnetic hard drive?
If it were failing storage, wouldn't I likely see garbled log files and such?
-
@dotdash I got this after between 1 to 4 hours waiting for openvpn client export to install:
I can copy that link into my browser and it goes right to it immediately. So pfsense is also broken here if it doesn't perform any retry's.
-
@Derelict Can I install to a USB stick plugged into an SG-3100 as a test for bad storage?
-
Ok guys. BRAND NEW RE-IMAGE. I run through the wizard to get my lan/wan info entered so I can connect to the internet. CPU usage normal. Everything appears normal. No installed packages.
I then go to package manager / installed packages and it hangs for about 5 minutes and then fails with this:
Meanwhile it pegs the cpu at 100% and still pegged:
-
From console access, or SSH acces :
Use option 8 and type
dig _http._tcp.pkg.pfsense.org SRV +short
It should answer :
10 10 80 files00.netgate.com. 10 10 80 files01.netgate.com.
Type exit, you'll be back in the main menu, and use option 13 :
..... Enter an option: 13 >>> Updating repositories metadata... Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. Your packages are up to date pfSense - Netgate Device ID: 233983e240a7b45d741b .....
edit : if you can 'decode' this https://forum.netgate.com/topic/140133/unable-to-retrieve-package-information - there is an interesting end ....
and some more testing procedures. -
@Gertjan yes sir. Thank you. It's just crazy that this thing is already hanging and failing right from the starting gun.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@Gertjan yes sir. Thank you. It's just crazy that this thing is already hanging and failing right from the starting gun.
Do you by chance use IPv6 in your setup? Specifically, does your ISP give you an IPv6 address along with an IPv4, or is it just IPv4?
The symptoms you are having scream something with either DNS, connectivity or routing. One of your earlier posts showed a "no route to host" error, but that error occurred after some packages in the set had downloaded. Is your WAN interface flapping perhaps? Something happened within your network or hardware on the SG-3100 that caused it to lose the ability to "see" the server it was downloading the files from.
The
pkg
utility is a FreeBSD item and not specific to pfSense. And it is flakey when it does not have network access when it wants it. It also does not always fail gracefully. -
@bmeeks Good morning Bill. No sir. I do not use IPv6 nor did my ISP offer it when I purchased my service.
When those no route to host failures happen, I copied and pasted the link it gives into a browser and it goes right to it immediately.
That brings up another question: Is it possible for me to just download the packages and dependencies via my browser and then copy them to the correct place in pfsense and command it to install packages from there instead of checking the internet for them? Kinda like an offline install for a windows update?
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Good morning Bill. No sir. I do not use IPv6 nor did my ISP offer it when I purchased my service.
When those no route to host failures happen, I copied and pasted the link it gives into a browser and it goes right to it immediately.
That brings up another question: Is it possible for me to just download the packages and dependencies via my browser and then copy them to the correct place in pfsense and command it to install packages from there instead of checking the internet for them? Kinda like an offline install for a windows update?
Is your Internet traffic from the device with the browser going through the pfSense firewall? If it is a wireless connection, are you 100% positive it is going through pfSense at the time you made the browser request? Some mobile devices, if they have cellular data, will auto-switch to cellular without any indication if their wi-fi access is interrupted. Don't know if that applies to you, but just throwing out some ideas.
Are the pfSense firewall and your other device both using the identical DNS server configuration? Could one be using a different DNS server for name resolution?
I keep asking these questions because the symptoms you describe, and that "no route to host" error message, sort of scream either connectivity issues, DNS issues or both with the SG-3100. Could you provide a simple drawing showing how the SG-3100 is connected into your network?
As for downloading the packages and installing them locally, yes you can do that but it will be an iterative process. There are often times multiple dependent packages required. When you install a package, if it wants another one to satisfy a dependency, it will error out. So you will have to download that package and keep going through the iterations until you make
pkg
happy.The packages repository URL is here: https://files00.netgate.com/pfSense_v2_4_4_amd64-pfSense_v2_4_4/All/. To see how to use
pkg
to install local package files, refer to this help page: https://www.freebsd.org/cgi/man.cgi?pkg(7). The command you will use ispkg add <file>
. -
@bmeeks Absolutely positive. I'm working from a desktop and the only route to the internet is through the pfsense box. Hostnames and DNS are all handled through the pfsense box via DHCP.
UPDATE: I have discovered that if I keep trying to install a package, it will eventually succeed after about 25 to 50 tries. So it seems to me, the only reason this has become such a huge problem is because the package updater utility sucks terribly. It must not have any stop/retry/resume capability.
-
@bmeeks The path to and from my desktop to the internet is as follows:
Actiontec C3000A DSL modem (public IP) > (Public IP) Pfsense Box (Private IP) Netgear 16 port switch > Desktop (Private IP).
Wifi AP (Private IP) connected to Netgear switch for wifi connectivity.
This has been my setup for the last year and it has worked flawlessly. And still does, except for the pfsense box's ability to download and install packages.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Absolutely positive. I'm working from a desktop and the only route to the internet is through the pfsense box. Hostnames and DNS are all handled through the pfsense box via DHCP.
UPDATE: I have discovered that if I keep trying to install a package, it will eventually succeed after about 25 to 50 tries. So it seems to me, the only reason this has become such a huge problem is because the package updater utility sucks terribly. It must not have any stop/retry/resume capability.
No, there is something unique wrong with your setup. The
pkg
utility works fine for the vast majority of pfSense and FreeBSD users. I don't know what your issue is, but I can assure you that it is not a widespread problem withpkg
. If it was, everyone all over the world would be screaming. There would never be a valid reason for the utility to expect to attempt a file download 20 to 50 times before it succeeded. True someone could add a "resume" feature one day, but for most folks that would be pointless. The downloaded package files are usually just a few megabytes in size at most, and come down very quickly on most connections these days.So we need to just focus on what is wrong with your setup. One hunch would be a hardware issue with your NIC, but since you say the rest of your network works fine, that would sort of take that out of the equation. That theory was why I was asking for your network configuration.
Did you ever run the disk test I posted the command for? It is possible you have a disk issue, however I would expect to see some hardware errors logged in that case. The
pkg
utility will write the files to disk as it downloads them. A failing disk (actually a type of SSD for the SG-3100) would cause issues.Are you still seeing any "no route to host" error messages like the one you posted previously? That really indicates a connectivity issue at some level in the box.
-
@bmeeks Yes I can run speed test after speed test, or consistent pings to specific hosts with no drops and consistent high speeds. One fellow did mention he thinks my box might have failing storage as well. I ran that test you asked. Where do I find the results?
I'm still getting routine no route to host fails when trying to install packages.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Yes I can run speed test after speed test, or consistent pings to specific hosts with no drops and consistent high speeds. One fellow did mention he thinks my box might have failing storage as well. I ran that test you asked. Where do I find the results?
I'm still getting routine no route to host fails when trying to install packages.
Is any other package actually successfully installed on the box yet? If so, disable everything and simply concentrate on installing just the OpenVPN Client Export package (I think that was one you were installing). Other packages, if they are installed, can certainly interfere and cause issues.
Also, does anything suggested in this Reddit thread apply or help you?
https://www.reddit.com/r/PFSENSE/comments/9au4hj/no_route_to_host_install_packages_locally/
-
@bmeeks I will say that occasionally my personal and work mobile phones will have a little trouble opening data in certain apps through wifi. But I have never noticed any connectivity issues at all from my hard wired desktop machines that all go through the pfsense router.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I will say that occasionally my personal and work mobile phones will have a little trouble opening data in certain apps through wifi. But I have never noticed any connectivity issues at all from my hard wired desktop machines that all go through the pfsense router.
I am fairly certain you have something going on with your network configuration. "No route to host" means just what it says -- the firewall itself can't get out to the Internet, or it gets out only sporadically. Check the things listed in that Reddit thread like default route, make sure you have no gateway defined on the LAN, etc.