pkg-static update still using 100% cpu! Unacceptable!
-
@Derelict Can I install to a USB stick plugged into an SG-3100 as a test for bad storage?
-
Ok guys. BRAND NEW RE-IMAGE. I run through the wizard to get my lan/wan info entered so I can connect to the internet. CPU usage normal. Everything appears normal. No installed packages.
I then go to package manager / installed packages and it hangs for about 5 minutes and then fails with this:
Meanwhile it pegs the cpu at 100% and still pegged:
-
From console access, or SSH acces :
Use option 8 and type
dig _http._tcp.pkg.pfsense.org SRV +short
It should answer :
10 10 80 files00.netgate.com. 10 10 80 files01.netgate.com.
Type exit, you'll be back in the main menu, and use option 13 :
..... Enter an option: 13 >>> Updating repositories metadata... Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. Your packages are up to date pfSense - Netgate Device ID: 233983e240a7b45d741b .....
edit : if you can 'decode' this https://forum.netgate.com/topic/140133/unable-to-retrieve-package-information - there is an interesting end ....
and some more testing procedures. -
@Gertjan yes sir. Thank you. It's just crazy that this thing is already hanging and failing right from the starting gun.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@Gertjan yes sir. Thank you. It's just crazy that this thing is already hanging and failing right from the starting gun.
Do you by chance use IPv6 in your setup? Specifically, does your ISP give you an IPv6 address along with an IPv4, or is it just IPv4?
The symptoms you are having scream something with either DNS, connectivity or routing. One of your earlier posts showed a "no route to host" error, but that error occurred after some packages in the set had downloaded. Is your WAN interface flapping perhaps? Something happened within your network or hardware on the SG-3100 that caused it to lose the ability to "see" the server it was downloading the files from.
The
pkg
utility is a FreeBSD item and not specific to pfSense. And it is flakey when it does not have network access when it wants it. It also does not always fail gracefully. -
@bmeeks Good morning Bill. No sir. I do not use IPv6 nor did my ISP offer it when I purchased my service.
When those no route to host failures happen, I copied and pasted the link it gives into a browser and it goes right to it immediately.
That brings up another question: Is it possible for me to just download the packages and dependencies via my browser and then copy them to the correct place in pfsense and command it to install packages from there instead of checking the internet for them? Kinda like an offline install for a windows update?
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Good morning Bill. No sir. I do not use IPv6 nor did my ISP offer it when I purchased my service.
When those no route to host failures happen, I copied and pasted the link it gives into a browser and it goes right to it immediately.
That brings up another question: Is it possible for me to just download the packages and dependencies via my browser and then copy them to the correct place in pfsense and command it to install packages from there instead of checking the internet for them? Kinda like an offline install for a windows update?
Is your Internet traffic from the device with the browser going through the pfSense firewall? If it is a wireless connection, are you 100% positive it is going through pfSense at the time you made the browser request? Some mobile devices, if they have cellular data, will auto-switch to cellular without any indication if their wi-fi access is interrupted. Don't know if that applies to you, but just throwing out some ideas.
Are the pfSense firewall and your other device both using the identical DNS server configuration? Could one be using a different DNS server for name resolution?
I keep asking these questions because the symptoms you describe, and that "no route to host" error message, sort of scream either connectivity issues, DNS issues or both with the SG-3100. Could you provide a simple drawing showing how the SG-3100 is connected into your network?
As for downloading the packages and installing them locally, yes you can do that but it will be an iterative process. There are often times multiple dependent packages required. When you install a package, if it wants another one to satisfy a dependency, it will error out. So you will have to download that package and keep going through the iterations until you make
pkg
happy.The packages repository URL is here: https://files00.netgate.com/pfSense_v2_4_4_amd64-pfSense_v2_4_4/All/. To see how to use
pkg
to install local package files, refer to this help page: https://www.freebsd.org/cgi/man.cgi?pkg(7). The command you will use ispkg add <file>
. -
@bmeeks Absolutely positive. I'm working from a desktop and the only route to the internet is through the pfsense box. Hostnames and DNS are all handled through the pfsense box via DHCP.
UPDATE: I have discovered that if I keep trying to install a package, it will eventually succeed after about 25 to 50 tries. So it seems to me, the only reason this has become such a huge problem is because the package updater utility sucks terribly. It must not have any stop/retry/resume capability.
-
@bmeeks The path to and from my desktop to the internet is as follows:
Actiontec C3000A DSL modem (public IP) > (Public IP) Pfsense Box (Private IP) Netgear 16 port switch > Desktop (Private IP).
Wifi AP (Private IP) connected to Netgear switch for wifi connectivity.
This has been my setup for the last year and it has worked flawlessly. And still does, except for the pfsense box's ability to download and install packages.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Absolutely positive. I'm working from a desktop and the only route to the internet is through the pfsense box. Hostnames and DNS are all handled through the pfsense box via DHCP.
UPDATE: I have discovered that if I keep trying to install a package, it will eventually succeed after about 25 to 50 tries. So it seems to me, the only reason this has become such a huge problem is because the package updater utility sucks terribly. It must not have any stop/retry/resume capability.
No, there is something unique wrong with your setup. The
pkg
utility works fine for the vast majority of pfSense and FreeBSD users. I don't know what your issue is, but I can assure you that it is not a widespread problem withpkg
. If it was, everyone all over the world would be screaming. There would never be a valid reason for the utility to expect to attempt a file download 20 to 50 times before it succeeded. True someone could add a "resume" feature one day, but for most folks that would be pointless. The downloaded package files are usually just a few megabytes in size at most, and come down very quickly on most connections these days.So we need to just focus on what is wrong with your setup. One hunch would be a hardware issue with your NIC, but since you say the rest of your network works fine, that would sort of take that out of the equation. That theory was why I was asking for your network configuration.
Did you ever run the disk test I posted the command for? It is possible you have a disk issue, however I would expect to see some hardware errors logged in that case. The
pkg
utility will write the files to disk as it downloads them. A failing disk (actually a type of SSD for the SG-3100) would cause issues.Are you still seeing any "no route to host" error messages like the one you posted previously? That really indicates a connectivity issue at some level in the box.
-
@bmeeks Yes I can run speed test after speed test, or consistent pings to specific hosts with no drops and consistent high speeds. One fellow did mention he thinks my box might have failing storage as well. I ran that test you asked. Where do I find the results?
I'm still getting routine no route to host fails when trying to install packages.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks Yes I can run speed test after speed test, or consistent pings to specific hosts with no drops and consistent high speeds. One fellow did mention he thinks my box might have failing storage as well. I ran that test you asked. Where do I find the results?
I'm still getting routine no route to host fails when trying to install packages.
Is any other package actually successfully installed on the box yet? If so, disable everything and simply concentrate on installing just the OpenVPN Client Export package (I think that was one you were installing). Other packages, if they are installed, can certainly interfere and cause issues.
Also, does anything suggested in this Reddit thread apply or help you?
https://www.reddit.com/r/PFSENSE/comments/9au4hj/no_route_to_host_install_packages_locally/
-
@bmeeks I will say that occasionally my personal and work mobile phones will have a little trouble opening data in certain apps through wifi. But I have never noticed any connectivity issues at all from my hard wired desktop machines that all go through the pfsense router.
-
@RedDelPaPa said in pkg-static update still using 100% cpu! Unacceptable!:
@bmeeks I will say that occasionally my personal and work mobile phones will have a little trouble opening data in certain apps through wifi. But I have never noticed any connectivity issues at all from my hard wired desktop machines that all go through the pfsense router.
I am fairly certain you have something going on with your network configuration. "No route to host" means just what it says -- the firewall itself can't get out to the Internet, or it gets out only sporadically. Check the things listed in that Reddit thread like default route, make sure you have no gateway defined on the LAN, etc.
-
@bmeeks I've managed to get pfBlocker back on, and mail report. Still can't get suricata back on.
Heading out to lunch. I will check on this when I get back. Thanks for all your help, Bill.
-
Here is one other set of
pkg
troubleshooting steps/commands you can try --Running that "update" sub-command will maybe help identify what the issue is.
Also, with pfBlocker, if you have large IP lists that it is downloading/updating and if you are using DNSBL, that can cause issues. For instance, with DNSBL, each time it downloads an updated list (or an initial list), it will restart the
unbound
DNS resolver service. Whileunbound
is restarting, it can't service DNS requests from something likepkg
. So lots of potential moving parts to consider while troubleshooting. That's why I suggested disabling every installed package while you are trying to install another one. This is especially true for pfBlocker. -
@bmeeks Checked that reddit post and everything looks to be as it should IPv4 is my default gateway and the address is correct. No LAN gateway configured.