Unable to install or upgrade using pkg

PertFlavus · May 4, 2016, 6:37 AM

Hey guys,

When I try to upgrade from 2.3 to 2.3_1 my system hangs. On closer inspection the pkg process is maxing out one of my cores without accomplishing much. It gets so far as asking if I want to proceed and then hangs - doing nothing.

From top:
last pid: 50113; load averages: 1.02, 0.92, 0.54 up 0+00:07:22 23:30:22
45 processes: 2 running, 43 sleeping
CPU: 1.5% user, 0.0% nice, 48.7% system, 0.0% interrupt, 49.8% idle
Mem: 47M Active, 35M Inact, 89M Wired, 1088K Cache, 72M Buf, 2179M Free
Swap: 8192M Total, 8192M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
13442 root 1 103 0 15568K 8368K CPU0 0 2:16 100.00% pkg
31200 unbound 2 20 0 29680K 13464K kqread 1 0:02 0.00% unbound
34344 root 1 20 0 11256K 2484K CPU1 1 0:00 0.00% top
15153 root 1 20 0 10232K 1936K bpf 0 0:00 0.00% filterlog

I have tried updating ntp individually as well as installing lsof - but the result is the same as if I am using the gui for the upgrade
[2.3-RELEASE][admin@pfSense.appus.org]/root: pkg install lsof
Updating pfSense-core repository catalogue…
pfSense-core repository is up-to-date.
Updating pfSense repository catalogue...
pfSense repository is up-to-date.
All repositories are up-to-date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
lsof: 4.90.a,8 [pfSense]

The process will require 220 KiB more space.
100 KiB to be downloaded.

Proceed with this action? [y/N]: y

I can ctrl+c the task and usage returns to normal.

I'm not sure what to look for here. System.log doesn't have anything mentioning the upgrade.

This was previously installed as 2.2.x and upgraded to 2.3. Once upgraded I reset the config and went from scratch. I really didn't do anything too outlandish.. just set up the DNS server with some custom hostname overrides, a few port forwards, and static dhcp mappings.

Rebooting has not helped. I haven't tried uninstalling ntp just because it would also uninstall pfsense and I don't know what that will do.

Here's some info on my file system:
[2.3-RELEASE][admin@pfSense.appus.org]/var/log: df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ad4s1a 46G 1.4G 41G 3% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/md0 124M 116K 114M 0% /tmp
/dev/md1 124M 18M 96M 16% /var
devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev
[2.3-RELEASE][admin@pfSense.appus.org]/var/log: fstab -l
fstab: Command not found.
[2.3-RELEASE][admin@pfSense.appus.org]/var/log: gpart show
=> 63 117231345 ada0 MBR (56G)
63 117231345 1 freebsd [active] (56G)

=> 0 117231345 ada0s1 BSD (56G)
0 16 - free - (8.0K)
16 100454113 1 freebsd-ufs (48G)
100454129 16777216 2 freebsd-swap (8.0G)

Syslog doesn't have anything since the reboot, even though I tried installing lsof and upgrading after.
[2.3-RELEASE][admin@pfSense.appus.org]/var/log: tail system.log
May 3 23:23:33 pfSense php-fpm[31514]: /rc.start_packages: Restarting/Starting all packages.
May 3 23:23:33 pfSense kernel: done.
May 3 23:23:35 pfSense login: login on ttyv0 as root
May 3 23:23:35 pfSense sshlockout[26585]: sshlockout/webConfigurator v3.0 starting up
May 3 23:23:44 pfSense sshd[28603]: Accepted keyboard-interactive/pam for admin from 10.0.0.150 port 61722 ssh2
May 3 23:24:24 pfSense sshd[31850]: Accepted keyboard-interactive/pam for admin from 10.0.0.150 port 61750 ssh2
May 3 23:26:38 pfSense sshd[70330]: Accepted keyboard-interactive/pam for admin from 10.0.0.150 port 61803 ssh2
May 3 23:27:33 pfSense check_reload_status: Linkup starting re0
May 3 23:27:33 pfSense kernel: re0: link state changed to DOWN
CLOGT[▒▒[2.3-RELEASE][admin@pfSense.appus.org]/var/log:

I'm sure this is something pretty specific but I am at a loss. any help would be welcome.
Matt

johnpoz · May 4, 2016, 12:20 PM

not sure why its trying to install lsof… while I see they added that to the pfsense repo. I currently do not have it installed, nor did it attempt to install when updated to 2.3_1

[2.3-RELEASE][root@pfSense.local.lan]/root: pkg info lsof
pkg: No package(s) matching lsof
[2.3-RELEASE][root@pfSense.local.lan]/root: pkg search lsof
lsof-4.90.a,8 Lists information about open files (similar to fstat(1))
[2.3-RELEASE][root@pfSense.local.lan]/root:

cmb · May 4, 2016, 9:46 PM

It's not trying to install lsof on its own, OP's just trying that manually. Need to know what exactly that pkg command is doing. What do you get for:

ps auwwx|grep 13442

PertFlavus · May 5, 2016, 2:18 AM

@cmb:

It's not trying to install lsof on its own, OP's just trying that manually. Need to know what exactly that pkg command is doing. What do you get for:
ps auwwx|grep 13442 

You are right that I ran lsof myself as I was trying to see what pkg was doing. From a normal gui upgrade:
last pid: 6619; load averages: 1.00, 0.72, 0.37 up 0+19:56:54 19:19:54
48 processes: 3 running, 45 sleeping
CPU: 4.8% user, 0.0% nice, 49.2% system, 0.0% interrupt, 46.0% idle
Mem: 25M Active, 71M Inact, 91M Wired, 1084K Cache, 35M Buf, 2162M Free
Swap: 8192M Total, 8192M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
73102 root 1 103 0 15568K 9876K RUN 1 0:50 100.00% pkg
61857 root 1 44 0 99744K 28844K CPU0 0 0:08 12.60% php-fpm
31200 unbound 2 20 0 41968K 23292K kqread 1 1:20 0.00% unbound
73947 root 1 52 20 10456K 1980K wait 1 0:29 0.00% sh

[2.3-RELEASE][admin@pfSense.appus.org]/root: ps auwwx | grep 73102
root 73102 98.3 0.4 15568 9876 - R 7:19PM 0:37.72 pkg -o EVENT_PIPE=/tmp/pfSense-upgrade.sock upgrade -F

cmb · May 5, 2016, 5:40 AM

Try to kill -9 that pkg process and run it manually as 'pkg upgrade', that should show more useful output. What do you see for that?

johnpoz · May 5, 2016, 11:35 AM

But that is not what he stated, he stated "When I try to upgrade from 2.3 to 2.3_1 my system hangs."

And then he shows it trying to install lsof… He made no mention that he was trying to install standalone pkgs..

My bad I guess for not looking closer..

PertFlavus · May 6, 2016, 1:45 AM

@cmb:

Try to kill -9 that pkg process and run it manually as 'pkg upgrade', that should show more useful output. What do you see for that?

Even less.
[2.3-RELEASE][admin@pfSense.appus.org]/root: ps auwwx | grep 63769
root 63769 100.0 0.4 15568 9912 0 R+ 5:30PM 1:11.33 pkg upgrade

I expanded my window width - but that's all it returned.

from running the pkg upgrade
[2.3-RELEASE][admin@pfSense.appus.org]/root: pkg update
Updating pfSense-core repository catalogue…
pfSense-core repository is up-to-date.
Updating pfSense repository catalogue...
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 100 KiB 102.1kB/s 00:01
Processing entries: 100%
pfSense repository update completed. 370 packages processed.
[2.3-RELEASE][admin@pfSense.appus.org]/root: pkg upgrade
Updating pfSense-core repository catalogue…
pfSense-core repository is up-to-date.
Updating pfSense repository catalogue...
pfSense repository is up-to-date.
All repositories are up-to-date.
Checking for upgrades (5 candidates): 100%
Processing candidates (5 candidates): 100%
The following 2 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
pfSense: 2.3 -> 2.3_1 [pfSense]
ntp: 4.2.8p6 -> 4.2.8p7 [pfSense]

The process will require 2 KiB more space.
452 KiB to be downloaded.

Proceed with this action? [y/N]: y

and from top:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
63769 root 1 103 0 15568K 9912K CPU1 1 2:38 100.00% pkg
31200 unbound 2 20 0 46064K 28008K kqread 0 2:43 0.00% unbound

@johnpoz:

But that is not what he stated, he stated "When I try to upgrade from 2.3 to 2.3_1 my system hangs."

And then he shows it trying to install lsof… He made no mention that he was trying to install standalone pkgs..

My bad I guess for not looking closer..

I'm not going to criticize you for trying to help. I just got the same results with either output so I didn't think it made a difference and didn't want to give the same output multiple times.. so I figured the output showing even other packages were affected was more useful.

At this point I have the same question you all do. What the heck is pkg doing or unable to do? I was hoping lsof would give me a lead but seeing as how I can't install it… nope!

edit:
.. i'm definitely reinstalling. I just noticed this is a 32bit install. No reason not to be 64bit.

PertFlavus · May 9, 2016, 1:51 AM

A reinstall worked, I also decided its time to retire my aging fit-pc and ordered an SG-2440.