2.2.3 to 2.2.4 Taking Very Long Time

cmb

Crap media + SU+J + sync -> performance disaster.

Yeah, crap flash is going to be really slow from that. Though probably just sync making it slow, assuming it was installed pre-2.2.3, as upgraded full installs weren't changed to SU+J. I also never noticed any performance diff with or without SU+J, with crap flash it was just a high likelihood of ending up with a corrupt filesystem on unclean shut down if you were writing to the flash shortly before.

Let it run, it should finish fine. The sync will go away post-upgrade so future upgrades will be back to the same speed as before.

NOYB

Thanks. I'll just be patient then. Is there a point at which to just give up and take some action?

Note the USB flash doesn't appear to being accessed much. Maybe a couple seconds per minute or so. Does that reveal anything such as the flash not being the cause?

NOYB

Three hours and counting.

![pfSense Upgrade Top.png](/public/imported_attachments/1/pfSense Upgrade Top.png)
![pfSense Upgrade Top.png_thumb](/public/imported_attachments/1/pfSense Upgrade Top.png_thumb)

cmb

Running mtree, has to be very near the end of the process. If mtree's still running that hard in another couple hours, I'd just reboot the box. Won't hurt once it's gotten to that point, it's already installed the files and mtree shouldn't need to do anything with any of the files.

NOYB

Okay. Thanks for the input.

NOYB

Figured 7.5 hours was long enough. Rebooted. During startup noticed the message about the foreach issue in the config upgrade scroll by that some have reported this previously. Don't see it in the system logs though.

![System Processor.png](/public/imported_attachments/1/System Processor.png)
![System Processor.png_thumb](/public/imported_attachments/1/System Processor.png_thumb)
![System Memory.png](/public/imported_attachments/1/System Memory.png)
![System Memory.png_thumb](/public/imported_attachments/1/System Memory.png_thumb)

nikkon

Had the same issue on 3 vm's an 2 bare metal machines.
After a restart it worked fine!
still not a normal behaviour.

cmb

@nikkon:

Had the same issue on 3 vm's an 2 bare metal machines.
After a restart it worked fine!

With mtree chewing up 100% CPU?

cmb

@NOYB:

During startup noticed the message about the foreach issue in the config upgrade scroll by that some have reported this previously. Don't see it in the system logs though.

That's just what happens when it goes through the config upgrade and you don't have any IPsec config, it was fixed to silence the noise.

nikkon

With mtree chewing up 100% CPU?

nop. it seems ok

cmb

@nikkon:

With mtree chewing up 100% CPU?

nop. it seems ok

Before the reboot I mean.

colinscholz

I had the same issue. After about 30 minutes of it saying something like updating firmware please wait, I went to reboot. I didn't have access to the console (VM) because it was pretty much locked, so I just ssh'd in. It said it was on 2.2.4 so I hit 5 y and it rebooted gracefully and that was that. No errors in the system log after and 2.2.4 was installed. It was very odd since .1-.3 took all of 2 minutes if that.

velbon

I wish i had never upgraded to 2.2.4. Please were you able to solve this issue because i am having the same problem and my upgrade had been for 2 days now.
I want to know if doing hardware reboot will actually take the system version to 2.2.4 or not? or do i need to do a fresh installation of 2.2.4 to solve the problem.

Abhishek

I am also facing same issue , i kept upgrade at 8Am , now its 04:19 PM still not upgrade done


 	Packages are currently being reinstalled in the background.

Do not make changes in the GUI until this is complete


Aug 4 16:16:05 	php-fpm[26492]: /pkg_edit.php: Successful login for user 'admin' from: 192.168.1.211
Aug 4 16:16:05 	php-fpm[26492]: /pkg_edit.php: Successful login for user 'admin' from: 192.168.1.211
Aug 4 16:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:15:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 16:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:02:44 	snmpd[41828]: could not encode error response
Aug 4 16:02:39 	snmpd[41828]: could not encode error response
Aug 4 16:02:34 	snmpd[41828]: could not encode error response
Aug 4 16:02:29 	snmpd[41828]: could not encode error response
Aug 4 16:02:18 	sshd[24204]: Did not receive identification string from 192.168.1.254
Aug 4 16:01:26 	sshd[2067]: Did not receive identification string from 192.168.1.254
Aug 4 16:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 16:00:00 	check_reload_status: Syncing firewall
Aug 4 16:00:00 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg ' returned exit code '1', the output was 'SARG: File not found: /var/log/squid/access.log'
Aug 4 16:00:00 	php: pfblockerng.php: [pfBlockerNG] Starting sync process.
Aug 4 16:00:00 	php: sarg.php: Sarg: force refresh now with args, compress(on) and none action after sarg finish.
Aug 4 15:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:50:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:45:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:40:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:40:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:40:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:40:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:35:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:35:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:35:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:35:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:30:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:27:52 	php-fpm[7982]: /reboot.php: Successful login for user 'admin' from: 192.168.1.100
Aug 4 15:27:52 	php-fpm[7982]: /reboot.php: Successful login for user 'admin' from: 192.168.1.100
Aug 4 15:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:25:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:20:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:20:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:15:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:10:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:05:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 15:00:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 15:00:00 	check_reload_status: Syncing firewall
Aug 4 15:00:00 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg ' returned exit code '1', the output was 'SARG: File not found: /var/log/squid/access.log'
Aug 4 15:00:00 	php: pfblockerng.php: [pfBlockerNG] Starting sync process.
Aug 4 15:00:00 	php: sarg.php: Sarg: force refresh now with args, compress(on) and none action after sarg finish.
Aug 4 14:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:55:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:55:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:50:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:50:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:45:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:45:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:45:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:45:00 	snort[22978]: invalid appid in appStatRecord (2046)
Aug 4 14:40:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:40:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:40:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:35:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:35:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 14:35:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:30:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:20:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:05:56 	php-fpm[87057]: /index.php: Successful login for user 'admin' from: 192.168.1.2
Aug 4 14:05:56 	php-fpm[87057]: /index.php: Successful login for user 'admin' from: 192.168.1.2
Aug 4 14:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 14:02:48 	snmpd[41828]: could not encode error response
Aug 4 14:02:43 	snmpd[41828]: could not encode error response
Aug 4 14:02:38 	snmpd[41828]: could not encode error response
Aug 4 14:02:33 	snmpd[41828]: could not encode error response
Aug 4 14:02:23 	sshd[92330]: Did not receive identification string from 192.168.1.254
Aug 4 14:01:36 	sshd[89702]: Did not receive identification string from 192.168.1.254
Aug 4 14:00:00 	check_reload_status: Syncing firewall
Aug 4 14:00:00 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg ' returned exit code '1', the output was 'SARG: File not found: /var/log/squid/access.log'
Aug 4 14:00:00 	php: sarg.php: Sarg: force refresh now with args, compress(on) and none action after sarg finish.
Aug 4 14:00:00 	php: pfblockerng.php: [pfBlockerNG] Starting sync process.
Aug 4 13:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:50:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:45:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:45:00 	snort[22978]: invalid appid in appStatRecord (2046)
Aug 4 13:40:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:35:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:20:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:11:06 	check_reload_status: Syncing firewall
Aug 4 13:11:06 	php-fpm[3474]: /snort/snort_interfaces_global.php: [Snort] Removed 0 obsoleted rules category files.
Aug 4 13:11:06 	php-fpm[3474]: /snort/snort_interfaces_global.php: [Snort] Hide Deprecated Rules is enabled. Removing obsoleted rules categories.
Aug 4 13:10:01 	php-fpm[31895]: /index.php: Successful login for user 'admin' from: 192.168.1.211
Aug 4 13:10:01 	php-fpm[31895]: /index.php: Successful login for user 'admin' from: 192.168.1.211
Aug 4 13:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:05:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 13:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 13:05:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 13:00:00 	check_reload_status: Syncing firewall
Aug 4 13:00:00 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg ' returned exit code '1', the output was 'SARG: File not found: /var/log/squid/access.log'
Aug 4 13:00:00 	php: pfblockerng.php: [pfBlockerNG] Starting sync process.
Aug 4 13:00:00 	php: sarg.php: Sarg: force refresh now with args, compress(on) and none action after sarg finish.
Aug 4 13:00:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 13:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:53:01 	php-fpm[58210]: /snort/snort_download_rules.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2973.tar.gz...
Aug 4 12:50:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:40:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:35:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:35:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:35:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:30:52 	php: rc.update_urltables: /etc/rc.update_urltables: pfB_PS_v4 does not need updating.
Aug 4 12:30:52 	php: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v6 does not need updating.
Aug 4 12:30:52 	php: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v4 does not need updating.
Aug 4 12:30:52 	php: rc.update_urltables: /etc/rc.update_urltables: Starting URL table alias updates
Aug 4 12:30:33 	php-fpm[58972]: /system_firmware.php: Successful login for user 'admin' from: 192.168.1.100
Aug 4 12:30:33 	php-fpm[58972]: /system_firmware.php: Successful login for user 'admin' from: 192.168.1.100
Aug 4 12:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:30:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:30:00 	php: rc.update_urltables: /etc/rc.update_urltables: Sleeping for 52 seconds.
Aug 4 12:30:00 	php: rc.update_urltables: /etc/rc.update_urltables: Starting up.
Aug 4 12:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:25:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:20:04 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:20:04 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:15:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:15:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:15:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:10:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:05:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:04:45 	snort[6952]: [119:33:1] (http_inspect) UNESCAPED SPACE IN HTTP URI [Classification: Unknown Traffic] [Priority: 3] {TCP} 202.122.17.58:53701 -> 103.18.67.61:8080
Aug 4 12:04:45 	snort[6952]: [119:33:1] (http_inspect) UNESCAPED SPACE IN HTTP URI [Classification: Unknown Traffic] [Priority: 3] {TCP} 202.122.17.58:53701 -> 103.18.67.61:8080
Aug 4 12:04:42 	snort[6952]: [119:33:1] (http_inspect) UNESCAPED SPACE IN HTTP URI [Classification: Unknown Traffic] [Priority: 3] {TCP} 202.122.17.58:4268 -> 103.18.67.61:8080
Aug 4 12:04:42 	snort[6952]: [119:33:1] (http_inspect) UNESCAPED SPACE IN HTTP URI [Classification: Unknown Traffic] [Priority: 3] {TCP} 202.122.17.58:4268 -> 103.18.67.61:8080
Aug 4 12:03:04 	snmpd[41828]: could not encode error response
Aug 4 12:02:59 	snmpd[41828]: could not encode error response
Aug 4 12:02:54 	snmpd[41828]: could not encode error response
Aug 4 12:02:49 	snmpd[41828]: could not encode error response
Aug 4 12:02:39 	sshd[87045]: Did not receive identification string from 192.168.1.254
Aug 4 12:01:36 	sshd[59317]: Did not receive identification string from 192.168.1.254
Aug 4 12:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 12:00:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:00:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 12:00:00 	check_reload_status: Syncing firewall
Aug 4 12:00:00 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg ' returned exit code '1', the output was 'SARG: File not found: /var/log/squid/access.log'
Aug 4 12:00:00 	php: pfblockerng.php: [pfBlockerNG] Starting sync process.
Aug 4 12:00:00 	php: sarg.php: Sarg: force refresh now with args, compress(on) and none action after sarg finish.
Aug 4 11:55:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:55:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:50:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:50:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:50:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:45:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:45:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:40:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:40:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:35:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:35:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:35:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:35:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:30:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:30:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:25:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:25:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:20:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:20:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:20:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:20:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:15:02 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:15:02 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:15:02 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:15:02 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:10:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:10:00 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:05:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:05:01 	snort[22978]: invalid appid in appStatRecord (186)
Aug 4 11:05:01 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:00:00 	snort[22978]: invalid appid in appStatRecord (502)
Aug 4 11:00:00 	snort[22978]: invalid appid in appStatRecord (186)

NOYB

Velbon,

I was able to ssh to get a console session and run top to see that mtree process was consuming nearly 100% processor. On the recommendation of CMB I did restart (menu option 5) and it came up as 2.2.4 with no further noticeable troubles.

Since you cannot ssh then either your situation is different or ssh is not enabled.

This is not a recommendation but just sharing what I would probably do.
Having given it 2 days (I wouldn't have been that patient) I'd reboot and see what happens. Determine the state and decide whether or not to reinstall fresh. Hopefully you did a config backup before starting the upgrade.

NOYB

@Abhishek:

I am also facing same issue , i kept upgrade at 8Am , now its 04:19 PM still not upgrade done

That looks like a different issue. Mine had nothing to do with package reinstallation.