Upgrade failing - 2.4.5-RELEASE-p1 (arm64) to 21.02.2
-
Environment
SG-1100 with the following plugins:
- iperf
- mtr-nox11
- nmap
- openvpn-client-export
- Service_Watchdog
- Status_Traffic_Totals
Issue
Upgrade will not complete from GUI or when manually running
pkg-static update -f
How to reproduce
Running a normal upgrade process from the GUI hangs forever. The logs show that pkg-static is repeatedly failing with messages like the following.
pid 11470 (pkg-static), jid 0, uid 0: exited on signal 11 (core dumped)
I tried to run the upgrade by running
pkg-static update -f
from Diagnostics --> Command Prompt --> Execute Shell Command. This is the output that I see.Updating pfSense-core repository catalogue... 1082884096:error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib:/usr/local/poudriere/jails/pfSense_plus-v21_02_aarch64/usr/src/crypto/openssl/ssl/statem/statem_lib.c:283: Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/CN=repo00.netgate.com 1082884096:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/usr/local/poudriere/jails/pfSense_plus-v21_02_aarch64/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1915: Child process pid=78396 terminated abnormally: Segmentation fault
An attempt to reset the repo data did not work either. I tried to run
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
from Diagnostics --> Command Prompt --> Execute Shell Command. This is the output that I see.Updating pfSense-core repository catalogue... 1082884096:error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib:/usr/local/poudriere/jails/pfSense_plus-v21_02_aarch64/usr/src/crypto/openssl/ssl/statem/statem_lib.c:283: Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/CN=repo00.netgate.com 1082884096:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/usr/local/poudriere/jails/pfSense_plus-v21_02_aarch64/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1915: Child process pid=57048 terminated abnormally: Segmentation fault
-
@masons said in Upgrade failing - 2.4.5-RELEASE-p1 (arm64) to 21.02.2:
Segmentation fault
See this workaround.
-
Hi @steveits,
Thanks for the suggestion. The details of that linked article do look different than what I'm seeing. However, I'll schedule a time to go through the process of bringing the network down and see if it resolves the issue. I'll update this thread with the results.
-
I followed the process of running Halt, removing power and then powering up the device. I was then able to successfully apply the update. I should note that the update available was no longer 21.02.2, but 21.05.
Unfortunately, the IPSEC VPN that was in place didn't come back up.... :/ From the other side of the tunnel it looks like there's a problem negotiating the tunnel. I'm working on troubleshooting this issue now.
Jun 4 12:04:32 charon 7435 08[IKE] <con1000|81> IKE_SA con1000[81] state change: CONNECTING => DESTROYING Jun 4 12:04:32 charon 7435 08[CFG] <con1000|81> configured proposals: IKE:AES_GCM_16_128/PRF_AES128_XCBC/MODP_2048 Jun 4 12:04:32 charon 7435 08[IKE] <con1000|81> received NO_PROPOSAL_CHOSEN notify error Jun 4 12:04:32 charon 7435 08[ENC] <con1000|81> parsed IKE_SA_INIT response 0 [ N(NO_PROP) ] Jun 4 12:04:32 charon 7435 08[NET] <con1000|81> received packet: from xxx.xxx.xxx.xxx[500] to xxx.xxx.xxx.xxx[500] (36 bytes) Jun 4 12:04:32 charon 7435 10[NET] <con1000|81> sending packet: from xxx.xxx.xxx.xxx[500] to xxx.xxx.xxx.xxx[500] (456 bytes) Jun 4 12:04:32 charon 7435 10[ENC] <con1000|81> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ] Jun 4 12:04:32 charon 7435 10[CFG] <con1000|81> sending supported signature hash algorithms: sha256 sha384 sha512 identity Jun 4 12:04:32 charon 7435 10[CFG] <con1000|81> configured proposals: IKE:AES_GCM_16_128/PRF_AES128_XCBC/MODP_2048 Jun 4 12:04:32 charon 7435 10[IKE] <con1000|81> IKE_SA con1000[81] state change: CREATED => CONNECTING Jun 4 12:04:32 charon 7435 10[IKE] <con1000|81> initiating IKE_SA con1000[81] to xxx.xxx.xxx.xxx
-
Well that was strange... On the SG-1100, I reviewed the IPSEC VPN config and it all looked right, but the tunnel still wouldn't come up. So, I just re-saved the phase 1 and phase 2 configs and then the tunnel came up.
The moral of the story is: power cycle before applying this update and be ready to kick your IPSEC VPN.