Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-1100 Upgrade 23.09.1 to 24.03_1 failure

    Scheduled Pinned Locked Moved
    Problems Installing or Upgrading pfSense Software
    4
    17
    255
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      manzato
      last edited by

      I've tried several times to upgrade my router but it fails every time in the same place.

      Here is the partial log:

      >>> Updating repositories metadata...
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ....... done
Processing entries: .......... done
pfSense repository update completed. 710 packages processed.
All repositories are up to date.
>>> Locking package pkg...done.
>>> Renaming current boot environment from default to default_20241102013231...done.
>>> Cloning current boot environment default_20241102013231...done.
>>> Removing vital flag from php82...done.
>>> Upgrading packages in cloned boot environment default...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking for upgrades (126 candidates): .......... done
Processing candidates (126 candidates): .......... done
The following 231 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
	aws-sdk-php82: 3.273.3
	pfSense-Status_Monitoring-php82: 1.8_3
	php82: 8.2.11
	php82-bcmath: 8.2.11
	php82-bz2: 8.2.11
	php82-ctype: 8.2.11
	php82-curl: 8.2.11
................
[7/231] Reinstalling libiconv-1.17...
[7/231] Extracting libiconv-1.17: .......... done
[8/231] Upgrading python311 from 3.11.4 to 3.11.7...
[8/231] Extracting python311-3.11.7: .....
[8/231] Extracting python311-3.11.7... done
Failed

      Any pointers?

      S 1 Reply Last reply Reply Quote 0
      • S
        SteveITS Rebel Alliance @manzato
        last edited by SteveITS

        @manzato Check if the disk is full, particularly old boot environments.

        ...weird it posted as I was typing.

        https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-shrink.html

        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
        Upvote 👍 helpful posts!

        M 1 Reply Last reply Reply Quote 0
        • M
          manzato @SteveITS
          last edited by

          @SteveITS it looks like there is enough space:

          [23.09.1-RELEASE][root@pfSense.localdomain]/root: zfs list 
NAME                                 USED  AVAIL  REFER  MOUNTPOINT
pfSense                             1.60G  5.18G    96K  none
pfSense/ROOT                         936M  5.18G    96K  none
pfSense/ROOT/default                 936M  5.18G   929M  /
pfSense/ROOT/default/cf             1.47M  5.18G  1.47M  /cf
pfSense/ROOT/default/var_cache_pkg    96K  5.18G    96K  /var/cache/pkg
pfSense/ROOT/default/var_db_pkg     5.36M  5.18G  5.36M  /var/db/pkg
pfSense/home                          96K  5.18G    96K  /home
pfSense/reservation                   96K  5.86G    96K  none
pfSense/tmp                         2.30M  5.18G  2.30M  /tmp
pfSense/var                         2.75M  5.18G   388K  /var
pfSense/var/cache                     96K  5.18G    96K  /var/cache
pfSense/var/db                      1.28M  5.18G  1.28M  /var/db
pfSense/var/log                      912K  5.18G   912K  /var/log
pfSense/var/tmp                      112K  5.18G   112K  /var/tmp
[23.09.1-RELEASE][root@pfSense.localdomain]/root: zfs list -t snapshot
no datasets available
[23.09.1-RELEASE][root@pfSense.localdomain]/root: df -hi
Filesystem                            Size    Used   Avail Capacity iused ifree %iused  Mounted on
pfSense/ROOT/default                  6.1G    929M    5.2G    15%     53k   11M    0%   /
devfs                                 1.0K      0B    1.0K     0%       0     0     -   /dev
pfSense/var                           5.2G    388K    5.2G     0%     103   11M    0%   /var
pfSense/tmp                           5.2G    2.3M    5.2G     0%      87   11M    0%   /tmp
pfSense/home                          5.2G     96K    5.2G     0%       7   11M    0%   /home
pfSense/var/log                       5.2G    912K    5.2G     0%      52   11M    0%   /var/log
pfSense/var/tmp                       5.2G    112K    5.2G     0%      14   11M    0%   /var/tmp
pfSense/var/cache                     5.2G     96K    5.2G     0%       8   11M    0%   /var/cache
pfSense/var/db                        5.2G    1.3M    5.2G     0%      45   11M    0%   /var/db
pfSense/ROOT/default/cf               5.2G    1.5M    5.2G     0%      49   11M    0%   /cf
pfSense/ROOT/default/var_cache_pkg    5.2G     96K    5.2G     0%       7   11M    0%   /var/cache/pkg
pfSense/ROOT/default/var_db_pkg       5.2G    5.4M    5.2G     0%      14   11M    0%   /var/db/pkg
tmpfs                                 4.0M    132K    3.9M     3%      42   14k    0%   /var/run
[23.09.1-RELEASE][root@pfSense.localdomain]/root:
          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Try running the upgrade from the command line. That gives more debug info and it also frees some resources by not having the webgui open:

            pfSense-uprade -d

            M 1 Reply Last reply Reply Quote 0
            • M
              manzato @stephenw10
              last edited by

              @stephenw10 said in SG-1100 Upgrade 23.09.1 to 24.03_1 failure:

              stephenw10
              NETGATE
              ADMINISTRATOR
              about 3 hours ago

              Try running the upgrade from the command line. That gives more debug info and it also frees some resources by not having the webgui open:

              pfSense-uprade -d

              Reply Quote
              0

              Ran pfSense-upgrade -d and arrived to the same issue (lost the log), and then pfSense-upgrade -d -R to prevent auto boot:

              	pfSense-u-boot-2100-20210930_1 [pfSense] (ABI changed: 'freebsd:14:*' -> 'freebsd:15:*')
	pfSense-u-boot-env-20230123 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	pkcs11-helper-1.29.0 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	py311-libzfs-1.1.2023020700 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	py311-setuptools-63.1.0_1 [pfSense] (ABI changed: 'freebsd:14:*' -> 'freebsd:15:*')
	qstats-0.2 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	radvd-2.19_2 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	rate-0.9_2 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	rrdtool-1.8.0_2 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	scponly-4.8.20110526_5 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	smartmontools-7.4 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	ssh_tunnel_shell-0.2_1 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	uclcmd-0.2.20211204 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	voucher-0.1_3 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	vstr-1.0.15_1 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	whois-5.5.7 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	wol-0.7.1_4 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	xinetd-2.3.15_2 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	zip-3.0_1 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')
	zstd-1.5.5 [pfSense] (ABI changed: 'freebsd:14:aarch64:64' -> 'freebsd:15:aarch64:64')

Number of packages to be removed: 51
Number of packages to be installed: 55
Number of packages to be upgraded: 47
Number of packages to be reinstalled: 78

The operation will free 32 MiB.
[1/231] Reinstalling indexinfo-0.3.1...
[1/231] Extracting indexinfo-0.3.1: .... done
[2/231] Reinstalling mpdecimal-2.5.1...
[2/231] Extracting mpdecimal-2.5.1: .......... done
[3/231] Upgrading gettext-runtime from 0.22_1 to 0.22.3...
[3/231] Extracting gettext-runtime-0.22.3: .......... done
[4/231] Upgrading readline from 8.2.1 to 8.2.7...
[4/231] Extracting readline-8.2.7: .......... done
[5/231] Reinstalling libffi-3.4.4...
[5/231] Extracting libffi-3.4.4: .......... done
[6/231] Reinstalling libunistring-1.1...
[6/231] Extracting libunistring-1.1: .......... done
[7/231] Reinstalling libiconv-1.17...
[7/231] Extracting libiconv-1.17: .......... done
[8/231] Upgrading python311 from 3.11.4 to 3.11.7...
[8/231] Extracting python311-3.11.7: ......
[8/231] Extracting python311-3.11.7... done
[23.09.1-RELEASE][root@pfSense.localdomain]/root:
              1 Reply Last reply Reply Quote 0
              • M
                manzato
                last edited by

                I've upgraded the pkg tool (via pkg upgrade pkg) and giving it another try to the upgrade.

                M 1 Reply Last reply Reply Quote 0
                • M
                  manzato @manzato
                  last edited by

                  Managed to get a core dump when running the upgrade...

                  [176/180] Fetching expat-2.5.0.pkg: . done
[177/180] Fetching php83-sysvsem-8.3.1.pkg: . done
[178/180] Fetching php83-pecl-rrd-2.0.3.pkg: . done
[179/180] Fetching py311-libzfs-1.1.2023020700.pkg: ...... done
[180/180] Fetching sshguard-2.4.3_2,1.pkg: .......... done
Checking integrity...Bus error (core dumped)
                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Hmm, anything in the system logs when either of those fail?

                    Check for coredumps in /root. What actually dumped?

                    M 1 Reply Last reply Reply Quote 0
                    • M
                      manzato @stephenw10
                      last edited by

                      @stephenw10 I didn't find any core dumps... And actually the upgrade of pkg was "downgraded" after a reboot. I am able to write to the filesystem (tried writing to most of them).

                      Without the upgraded pkg, it isn't core dumping anymore, but still the upgrade fails and I can't find any smoking gun... no logs, nothing suspicious.

                      Any pointers / ideas?

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Try rebooting and then running the upgrade at the console without logging into the webgui so the maximum resources are available.

                        Otherwise you can always install 24.03 clean and restore the config.

                        M 1 Reply Last reply Reply Quote 0
                        • M
                          manzato @stephenw10
                          last edited by

                          @stephenw10 Tried the clean boot without the GUI, same issue. Where can I download the fresh install of 24?

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Get the Net Installer as shown here:
                            https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              manzato @stephenw10
                              last edited by

                              @stephenw10

                              I ran a fresh installation and got another failure:

                              519bed21-2380-46e7-a7a4-5b9e3dab17eb-image.png

                              I think I can assume there is a HW error on my device at this point.

                              juanzelliJ 1 Reply Last reply Reply Quote 0
                              • juanzelliJ
                                juanzelli @manzato
                                last edited by

                                @manzato Just checking to see if you tried a different USB stick too 😉

                                Netgate 4100 and Aruba IO network at home

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, that does seem like it could be a storage issue.

                                  M 1 Reply Last reply Reply Quote 0
                                  • M
                                    manzato @stephenw10
                                    last edited by

                                    @stephenw10 is there anything I can try to confirm it? Is it serviceable in any way? Or is it going to the trash can?

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Well I would try several times at least. And I would try using several USB sticks.

                                      When it fails like that you can drop to the shell and check what the exact errors are.

                                      I would start by just running: dmesg

                                      If it's a drive issue you will see a bunch of errors show there.

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post