Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-1100 update from 23.09.1 to 24.03 keeps failing

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    79 Posts 8 Posters 7.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mrneutron @SteveITS
      last edited by

      @SteveITS said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

      @mrneutron The RAM disk setting requires a restart to turn on.

      The log file will always have current timestamps. It's just a matter of where they are being saved.

      If you are concerned about logs, you may want to put a number in that Log Directory field, for instance, to copy logs to eMMC storage every "n" hours. Otherwise they are copied during a clean halt/restart.

      if on it will show in the dashboard Disks widget:
      ed4a7691-aeeb-4fe4-a27e-769be2d90286-image.png

      @SteveITS, right now, I'm trying to determine if my emmc chip is still working correctly. Thus, I'm trying to determine if any files are being successfully written to it.

      S 1 Reply Last reply Reply Quote 0
      • S
        SteveITS Galactic Empire @mrneutron
        last edited by

        @mrneutron Ah, I see. IIRC you need to restart to "lose" files with ZFS(?) since hides it due to caching. Try creating a file and restarting, to see if it's still there.

        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
        Upvote 👍 helpful posts!

        1 Reply Last reply Reply Quote 0
        • M
          mrneutron
          last edited by

          I used the touch command to create files in the / directory and /var/log directories.
          Both files were gone after a normal reboot.
          I guess this is more evidence that the emmc is read-only, now?

          I read that the mount command is supposed to show if the mounted partitions are ro or rw. It doesn't show ro or rw for any of them.

          [23.09.1-RELEASE][admin@redacted]/: mount
          pfSense/ROOT/default on / (zfs, local, noatime, nfsv4acls)
          devfs on /dev (devfs)
          pfSense/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
          pfSense/home on /home (zfs, local, noatime, nfsv4acls)
          pfSense/var on /var (zfs, local, noatime, nfsv4acls)
          pfSense/var/cache on /var/cache (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          pfSense/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)
          pfSense/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          pfSense/var/db on /var/db (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          pfSense/ROOT/default/cf on /cf (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          pfSense/ROOT/default/var_cache_pkg on /var/cache/pkg (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          pfSense/ROOT/default/var_db_pkg on /var/db/pkg (zfs, local, noatime, noexec, nosuid, nfsv4acls)
          tmpfs on /var/run (tmpfs, local)
          devfs on /var/dhcpd/dev (devfs)
          
          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by stephenw10

            I don't thing mount would show it. They are not mounted read-only. mount -p should show it though if they were.

            1 Reply Last reply Reply Quote 0
            • M
              mrneutron
              last edited by

              Huh...all the Linux examples I saw of the mount command had only mount (no -p).
              Is BSD usage different?

              Using mount -p they are all supposedly mounted rw, but they don't hold what I write to them.

              [23.09.1-RELEASE][admin@redacted]/: mount -p
              pfSense/ROOT/default    /                       zfs     rw,noatime,nfsv4acls    0 0
              devfs                   /dev                    devfs   rw              0 0
              pfSense/tmp             /tmp                    zfs     rw,nosuid,noatime,nfsv4acls     0 0
              pfSense/home            /home                   zfs     rw,noatime,nfsv4acls    0 0
              pfSense/var             /var                    zfs     rw,noatime,nfsv4acls    0 0
              pfSense/var/cache       /var/cache              zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              pfSense/var/tmp         /var/tmp                zfs     rw,nosuid,noatime,nfsv4acls     0 0
              pfSense/var/log         /var/log                zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              pfSense/var/db          /var/db                 zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              pfSense/ROOT/default/cf /cf                     zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              pfSense/ROOT/default/var_cache_pkg /var/cache/pkg               zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              pfSense/ROOT/default/var_db_pkg /var/db/pkg             zfs     rw,noexec,nosuid,noatime,nfsv4acls      0 0
              tmpfs                   /var/run                tmpfs   rw              0 0
              devfs                   /var/dhcpd/dev          devfs   rw              0 0
              
              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                The -p switch cause it to print the output in fstab format and includes verbose info.
                https://man.freebsd.org/cgi/man.cgi?query=mount

                Yes so ZFS think it's all mounted read-write but that doesn't help if the actual storage is read-only.

                1 Reply Last reply Reply Quote 0
                • M
                  mrneutron
                  last edited by

                  Yeah, it sure looks like the emmc in my SG-1100 is no longer writable (upgradable).
                  It still boots in the configuration I had it set in, but I wonder how long before it goes non-functional?

                  "Certain types of disks, such as SSD and eMMC disks, may fail into a read only state where disk writes fail or are discarded, but data can still be read."

                  from: https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Yes unfortunately I have to agree. You can try installing to USB. I tested that yesterday.

                    M 1 Reply Last reply Reply Quote 0
                    • M
                      mrneutron @stephenw10
                      last edited by

                      @stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

                      Yes unfortunately I have to agree. You can try installing to USB. I tested that yesterday.

                      You can run the pfsense OS off of a USB flash drive, plugged into the SG-1100?

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Yes, the new Net Installer allows you select another USB drive as the target.

                        M 1 Reply Last reply Reply Quote 0
                        • M
                          mrneutron @stephenw10
                          last edited by

                          @stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

                          Yes, the new Net Installer allows you select another USB drive as the target.

                          Can you use either USB port on the SG-1100? (like the USB3 port?)
                          Does loading into a USB flash drive automatically set the boot config to point to the USB flash drive as the boot drive?
                          If not, how do you manually edit the boot config?

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Technically you could use either port and the USB3 port would likely be much faster. However when you run either usbrecovery or usbboot it boots the first USB device and if you have two drives inserted that is the USB3 port. Which means that when you boot the installer the target drive must be in the USB2 slot. That shouldn't matter, I'd expect to be able to move the drive after installing. However I was unable to do so. The 1100 can be picky about booting USB drives though so it could just be the drive I'm using.

                            The default boot device is not automatically changed. You need to set the bootcmd uboot env from the uboot prompt like:

                            setenv bootcmd 'run usbboot; run emmcboot;'
                            saveenv
                            

                            Now in your case you may have an additional issue that the emmc is still holding a ZFS filesystem and cannot be wiped. If that is that case you'd need to either make sure the zfs pool names are different or use UFS on the USB drive.

                            M 1 Reply Last reply Reply Quote 0
                            • S SteveITS referenced this topic on
                            • M
                              mrneutron @stephenw10
                              last edited by

                              @stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

                              Technically you could use either port and the USB3 port would likely be much faster. However when you run either usbrecovery or usbboot it boots the first USB device and if you have two drives inserted that is the USB3 port. Which means that when you boot the installer the target drive must be in the USB2 slot. That shouldn't matter, I'd expect to be able to move the drive after installing. However I was unable to do so. The 1100 can be picky about booting USB drives though so it could just be the drive I'm using.

                              The default boot device is not automatically changed. You need to set the bootcmd uboot env from the uboot prompt like:

                              setenv bootcmd 'run usbboot; run emmcboot;'
                              saveenv
                              

                              Now in your case you may have an additional issue that the emmc is still holding a ZFS filesystem and cannot be wiped. If that is that case you'd need to either make sure the zfs pool names are different or use UFS on the USB drive.

                              Yeah, given the read-only status of the emmc, it definitely appears the file system can't be wiped.
                              Using UFS on a USB flash drive seems like a less-complicated workaround - less other stuff to change.
                              How do you change the ZFS pool names, if you want/need to?

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Yes, good question! There isn't an option to do that in the Net Installer TUI. Yet.

                                The legacy x86 installer allows setting the pool name. The legacy aarch64 installer never did because it used a prebuilt image.

                                So for now using UFS is the only option if you cannot remove a ZFS pool named 'pfSense' from the eMMC.

                                However I would argue that when running from USB the drive writes are even more important and running UFS with RAMdisks minimises that. So I would run UFS anyway.

                                Steve

                                F 1 Reply Last reply Reply Quote 0
                                • F
                                  flatsphere @stephenw10
                                  last edited by

                                  @stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

                                  Yes, good question! There isn't an option to do that in the Net Installer TUI. Yet.

                                  The legacy x86 installer allows setting the pool name. The legacy aarch64 installer never did because it used a prebuilt image.

                                  So for now using UFS is the only option if you cannot remove a ZFS pool named 'pfSense' from the eMMC.

                                  However I would argue that when running from USB the drive writes are even more important and running UFS with RAMdisks minimises that. So I would run UFS anyway.

                                  Steve

                                  Hi gang,

                                  Is there a resolution to this problem yet? My SG-1100 is having the same problem.

                                  Cheers!
                                  Rafael

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    So you see the 8 disks shown by uboot?

                                    F 1 Reply Last reply Reply Quote 0
                                    • F
                                      flatsphere @stephenw10
                                      last edited by

                                      @stephenw10 Hi Stephen,

                                      The latest attempt shows that I'm running out of memory.
                                      mount -p shoes this:

                                      eb417928-7298-46d6-99f4-fea3b7b8d563-image.png

                                      Following notices also appeared:

                                      Upgrade
                                      pkg (3): out of space @ 2024-05-15 09:06:30
                                      PHP errors
                                      PHP ERROR: Type: 1, File: /usr/local/pkg/pfblockerng/pfblockerng.inc, Line: 8804, Message: Uncaught TypeError: Unsupported operand types: int + string in /usr/local/pkg/pfblockerng/pfblockerng.inc:8804
                                      Stack trace:
                                      #0 /usr/local/www/pfblockerng/pfblockerng.php(628): sync_package_pfblockerng('cron')
                                      #1 /usr/local/www/pfblockerng/pfblockerng.php(155): pfblockerng_sync_cron()
                                      #2 {main}
                                      thrown @ 2024-05-15 10:00:35

                                      And following disk usage:

                                      ee5c8740-6858-4170-9aef-52531abf7870-image.png

                                      Cheers!
                                      Rafael

                                      S 1 Reply Last reply Reply Quote 0
                                      • S
                                        SteveITS Galactic Empire @flatsphere
                                        last edited by

                                        @flatsphere given the 2.1 GB you probably have old boot environments to delete:
                                        https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-shrink.html

                                        Also if your RAM disk is full consider disabling that.

                                        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                        Upvote 👍 helpful posts!

                                        F 1 Reply Last reply Reply Quote 0
                                        • F
                                          flatsphere @SteveITS
                                          last edited by

                                          @SteveITS said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

                                          https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-shrink.html

                                          I deleted all but the current boot environment, turned off VRAM and still failed.

                                          ae07ff9b-be0b-4e39-aafa-630f306d8b0e-image.png

                                          R

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Yeah you will either need bigger RAM disks or to just disable them for the update. I would disable them.

                                            However this looks like a completely different error. It's not indicative of a failed eMMC. At least not what you've shown us so far.

                                            F 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.