Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Power loss hell. Help with fsck and corrupted drive

    Scheduled Pinned Locked Moved General pfSense Questions
    7 Posts 3 Posters 3.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      tnine
      last edited by

      Hi all,
       We're recently had a slew of power failures at our office.  It's a small company, so we don't have a UPC yet.  Unfortunately one of the side effects of this is a corrupt file system.  Thankfully, we run pfSense off a USB drive, and I have 2 backups.  I was able to start the backup, then force mount the corrupt file system on the old USB and copy over the config.  Now that I'm back up and running, this leaves me with 2 questions.

      1. Fschk doesn't appear to automatically run when the system recovers from an unclean mount.  Is it possible to turn this behavior on?  I think that if fsck had been run after every power loss, the drive wouldn't have progressed to a state that is too corrupt to recover.

      2. If I set up a cron job to scp our config file nightly to a backup server, will the correctly retain all of our settings? I.E, is it functionally equivalent to downloading the config via the web interface?

      Thanks,
      Todd

      1 Reply Last reply Reply Quote 0
      • C
        clarknova
        last edited by

        @tnine:

        1. Fschk doesn't appear to automatically run when the system recovers from an unclean mount.  Is it possible to turn this behavior on?  I think that if fsck had been run after every power loss, the drive wouldn't have progressed to a state that is too corrupt to recover.

        Any reason you're not running the embedded version? It mounts the filesystem read-only so a sudden poweroff won't touch the fs.

        2. If I set up a cron job to scp our config file nightly to a backup server, will the correctly retain all of our settings? I.E, is it functionally equivalent to downloading the config via the web interface?

        I believe this depends again on whether you're using the embedded or full version, but somebody else will know for sure.

        db

        1 Reply Last reply Reply Quote 0
        • T
          tnine
          last edited by

          1. We don't have an external syslog server (yet), so I want to keep logging on our current system.

          2. I wrote a nightly backup script that uses wget to download the config file from the web interface.  I've attached it.  This works with version 1.2.3.

          router.txt

          1 Reply Last reply Reply Quote 0
          • C
            clarknova
            last edited by

            I don't see why an scp script on cron wouldn't work to backup your config file. It makes me wonder what that would gain you over the wget script you have now.

            A cron job to make periodic scp dumps of the logs could also free you up to move to embedded without giving up a persistent log, although admittedly you would be missing a chunk of logs since the last transfer if the system went down unexpectedly.

            db

            db

            1 Reply Last reply Reply Quote 0
            • T
              tnine
              last edited by

              Is it possible to change an existing installation to use the appliance install?  If not, how is the install process different than booting from CD then installing to a USB drive?  I didn't see any options that gave me a choice when installing from the CD.

              1 Reply Last reply Reply Quote 0
              • jimpJ
                jimp Rebel Alliance Developer Netgate
                last edited by

                @tnine:

                Hi all,
                 We're recently had a slew of power failures at our office.  It's a small company, so we don't have a UPC yet.  Unfortunately one of the side effects of this is a corrupt file system.  Thankfully, we run pfSense off a USB drive, and I have 2 backups.  I was able to start the backup, then force mount the corrupt file system on the old USB and copy over the config.  Now that I'm back up and running, this leaves me with 2 questions.

                A cheap UPS can be had for ~$50. There's no excuse not to have one on a critical system, even if it only lasts for a few minutes, combined with the NUT package it can signal your system to shutdown cleanly.

                1. Fschk doesn't appear to automatically run when the system recovers from an unclean mount.  Is it possible to turn this behavior on?  I think that if fsck had been run after every power loss, the drive wouldn't have progressed to a state that is too corrupt to recover.

                It most definitely does run fsck on bootup, and fixes automatically. I've watched it happen several times, in testing and in live environments. Unless the drive is extremely corrupt, it always boots back up.

                2. If I set up a cron job to scp our config file nightly to a backup server, will the correctly retain all of our settings? I.E, is it functionally equivalent to downloading the config via the web interface?

                That should work, or you could cron a wget from a different box. There are examples on the Doc Wiki.

                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                Need help fast? Netgate Global Support!

                Do not Chat/PM for help!

                1 Reply Last reply Reply Quote 0
                • C
                  clarknova
                  last edited by

                  @tnine:

                  Is it possible to change an existing installation to use the appliance install?

                  I couldn't find anything on that in the docs. If I were to try, this is what I would do. Again, I've never tried and I definitely don't guarantee it will not destroy your existing installation.

                  
                  echo embedded > /etc/platform
                  
                  

                  and then use the firmware update page in the GUI to install the embedded image.

                  As for a new install of embedded, this is not done using the Live CD. You have to download the embedded image then use dd or physdiskwrite to write the image directly to the boot device. These instructions are what you're looking for:
                  http://m0n0.ch/wall/installation_generic.php

                  db

                  db

                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post
                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.