Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Storage detached, semi-catastrophic failure

    Scheduled Pinned Locked Moved Hardware
    15 Posts 3 Posters 964 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimpJ
      jimp Rebel Alliance Developer Netgate
      last edited by

      That is almost certainly from the hardware in some way. That kind of event can't be generated by software, the drives just up and disappeared from the OS.

      It's doubtful that would be from any kind of power saving options but it may be worth checking in the BIOS/Disk controller to be sure.

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      A 1 Reply Last reply Reply Quote 0
      • JonathanLeeJ
        JonathanLee
        last edited by JonathanLee

        Do you have a new drive?? That looks like the HDD crashed.

        Side note that has nothing to do with this: years ago, I was working on a pbx and had to turn it off to replace a card, it would not turn on again. I was told by my support that I had to put the HDD in the freezer for 20 mins to make the metal contract so the motor could spin up the plates again....

        It had been running constantly for 20 years, and had never had an issue until I had to replace that part.

        Ran like a champ again for years after.

        I was told that some of the older drives are never meant to be turned off because they can't turn back on afterwards after running non stop for years, the heads would stick when it stopped.

        No lies that fixed the PBX too. I was thinking what in the world the whole time, should I stand on one leg and pat my tummy too... No lies that freezer thing worked in the end.

        Make sure to upvote

        1 Reply Last reply Reply Quote 0
        • JonathanLeeJ
          JonathanLee @AlexanderL
          last edited by JonathanLee

          @AlexanderL does it use something like PUIS mode enabled?

          Make sure to upvote

          A 1 Reply Last reply Reply Quote 0
          • A
            AlexanderL
            last edited by

            After a reboot, the drives were up and running again like nothing had happened, a scrub showed no inconsistencies.

            1 Reply Last reply Reply Quote 1
            • A
              AlexanderL @jimp
              last edited by AlexanderL

              @jimp

              Yes, hardware, firmware or low level OS related at least. The system is not old, we built it in the beginning of August so not more than 4½ months. Our theories so far goes:

              • Controller failure: Not very likely, they are on the chipset controller (Intel C256) and we do not expect any stability issues with it

              • Drive failure:As a reboot fixed it, both drives were healthy and zfs did not find any inconsistencies after a scrub, we don't suspect drive hardware failure. Maybe some oddity with the firmware, like a counter overflowing or similar, but again, these are enterprise PM893:s, we don't expect it

              • OS: Given the info from the two posts I linked, and due to the two drives going down at the same time, this is perhaps the most plausible reason. FreeBSD not liking something with this specific combination of hardware

              We'll check the BIOS and also plan to add a third drive to the mirror from another brand to see if this makes any difference.

              Thanks!

              1 Reply Last reply Reply Quote 0
              • A
                AlexanderL @JonathanLee
                last edited by

                @JonathanLee

                See my first reply, the system is up and running after a reboot.

                No HDDs, these are SSDs, so PUIS would not apply here. Other power saving functionality might though, I'll see what I can find.

                Thanks!

                JonathanLeeJ 1 Reply Last reply Reply Quote 1
                • JonathanLeeJ
                  JonathanLee @AlexanderL
                  last edited by

                  @AlexanderL Hows the temperature on the SSDs? They might be self shutting down for protection. Did you use a thermal pad to dissipate the SSD heat? I was reading a lot of amazon reviews on M.2 SATA SSD USB enclosures and they have a lot of complaints of how they get really really hot, plus most of the enclosures are made of aluminium and come with a thermal pad. I understand this is not a USB enclosure however my 2100 has no fan, if your running multiple SSDs heat might cause some issues. They run hot.

                  Make sure to upvote

                  A 1 Reply Last reply Reply Quote 0
                  • A
                    AlexanderL @JonathanLee
                    last edited by

                    @JonathanLee

                    They're ok, just ran smartctl -x on both drives and everything looks top. Max temp is 40degC, averaging around 30 for both. These are 2.5" drives, not M.2 so more area to spread the heat!

                    BUT, the firewall went down the same way just a few hours ago, so something is definitely going on, no more info in the current logs though.

                    I'm a bit new to FreeBSD but reasonably home in Linux. The documentation points to /etc/syslog.conf to handle logging levels and files, but it is more or less empty so I assume PfSense uses a different approach to logging? Do you know how I would go about getting more verbose logging enabled?

                    JonathanLeeJ 1 Reply Last reply Reply Quote 0
                    • JonathanLeeJ
                      JonathanLee @AlexanderL
                      last edited by

                      @AlexanderL

                      I know you can set up remote logging and point it to something else under status / system logs / settings

                      Screenshot 2023-12-22 at 11.05.59 AM.png

                      Make sure to upvote

                      A 1 Reply Last reply Reply Quote 0
                      • JonathanLeeJ
                        JonathanLee
                        last edited by

                        Also smart status

                        Screenshot 2023-12-22 at 11.07.54 AM.png

                        View Logs.

                        Make sure to upvote

                        1 Reply Last reply Reply Quote 0
                        • A
                          AlexanderL @JonathanLee
                          last edited by

                          @JonathanLee

                          Thanks, yes, been there, the S.M.A.R.T page offers basically the same as smartctl -x produces, and sadly, the system log settings does not offer any granular control over things like kernel log levels and such. I'll have to dig deeper!

                          JonathanLeeJ 1 Reply Last reply Reply Quote 0
                          • JonathanLeeJ
                            JonathanLee @AlexanderL
                            last edited by

                            @AlexanderL You might need a terminal session for some of them, or use edit if you know the file path in gui.

                            Make sure to upvote

                            A 1 Reply Last reply Reply Quote 0
                            • A
                              AlexanderL @JonathanLee
                              last edited by

                              @JonathanLee

                              Yes, I've been snooping around via SSH a bit already, I know where the log files are, but the syslog does not contain anything more than what is already logged to the console. I need to find the correct method to increase the log level without risking breaking something in PfSense.

                              JonathanLeeJ 1 Reply Last reply Reply Quote 0
                              • JonathanLeeJ
                                JonathanLee @AlexanderL
                                last edited by

                                @AlexanderL Good luck.

                                Make sure to upvote

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.