Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?

[MarekAndreansky]

Had a power loss and the box won't boot up. Checking via serial cable shows its in a boot loop.

This article describes it can be an issue of the UFS filesystem - https://teklager.se/en/knowledge-base/pfsense-ufs-filesystem-corruption-after-power-loss/

Do you know if UFS can still be corrupted if a sudden power down happens at the incorrect time?

Attaching the log. serial console output.txt

Unless anyone can recommend a fix, my course of action will be to reinstall. But I will use ZFS instead.

[heper]

@marekandreansky
https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-check.html

don't bother with zfs, get a small UPS instead

[MarekAndreansky]

@heper Thanks, did find that article but space was not working when I tried it.

Using an apu box, and can get into the boot menu via F10, but then it loads Pfsense and starts looping, then restarts.

Will retry space again, see if I need to press it sooner.

And why not use ZFS? Any performance impact or other risks?

And can the file system check help restore the box?

A UPS would not help in my case, as user A did not have internet due to network going down on provider P, and colleague B told A to power down the box. When I got to A it was too late, box was already down and the network did not come up as the box was corrupted.

And sadly can't lock the box away. And honestly, you can imagine how happy I was that they restarted the router when it was a provider issue...

[bingo600]

@marekandreansky
ZFS is more robust wrt. powerfailure.
Change to ZFS is a good thing.

Adding an UPS is a better choice , but then comes battery maintenance etc.

If no UPS , ZFS is the best choice.
In fact it is even if you have an UPS

Edit:
ZFS wont prevent corruption 100% , it's just better to survive it than UFS

[MarekAndreansky]

@bingo600 said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

@marekandreansky
ZFS is more robust wrt. powerfailure.
Change to ZFS is a good thing.

Adding an UPS is a better choice , but then comes battery maintenance etc.

If no UPS , ZFS is the best choice.
In fact it is even if you have an UPS

Thanks, why is ZFS a better choice? One point it gets is it can survive a power failure. Any other benefits? And are there drawbacks compared to UFS?

[bingo600]

@marekandreansky
ZFS was designed as an embeded filesystem with flash & corruption in mind. Look it up ...

Edit: When you install ZFS , and it complains ....
Remember you have to select minimum 1 disk for ZFS use.

So it requires 2 more clicks than UFS , during install

[MarekAndreansky]

@bingo600 said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

@marekandreansky
ZFS was designed as an embeded filesystem with flash & corruption in mind. Look it up ...

Will do. Any idea why its not the default option for pfSense?

[bingo600]

@marekandreansky said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

@bingo600 said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

@marekandreansky
ZFS was designed as an embeded filesystem with flash & corruption in mind. Look it up ...

Will do. Any idea why its not the default option for pfSense?

My guess ...

Noone dared to change the default

[bingo600]

@MarekAndreansky
When you install ZFS , and it complains ....
Remember you have to select minimum 1 disk for ZFS use.

So it requires 2 more clicks than UFS , during install

[MarekAndreansky]

@heper said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

@marekandreansky
https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-check.html

don't bother with zfs, get a small UPS instead

The file system check worked, needed to press space after the boot menu and it allowed me to activate the console as a single user.

After that, checking the filesystem six times and rebooting booted up the box. Thank you!

Unrelated, but any idea how to configure putty to not show duplicate letters like this in the console output? using Putty

  b
    b
    o
      o
      o
        o
        t
          t
          /
            /
            k
              k
              e
                e
                r
                  r
                  n
                    n
                    e
                      e
                      l
                        l
                        /
                          /
                          k
                            k
                            e
                              e
                              r
                                r
                                n
                                  n
                                  e
                                    e
                                    l
                                      l


                                        t
                                          t
                                          e
                                            e
                                            x
                                              x
                                              t
                                                t
                                                =
                                                  =
                                                  0
                                                    0
                                                    x
                                                      x
                                                      1
                                                        1
                                                        8
                                                          8
                                                          d
                                                            d
                                                            6
                                                              6
                                                              9
                                                                9
                                                                2
                                                                  2
                                                                  8
                                                                    8

[heper]

@marekandreansky

Probably match your serial settings on pfsense with whatever you configured in your terminal software

[Tzvia]

@marekandreansky Could be local echo needs to be turned off. Check in Configuration/Terminal, try force local echo OFF.

[Gertjan]

@marekandreansky said in Box dead after reboot, caused by UFS corruption ? ZFS vs UFS?:

Any idea why its not the default option for pfSense?

Read this.
The bad news, and good news : ZFS is already build into the kernel, which makes it much bigger.
Also, processing and maintaining all this information will take many more CPU cycles. Ok if you have a 'big' processor. ZFS won't work on small drives, and needs a lot more RAM.
A router / firewall is normally not file system bound, it use the storage device to boot, and store some boot settings and operational settings.
So, I guess, by default, it's very overkill to use ZFS as the default.

But these days people have firewalls have multi cores, with 250GB++ (raid ?!) drives, and several GB if memory, so ZFS could be an option.

The next time I re install pfSense, I'll choose ZFS, if I remember the option. The thing is, it will take years before I have the occasion to do so ^^