pfSense "Halt System" results in Reboot



  • I'm not sure if this is specific to my hardware, or a general challenge that others have encountered and overcome through pfSense / FreeBSD configuration. Either way, I'm posting here in the hopes that someone can point me in the right direction to resolve.

    Software: pfSense 2.4.3-p1 (FreeBSD 11.1-Release-p10)
    Hardware: Watchguard Firebox x750e (no power button)

    Selecting Diagnostics > Halt System from the pfSense webGUI does not put the system into HALT. Instead the box announces a halt but reboots. Similarly, running /sbin/shutdown -h +0 via Diagnostics > Command Prompt results in a reboot.

    Running the same command in a console session via PuTTY brings the system to a HALT state successfully, ending in a command prompt "Press any key to Reboot".

    This inability to successfully halt the system outside a console session is a problem as I wish to gracefully bring my pfSense box down in power outages using NUT (Network UPS Tools). Unfortunately power outages in my area don't align with my ability to get to the console!

    Has anyone else encountered this issue? If so, how did you overcome it? Am I missing a simple pfSense / FreeBSD configuration option to allow a halt instruction to be properly processed outside a console session?

    Thanks for any suggestions.


  • Rebel Alliance Developer Netgate

    [Please do not cross-post the same issue to multiple categories]

    Try the sysctl change recommended on https://www.netgate.com/docs/pfsense/hardware/troubleshooting-hardware-shutdown-and-power-off.html


  • Netgate Administrator

    That is specific to your hardware. As you say that box doesn't have an ATX power button so if it could power down there would be no way to power it back up short of pulling the power.

    In fact I don't think PSU in those is ATX or the relevant pins are not connected. It can't halt to standby.

    Steve



  • @stephenw10 can you explain what you mean by halt to standby?

    Perhaps I'm just not grasping what you're driving at, but I'm not looking to go to a suspend or hibernate condition where the running system state is saved. And I'm perfectly ok with the box not fully powering down. I'm looking to get to the point where the OS is unloaded and disk I/O ceases, so that the UPS can kill the line power without risking damage to the filesystem.

    This is achieved successfully within a console session by using /sbin/shutdown -h 0, but doesn't seem to work if I do it via the webGUI, or via a cron job (for example). In those cases, when the command is triggered outside a console session, the system reboots.


  • Netgate Administrator

    Ah, I see what you mean. Hmm. There may be a BIOS setting for that.... not sure though. I never found anything but I never spent much time looking.
    I suspect the OS is telling the system to go to standby but the BIOS is configured to never do that so it resets instead.

    Interesting the same command behaves differently from the CLI... odd.

    All, the X-Core-e boxes I've seen behave like that unfortunately.

    Steve



  • @stephenw10 said in pfSense "Halt System" results in Reboot:

    Interesting the same command behaves differently from the CLI... odd.

    It's the fact that it behaves "properly" within a console session and improperly outside of one, that got me thinking I could configure my way out of the situation. Otherwise I'd be willing to shrug my shoulders and accept it's 100% hardware related.

    Now if I could create a bash script that starts a virtual console (using screen, maybe?) and then executes the shutdown command within it, and that fools the system, I'd be willing to accept that as a hack. I just need to be able to pass it to the upsmon driver within NUT as a shutdown command and I'm happy.

    I'll keep tinkering.Thanks for your thoughts.


  • Netgate Administrator

    Depending what BIOS you have loaded you may or may not have ACPI enabled which could come into play here.
    If you do have it enabled you will see the CPU temp reported on the dashboard (or via sysctl).

    Steve



  • @jimp @stephenw10

    Thanks for your suggestions. ACPI is currently disabled in BIOS. The box uses a Phoenix AwardBIOS; here's a screenshot of the Power Management menu. I'll try various options (including enabling ACPI) and see where it gets me.

    0_1529157397866_BIOS_5.PNG



  • Disabling ACPI is a bad idea on any modern system, operating systems pretty much assume ACPI is available out of the box on any system they are installed on for configuration and power management. Unfortunately the net is still full of articles written around the time (late '90s and early 2000s) when it made sense to disable ACPI on certain systems because the implementations were so broken and people still take those articles as valid advice without taking note of the dates of the articles.



  • @kpa I think this might be one of those really sucky implementations of ACPI. The box is approximately 10 years old and although the BIOS has an option for enabling ACPI the hardware doesn't appear to support it.

    I've tried various BIOS configurations and nothing seems to resolve the issue I'm experiencing, though enabling ACPI makes it worse in that the box now always reboots after a halt request, whereas previously it did at least behave nicely when halt was initiated from the console.

    Here are the last few entries in the FreeBSD shutdown sequence when ACPI is enabled and a system halt is initiated:

    pfSense is now shutting down ...
    
    Jun 16 13:05:13 miniupnpd[56685]: shutting down MiniUPnPd
    
    Waiting (max 60 seconds) for system process `vnlru' to stop...done
    Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
    Waiting (max 60 seconds) for system process `syncer' to stop...
    Syncing disks, vnodes remaining...5 0 0 done
    All buffers synced.
    Swap device label/swap0 removed.
    Uptime: 2m58s
    (ada1:ata0:0:1:0): spin-down
    skc0: interrupt moderation is 100 us
    skc1: interrupt moderation is 100 us
    skc2: interrupt moderation is 100 us
    skc3: interrupt moderation is 100 us
    uhci0: wake_prep disabled wake for \_SB_.PCI0.USB0 (S5)
    uhci1: wake_prep disabled wake for \_SB_.PCI0.USB1 (S5)
    uhci2: wake_prep disabled wake for \_SB_.PCI0.USB2 (S5)
    uhci3: wake_prep disabled wake for \_SB_.PCI0.USB3 (S5)
    ehci0: wake_prep disabled wake for \_SB_.PCI0.USBE (S5)
    acpi0: Powering system off
    

    After which the hardware instantly reboots.

    In comparison, here are the last few lines with ACPI disabled, when a system halt is initiated from the console:

    pfSense is now shutting down ...
    
    Jun 16 08:53:28 ovpnc2: link state changed to DOWN
    miniupnpd[79370]: shutting down MiniUPnPd
    
    Waiting (max 60 seconds) for system process `vnlru' to stop...done
    Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
    Waiting (max 60 seconds) for system process `syncer' to stop...
    Syncing disks, vnodes remaining...6 0 0 0 done
    All buffers synced.
    Swap device label/swap0 removed.
    Uptime: 13m15s
    (ada1:ata0:0:1:0): spin-down
    skc0: interrupt moderation is 100 us
    skc1: interrupt moderation is 100 us
    skc2: interrupt moderation is 100 us
    skc3: interrupt moderation is 100 us
    
    The operating system has halted.
    Please press any key to reboot.
    

    At which point the system waits for console input.

    And for completeness, here are the last few entries in the FreeBSD shutdown sequence with ACPI disabled and a system halt initialed from Cron:

    System shutdown time has arrived
    
    pfSense is now shutting down ...
    
    Jun 16 11:48:15 miniupnpd[60894]: shutting down MiniUPnPd
    
    Jun 16 11:48:16 snort[49892]: *** Caught Term-Signal
    
    sk1: promiscuous mode disabled
    Waiting (max 60 seconds) for system process `vnlru' to stop...done
    Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
    Waiting (max 60 seconds) for system process `syncer' to stop...
    Syncing disks, vnodes remaining...6 0 0 done
    All buffers synced.
    Swap device label/swap0 removed.
    Uptime: 25m40s
    (ada1:ata0:0:1:0): spin-down
    skc0: interrupt moderation is 100 us
    skc1: interrupt moderation is 100 us
    skc2: interrupt moderation is 100 us
    skc3: interrupt moderation is 100 us
    
    The operating system has halted.
    Please press any key to reboot.
    
    Rebooting...
    

    In this last example, the system doesn't wait for console input before rebooting, perhaps because the halt instruction was not user-initiated?


  • Netgate Administrator

    Which actual bios version are you running there? The fact you have the option to enable ACPI implies it must be one of the later versions. But you should use 8.1 if you're not running that already. As I said though I've never seen this work as expected.

    Steve



  • @stephenw10

    I'm not sure what version of BIOS I am running and as it's Father's day here I'm on a self imposed vacation from tinkering. But I am coming to the same conclusion that you offered from the start. The hardware won't support a software-initiated full shut down and unfortunately pfsense doesn't include all the components of FreeBSD needed to manage around that. My testing has revealed that the operating system is smart enough to know whether a keyboard is attached (even via a console) and only presents the option to "Press any key to reboot" when it would be physically possible to do so. Otherwise it initiates a reboot.

    As the motherboard has no USB outlets attached (though I am informed that the pin-outs exist) I would need to fool pfsense into thinking a keyboard is attached when it's not. That could possibly be achieved using uinput to create a dummy keyboard in software but sadly uinput isn't in the pfsense build.

    The Watchguard x750e pfsense combo has served me well over the years, maybe it's time to stop pushing it and accept its limits. It hasn't fried yet from a power outage; I will invest in a new, ACPI friendly pfsense box in the near future and continue to ride my luck in the interim.

    Thanks Steve for your patience and support. Sorry I couldn't report a better outcome.

    Ian


  • Netgate Administrator

    It does have a PS2 header you might try. I doubt it will help however.

    Steve