UPS Battery test triggering pfSense shutdown.



  • Ok so some info

    APC Back UPS BX1400UI
    Current power consumption about 200 watts.

    So a automated test occurred and I lost internet, I assumed ISP outage as its dusk hours and had a few hours lately but then notice the unit had shut down.

    After I booted it backup I noticed battery had dropped from 100% to 56%. For a 6 second test.

    I then had the console up on monitor, and from ssh initiated a manual test.

    I got this.

    Power failure. Running on UPS batteries.

    Followed by this about 5 seconds later just before end of test.

    UPS battery power exhausted. Doing shutdown.

    After booting the unit back up I could see battery was down to 43%.

    There is clearly a battery problem for it to drain that fast, this question is not related to the UPS situation (it is about 3-4 years old now I think).

    My question is why the shutdown was initiated, the settings configured are for either 3 minutes remaining to be reached on the UPS status, or 5% or less remaining. Neither of these values were met so it seems its prematurely shutting down.

    Is it possible different values are been sent whilst it is actually on power e.g. less battery life, as I am only seeing the values after I boot it back up, and at that point its running again from the mains.

    The configuration specifically is specifically NUT is doing the monitoring, I have the network listener open on it on localhost, and apcupsd is also running fetching the data from NUT over localhost. The shutdown is been initiated by apcupsd not NUT.

    Is it possible battery failure is been detected so it shuts down for that reason, and the message is just misleading.

    For reference, PC and monitor stayed on during both tests, so the symptoms really are the huge drain and pfSense been shutdown, the shutdown made me notice and I would rather find out on a test than in an actual power cut.

    ![0_1601437066168_ups.png](Uploading 100%)





  • Do you see in the main pfSense log, just before the initial pfSense bootup message, a message that states that it what instructed to halt/stop ?
    The actual age of your battery would explain that the UPS wasn't delivering enough power (volts) to pfSense, so it died ?
    Or : did pfSense receive a "battery failure " message, so it shuts down right away, as it knows it's on battery power ?

    I always have a paper on my UPS with the date of the latest battery change. The oldest, right now, says 12/2018.



  • Just throwing this out there, but is it possible NUT saw the velocity of the voltage drop and anticipated the shutdown correctly?



  • @Gertjan said in UPS Battery test triggering pfSense shutdown.:

    Do you see in the main pfSense log, just before the initial pfSense bootup message, a message that states that it what instructed to halt/stop ?
    The actual age of your battery would explain that the UPS wasn't delivering enough power (volts) to pfSense, so it died ?
    Or : did pfSense receive a "battery failure " message, so it shuts down right away, as it knows it's on battery power ?

    I always have a paper on my UPS with the date of the latest battery change. The oldest, right now, says 12/2018.

    I dont see anything that names the service but when I googled the sentence as an exact phrase it was APCUPSD. Yes pfSense was shutdown due to a notification, it was a clean shutdown, watched it on the console.

    New UPS is arriving tomorrow, so will see if it behaves the same when that is plugged in.



  • @chrcoluk said in UPS Battery test triggering pfSense shutdown.:

    New UPS

    Changing the battery wasn't easier ? ;)



  • We have many APC UPSs on PCs and over the years a decent portion of them trigger "self test failed" and the PC reboots because the battery can't handle the load of the PC, and the PC loses power. This is more common with old batteries. APC recommends replacing the battery every 3-5 years depending on ambient temperature (higher is worse) and time on battery. I would say 5 years is pretty common for self test failures to start occurring.

    So to answer your question I would say it is absolutely possible for the self test to trigger a situation where the UPS thought it didn't have enough runtime to last the 3 minutes. I am however a bit surprised it completed the shutdown cleanly since as I said usually the device just loses power.



  • @Gertjan no as this UPS doesnt have swappable battery. Plus it was planned to replace it anyway. The estimated minutes before running out was over 5 minutes. (at 44%),



  • @teamits said in UPS Battery test triggering pfSense shutdown.:

    We have many APC UPSs on PCs and over the years a decent portion of them trigger "self test failed" and the PC reboots because the battery can't handle the load of the PC, and the PC loses power. This is more common with old batteries. APC recommends replacing the battery every 3-5 years depending on ambient temperature (higher is worse) and time on battery. I would say 5 years is pretty common for self test failures to start occurring.

    So to answer your question I would say it is absolutely possible for the self test to trigger a situation where the UPS thought it didn't have enough runtime to last the 3 minutes. I am however a bit surprised it completed the shutdown cleanly since as I said usually the device just loses power.

    Note the PC is on the UPS as well, that stayed powered. So power was actually been supplied via the battery, but it drained super fast tho, first test drained circa 40% in 6 seconds. It is a 700 watt UPS which was at around 30% load.



  • @chrcoluk said in UPS Battery test triggering pfSense shutdown.:

    test drained circa 40% in 6 seconds

    That would explain why the pfSense thought it had less than the 3 minutes of runtime remaining, and shut down.

    To me that means either the battery is old, or there is way too much power draw through the UPS (too many things plugged in and the battery can't handle it).


  • Netgate Administrator

    With a battery going flat that quick it would not surprise me to find the control software throws some unexpected results. Some of those calculations are probably waaay outside the expected performance window, they might just show as invalid.

    Just replace the battery/usp.

    Steve



  • The UPS seems to show same estimated battery life times as before, but it just drains super fast.

    New UPS is here, I will probably swap it out tomorrow after some sleep.



  • Ok I am using a new UPS now, I removed apcupsd as its not an APC UPS. I had to completely remove that package as when you disable it for some reason it still starts up.

    However I notice NUT has no configuration options for when to shut down, whilst in apcupsd it is configurable.

    My preferred setup is for pfSense only to shutdown when there is barely any time left on the battery, so e.g. below 5 or below a couple of minutes (it takes barely 5 seconds to shutdown). I then have PC connected via network to NUT network service want that to auto sleep/hibernate with say 30% battery left.

    Any of you guys who use NUT have an idea how the shutdown behaviour locally on pfSense is configured?


    Ran a manual battery test, PC on with game running, pfSense was not shut down this time (although dont know if NUT is configured to do that), and the estimated run time was static.

    https://imgur.com/NKyxHL5

    --

    Been doing a lot of reading into it, I think I now see why people who replied said they had issues with it not shutting down in time, so NUT on monitored UPS (which is auto added by the pfSense package) will issue a shutdown on critical battery state instead of low. Someone on reddit said they edited source code to change it to low, but interesting.


Log in to reply