NUT upssched

mike81

Hi,
sorry for any grammar mistakes but I'm not native English speaker.
For a couple of days now I try to configure timed shutdown using NUT.
I recently bought CyberPower UPS model CP1300EPFCLCD. It is connected to my pfSense box via USB cable.
I installed latest nut package i.e. 2.7.4_10.
UPS is recognized and my nut client (proxmox server) appears in syslog as a logged user.
When I pull the plug upsmon event ONBATT is triggered and logged in the syslog. The same goes for ONLINE event.
Problem begins when I try to use upssched to configure timed shutdown.

Below are my current GUI settings.

Services UPS Settings.png

I created upssched.conf file (path and permissions)

-rw-r--r--  1 root  wheel  873 Sep  1 17:35 /usr/local/etc/nut/upssched.conf

Content of my upssched.conf file:

CMDSCRIPT /usr/local/bin/upssched-cmd

# Command pipe and lock-file
PIPEFN /var/db/nut/upssched.pipe
LOCKFN /var/db/nut/upssched.lock

# Send alerts immediately on change in line power
AT ONBATT CP1300_UPS@localhost EXECUTE onbatt
AT ONLINE CP1300_UPS@localhost EXECUTE onpower

# Send alerts immediately if low battery limit is reached
AT LOWBATT CP1300_UPS@localhost EXECUTE onlowbatt

# (Optional) Silence the beeper after 2 minutes
AT ONBATT CP1300_UPS@localhost START-TIMER mute_beeper 120
AT ONLINE CP1300_UPS@localhost CANCEL-TIMER mute_beeper

# Shutdown after 5 minutes on battery (5 * 60 = 300)
AT ONBATT CP1300_UPS@localhost START-TIMER onbatt_shutdown 300

# Cancel timer if power's restored
AT ONLINE CP1300_UPS@localhost CANCEL-TIMER onbatt_shutdown

My upssched-cmd script (path and permissions):

-r-xr-xr-x  1 root  wheel  1667 Aug 30 19:04 /usr/local/bin/upssched-cmd

Content of my upssched-cmd file:

#!/bin/sh

# START: User-specific settings                            
#
UPS_USERNAME="some-user"
UPS_PASSWORD="some-pass"
UPS_LINK="CP1300_UPS@localhost"
# END  

# START: Battery info
#
STATUS=$( upsc $UPS_LINK ups.status )
CHARGE=$( upsc $UPS_LINK battery.charge )
CHMSG="[$STATUS]:$CHARGE%"
# END  

logger -i -t upssched-cmd Calling upssched-cmd $1

case $1 in
    onbatt)
        # make sure beeper is enabled
        upscmd -u ${UPS_USERNAME} -p ${UPS_PASSWORD} ${UPS_LINK} beeper.enable
        # alert
        message="$UPS_LINK, $CHMSG - Power outage, on battery"
        logger -t upssched-cmd "$message"
        ;;
    onlowbatt)
        message="$UPS_LINK, $CHMSG - Power battery low - shutdown now!"
        logger -t upssched-cmd "$message"
        ;;
    onpower)
        message="$UPS_LINK, $CHMSG - Power restored"
        logger -t upssched-cmd "$message"
        ;;
    mute_beeper)
         message="(2) minute limit exceeded, muting beeper"
         upscmd -u ${UPS_USERNAME} -p ${UPS_PASSWORD} ${UPS_LINK} beeper.mute
         ;;
    onbatt_shutdown)
        message="$UPS_LINK, $CHMSG - Triggering shutdown after (5) minutes on battery"
        logger -t upssched-cmd "$message"

        /usr/local/sbin/upsmon -c fsd
        ;;
    replace_batt)
        message="Quick self-test indicates battery requires replacement"
        logger -t upssched-cmd "$message"
        ;;
    *)
        logger -t upssched-cmd "Unrecognized command: $1"
        exit 1 
        ;;
esac

I know that my upssched-cmd is working because when I manually execute it from the console I get appropriate message in the syslog but it is not executed by upssched. Strange thing is that I don't get any error messages from upssched in the syslog just standard 'upsmon 59258 UPS CP1300_UPS on battery' / 'upsmon 59258 UPS CP1300_UPS on line power' info from upsmon.

I have also check if NOTIFYCMD is working by calling my test script instead of NOTIFYCMD /usr/local/sbin/upssched and it is being triggered when I pull the plug.
In the syslog I can see message 'Script executed by uucp'

My test script:

#!/bin/sh

WHOAMI=`whoami`
printf "Script executed by $WHOAMI" | /usr/bin/wall 

logger "Script executed by $WHOAMI"

exit 0

What else I tried:

In my upssched.conf I changed

# Command pipe and lock-file
PIPEFN /var/db/nut/upssched.pipe
LOCKFN /var/db/nut/upssched.lock

to these lines

# Command pipe and lock-file
PIPEFN /var/db/nut/upssched/upssched.pipe
LOCKFN /var/db/nut/upssched/upssched.lock

and set following permissions to the 'upssched' folder:

drwxrwxrwx 2 uucp uucp

to see if pipe and lock files are created but there is nothing there.
I also set 777 permissions to my upssched-cmd script but it didn't help either.

I even recreated example from this post UPSMON independent shutdown but it didn't work for me either.

I don't know what else I can check and why it isn't working but i suspect it might have something to do with permissions. Maybe some of you have similar working setup and could help? :)

Thanks in advance,
Michal

dennypage

@mike81 The first thing I would do is to replicate @Gertjan's working config. Don't pick and choose things that you think might be appropriate--use his exact config. Change the name of your ups to "ups" to match his config. Enable notifications. Use just the single additional line for upsmon.conf. Use the exact name, contents and ownership his files, without any additional lines, comments, etc. Once you get a working config, then you can start modifying.

BTW, in almost all cases, the battery overrides are a bad idea. Yes, I know people use them all over the place, but that doesn't make it a good idea. Unless you have experienced an actual problem that you are fixing, they should be removed. It's best to let NUT and the UPS alone to do their job.

Gertjan

@mike81 said in NUT upssched:

PIPEFN /var/db/nut/upssched/upssched.pipe
LOCKFN /var/db/nut/upssched/upssched.lock

What's wrong with keeping them in /var/db/nut/ ?

@mike81 said in NUT upssched:

to see if pipe and lock files are created but there is nothing there.

Read https://networkupstools.org/docs/man/upssched.conf.html for details about LOCKFN.

I have to update - if needed, the other thread.
I've switched from a home build device to a SG-4100Max a coulpe of month ago. I activated NUT, and I know my Syno NAS is attached to it, as are two Windows PC's with the help of https://github.com/gawindx/WinNUT-Client/releases - the v2.0.7722.30975 version.

mike81

@dennypage Thank you for reply.
So following your advice I replicated @Gertjan's working config. The only difference is that in my config I listen on default 127.0.0.1 and use NAT port forward so there is nothing in 'Additional configuration lines for upsd.conf' field on UPS settings tab. I also use different login credentials for my NUT client.
With these changes it started working :)

This was my syslog when I tested power outage.

The only thing that didn't work was e-mail notification. I enabled notifications on UPS Settings tab and configured SMTP on pfSense. Tested SMTP an it's working fine. If I understand correctly after 20s on battery I should get an e-mail. I didn't and I don't see any 'message sent' entries in my log either. Do you have any idea why emails are not working?
Now my next step will be to gradually revert back to my old config to see when it breaks but before I do that I would like to figure out why with the current config emails are not being delivered.

P.S.
Tnx for the tip regarding battery overrides.

mike81

@gertjan

@gertjan said in NUT upssched:

What's wrong with keeping them in /var/db/nut/ ?

nothing wrong with /var/db/nut/. I just wanted to check if setting write permissions for everyone would fix things but I didn't want to test it directly on nut folder.

dennypage

@mike81 said in NUT upssched:

The only thing that didn't work was e-mail notification. I enabled notifications on UPS Settings tab and configured SMTP on pfSense. Tested SMTP an it's working fine.

Two things:

In your earlier configuration, I don't see where you are sending email. Are you expecting logger to do that for you? Note the lack of an entry in the log saying something to the effect of "/usr/local/pkg/nut/nut_email.php: Message sent to..."
I would suggest that you try with a standard configuration (no entries in Additional configuration lines for upsmon.conf) and confirm that email works with that. And be sure you have the Enable notification box checked.

@Gertjan, would you mind posting your upsmon.conf, upssched.conf, and upssched-cmd please so that @mike81 can compare his result please? Thanks!

Gertjan

@dennypage said in NUT upssched:

would you mind ....

When I posted above, I restarted playing a bit with NUT, as I didn't really finished the set up since I switched to a SG-4100, a couple of months ago.
All I managed to to is discovering that my battery from the attached UPS is about to die on me.
I have a spare battery, I really need to change it, I do so tomorrow.

@mike81 when you check the NUT notification check box, have a look at the upsmon.conf file ( the one on the disk,; here /usr/local/etc/nut/ not the one in the GUI).
From what I recall, you will find a new line 4 with a new NOTIFYCMD line that should notify you using any notification you have set up. That could be smtp, or something else.

I'll finish that upssched stuff, and post back here with my findings.

dennypage

When you check the Enable notification box, upsmon.conf will look like this:

MONITOR ups 1 local-monitor xxxxxxxxxxxxxxxxxxx master
SHUTDOWNCMD "/sbin/shutdown -p +0"
POWERDOWNFLAG /etc/killpower
NOTIFYCMD /usr/local/pkg/nut/nut_email.php
NOTIFYFLAG ONLINE   SYSLOG+WALL+EXEC
NOTIFYFLAG ONBATT   SYSLOG+WALL+EXEC
NOTIFYFLAG LOWBATT  SYSLOG+WALL+EXEC
NOTIFYFLAG FSD    SYSLOG+WALL+EXEC
NOTIFYFLAG COMMOK   SYSLOG+WALL+EXEC
NOTIFYFLAG COMMBAD  SYSLOG+WALL+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
NOTIFYFLAG REPLBATT SYSLOG+WALL+EXEC
NOTIFYFLAG NOCOMM   SYSLOG+WALL+EXEC
NOTIFYFLAG NOPARENT SYSLOG+WALL+EXEC

/usr/local/pkg/nut/nut_email.php is the standard script that sends email for the NUT package.

@gertjan, in the output that you published in the other thread, I noticed that nut_email.php is reported as being invoked in the system log. But in looking at the cmd script you posted, I didn't see the invocation of nut_email.php.
With the config screenshot you posted, there would have been two NOTIFYCMD entries in your upsmon.conf. The first is added by the package when checking the Enable Notification box, and the second (yours) comes from the Extra arguments to upsmon.conf section. NUT doesn't supports multiple NOTIFYCMDs, so the second entry will override the first entry [This is behavior I intended with the ordering of output to upsmon.conf].
So somewhere in your script you had to invoke nut_email.php, and I was hoping you would post it so @mike81 could pick it up. I missed that you had said above that you were no longer using the config. Sorry about that.

Anyway... @mike81 you will need to add something to actually send the email.

Gertjan

@dennypage said in NUT upssched:

NUT doesn't supports multiple NOTIFYCMDs

I wasn't sure about that, I was asking myself the same question about multiple NOTIFYCMDs.
And now you've said the same thing. So it must be true ;)
So :

as I prefer to use the NUT event scheduler

NOTIFYCMD /usr/local/sbin/upssched

Not finished, but I'm testing with this :

#!/bin/sh

UPS="ups"
STATUS=$( upsc $UPS ups.status )
CHARGE=$( upsc $UPS battery.charge )
CHMSG="[$STATUS]:$CHARGE%"

# upssched-cmd  --slave --
# /usr/bin/logger -i -t upssched-cmd Calling upssched-cmd $1

case $1 in
    online) 
    MSG="$UPS, $CHMSG - power supply has been restored."
    ;;
    onbatt)
    MSG="$UPS, $CHMSG - power failure - save your work!"
    ;;
    lowbatt) 
    MSG="$UPS, $CHMSG - shutdown now!"
    ;;
    upsgone)
    MSG="$UPS, $CHMSG - UPS has been gone for awhile"
    ;;
    powerdown)
    MSG="$UPS, $CHMSG - Going powerdown"
    ;;
    commbad)
    /usr/bin/logger -i -t upssched-cmd "Communcation with UPS is lost"
    ;;
    *) 
    /usr/bin/logger -i -t upssched-cmd "Unkown arg: \"$1\", $CHMSG"
    exit 1
    ;;
esac
/usr/bin/logger -i -t upssched-cmd $MSG
/usr/local/pkg/nut/nut_email.php $MSG
exit 0

and as you can see, "/usr/local/pkg/nut/nut_email.php" has been stashed in at the bottom, using an argument $MSG, that been crafted earlier in the shell script.
This works, I get mails with some content like :

15:36:39 UPS Notification from pfSense.brit-hotel-fumel.net - Tue, 06 Sep 2022 15:36:39 +0200
ups, [OB DISCHRG]:61% - power failure - save your work!

The battery was under the treshold, my slave systems went down ....

I restored the power before pfSense really went down also :

15:36:50 UPS Notification from pfSense.my-network.net - Tue, 06 Sep 2022 15:36:50 +0200
ups, [OL DISCHRG]:46% - power supply has been restored.

Didn't have time to change my battery. I guess it's time.

Btw : don't do what I did : create a "upssched-cmd.sh" file using 'Windows", and upload it to pfSense. I saw an awful lot of strange behaviour, to find finally that the file was NOT Unix encodes, but Windows (double line feeds). It took me the better part of the morning to find this out (again).

Silly thing : look at /usr/local/pkg/nut/nut_email.php
The last two lines :

send_smtp_message($message, $subject);

@notify_all_remote($subject . " - " . $message);

If the smtp sending method is set up, then the mail will get send twice !
Because notify_all_remote() will use all notify methods available, including smtp.

My advise : comment out this line :

// send_smtp_message($message, $subject);

as notify_all_remote() will take care of things.

edit :

The related logs lines :

upsmon 83356 - - UPS ups on battery
upssched 6516 - - Timer daemon started
upssched 6516 - - New timer: onbatt (15 seconds)
upssched 6516 - - Event: onbatt
upssched-cmd 35181 - - Calling upssched-cmd onbatt
upssched-cmd 35511 - - ups, [OB DISCHRG]:61% - power failure - save your work!
php 43165 - - notify_monitor.php: Message sent to my-mail@gmail.com OK
upsmon 83356 - - UPS ups on line power
upssched 6516 - - Cancel onbatt, event: online
upssched-cmd 31974 - - Calling upssched-cmd online
upssched-cmd 32290 - - ups, [OL DISCHRG]:46% - power supply has been restored.
upssched 6516 - - Timer queue empty, exiting
php 43165 - - notify_monitor.php: Message sent to my-mail@gmail.com OK

dennypage

@gertjan Thank you for posting the updated script. The one posted in the other thread didn't have the call to nut_email.php in it, so I was wondering. @mike81, you should have everything you need now.

@gertjan said in NUT upssched:

Silly thing : look at /usr/local/pkg/nut/nut_email.php

Yea. One of the Netgate guys added the call to notify_all_remote but didn't remove the call to send_smtp_message. I'll add that to the current PR. Thanks for pointing that out.

mike81

@dennypage said in NUT upssched:

In your earlier configuration, I don't see where you are sending email. Are you expecting logger to do that for you?

When I looked at @Gertjan syslog from other thread I noticed the same thing

@dennypage said in NUT upssched:

@gertjan, in the output that you published in the other thread, I noticed that nut_email.php is reported as being invoked in the system log. But in looking at the cmd script you posted, I didn't see the invocation of nut_email.php.

and I thought it should work.

I tested @Gertjan modified upssched-cmd script by adding to the end of my current config the same 2 lines of as @Gertjan did:
/usr/local/pkg/nut/nut_email.php $MSG
exit 0

I left Enable notifications box checked so my uspmon.conf looked like this:

MONITOR ups 1 local-monitor a10bf250fc1d1fc9935b master
SHUTDOWNCMD "/sbin/shutdown -p +0"
POWERDOWNFLAG /etc/killpower
NOTIFYCMD /usr/local/pkg/nut/nut_email.php
NOTIFYFLAG ONLINE   SYSLOG+WALL+EXEC
NOTIFYFLAG ONBATT   SYSLOG+WALL+EXEC
NOTIFYFLAG LOWBATT  SYSLOG+WALL+EXEC
NOTIFYFLAG FSD	  SYSLOG+WALL+EXEC
NOTIFYFLAG COMMOK   SYSLOG+WALL+EXEC
NOTIFYFLAG COMMBAD  SYSLOG+WALL+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
NOTIFYFLAG REPLBATT SYSLOG+WALL+EXEC
NOTIFYFLAG NOCOMM   SYSLOG+WALL+EXEC
NOTIFYFLAG NOPARENT SYSLOG+WALL+EXEC


NOTIFYCMD /usr/local/sbin/upssched

So I tested this config to see what happens.

On my first test I don't know why I got bunch of strange errors but I received 3 emails ( 1 for onbatt event and 2 for online)

Next I unchecked Enable notifications box

and the content of uspmon.conf changed to this:

MONITOR ups 1 local-monitor b30ada182919a63ebbd2 master
SHUTDOWNCMD "/sbin/shutdown -p +0"
POWERDOWNFLAG /etc/killpower


NOTIFYCMD /usr/local/sbin/upssched

Notice there is no NOTIFYFLAG present.

Then I did second test and I got only this 2 entries in my syslog. No entries from upssched or upssched-cmd.

I reenabled notifications and tested again. This time I didn't get any strange errors and everything was working as expected.

On my 2 test upssched-cmd was not called because there wasn't a single NOTIFYFLAG present in uspmon.conf file.

So my question is whether this is intentional or is it a bug?

Thank you @dennypage and @Gertjan for your help. Now I should be able to get my config working.

dennypage

@mike81 said in NUT upssched:

Notice there is no NOTIFYFLAG present.
...
So my question is whether this is intentional or is it a bug?

Intentional. Unless you check the Enable Notifications box, there are no NOTIFYCMD or NOTIFYFLAG entries written to the upsmon.conf file.

Gertjan

@mike81 said in NUT upssched:

Notice there is no NOTIFYFLAG present.
Then I did second test and I got only this 2 entries in my syslog. No entries from upssched or upssched-cmd.

I reenabled notifications and tested again. This time I didn't get any strange errors and everything was working as expected.

When there are no NOTIFYFLAGs, the NOTIFYCMD ( calling /usr/local/sbin/upssche) doesn't get used..
Your log shows just the basic upsmon operations.

I saw the "read configuration failed, trying again" messages.
I had to dive in the manual what they actually mean.
My /root/upssched-cmd file ( I placed it in /root/ ) contained errors. The scripts hanged, so upsched timed out. sh or bash scripting is an art.

The upssched program, with the help of a script file, makes it possible to create complex scenarios where devices can get halted in a defined order, can use multiple UPSes etc etc.

For just sending a mail : check the box and you'll be set.

mike81

I managed to get my config working :)
I thought that I share it here so maybe someone will find it useful.

My UPS Settings:

Services UPS Settings-final-c.png

/usr/local/etc/nut/upssched.conf 
-rw-r--r--   1 root  wheel    630 Sep 11 21:29 upssched.conf

CMDSCRIPT /usr/local/bin/upssched-cmd

# Command pipe and lock-file
PIPEFN /var/db/nut/upssched.pipe
LOCKFN /var/db/nut/upssched.lock

# Send alerts immediately on change in line power
AT ONBATT * EXECUTE onbatt
AT ONLINE * EXECUTE onpower

# Send alerts immediately if low battery limit is reached
AT LOWBATT * EXECUTE onlowbatt

# (Optional) Silence the beeper after 2 minutes
AT ONBATT * START-TIMER mute_beeper 120
AT ONLINE * CANCEL-TIMER mute_beeper

# Shutdown after 5 minutes on battery (5 * 60 = 300)
AT ONBATT * START-TIMER onbatt_shutdown 300

# Cancel timer if power's restored
AT ONLINE * CANCEL-TIMER onbatt_shutdown

/usr/local/bin/upssched-cmd
-r-xr-xr-x   1 root  wheel     1411 Sep 11 21:29 upssched-cmd

#!/bin/sh

# START: User-specific settings                            
#
UPS_USERNAME="local-admin"
UPS_PASSWORD="some-pass"
UPS_LINK="CP1300_UPS@localhost"
# END  

# START: Battery info
#
STATUS=$( upsc $UPS_LINK ups.status )
CHARGE=$( upsc $UPS_LINK battery.charge )
CHMSG="[$STATUS]:$CHARGE%"
# END  

logger -i -t upssched-cmd Calling upssched-cmd $1

case $1 in
    onbatt)
        # make sure beeper is enabled
        upscmd -u ${UPS_USERNAME} -p ${UPS_PASSWORD} ${UPS_LINK} beeper.enable
        # alert
        message="$UPS_LINK, $CHMSG - Power outage, on battery"
        ;;
    onlowbatt)
        message="$UPS_LINK, $CHMSG - Power battery low - shutdown now!"
        ;;
    onpower)
        message="$UPS_LINK, $CHMSG - Power restored"
        ;;
    mute_beeper)
         message="(2) minute limit exceeded, muting beeper"
         upscmd -u ${UPS_USERNAME} -p ${UPS_PASSWORD} ${UPS_LINK} beeper.mute
         ;;
    onbatt_shutdown)
        message="$UPS_LINK, $CHMSG - Triggering shutdown after (5) minutes on battery"
        /usr/local/sbin/upsmon -c fsd
        ;;
    replace_batt)
        message="Quick self-test indicates battery requires replacement"
        ;;
    *)
        message="Unrecognized command: $1"
	/usr/bin/logger -i -t upssched-cmd $message
        exit 1 
        ;;
esac
/usr/bin/logger -i -t upssched-cmd $message
/usr/local/pkg/nut/nut_email.php $message
exit 0

The reason why my original config didn't work (upssched not starting) was caused by wrong AT declarations.

I had to change them from
AT notifytype UPS-name EXECUTE command
to
AT notifytype * EXECUTE command

I'm not sure why I can't use specific UPS name (CP1300_UPS@localhost in my case) as it is recommended approach but otherwise upssched doesn't work.

Regarding the offdelay and ondelay parameters in Extra Arguments to driver field.

Initially I set them to offdelay=120 and ondelay=180 but it wasn't working as expected.
After 120 seconds UPS was turning off but 60 seconds later (180 - 120) it was turning back on again while still being on battery power.

I had to set ondelay=0 to prevent UPS from turning on until line power was restored.