NUT package (2.8.0 and below)

dennypage

@kevindd992002 said in NUT package:

The weird behavior was that after the UPS gets a shutdown command from pfsense NUT, it shuts itself down, cuts power off from the load and then turns back on right away even though there is no power from the mains yet. And then it shuts itself down again and continues this cycle until the battery gets depleted. Any ideas what's causing this?

Issues such as these seem to be common with low end Eatons...

Research the "ondelay" parameter and these variables (if your UPS has them):

battery.charge.restart
battery.runtime.restart
ups.delay.start
ups.start.battery
ups.timer.start

Also, you might remove any variables that may have been set such as battery.charge.low or battery.runtime.low.

In the end, you may not be able to fix it.

kevindd992002

@dennypage said in NUT package:

@kevindd992002 said in NUT package:

The weird behavior was that after the UPS gets a shutdown command from pfsense NUT, it shuts itself down, cuts power off from the load and then turns back on right away even though there is no power from the mains yet. And then it shuts itself down again and continues this cycle until the battery gets depleted. Any ideas what's causing this?

Issues such as these seem to be common with low end Eatons...

Research the "ondelay" parameter and these variables (if your UPS has them):

battery.charge.restart

battery.runtime.restart

ups.delay.start

ups.start.battery

ups.timer.start

Also, you might remove any variables that may have been set such as battery.charge.low or battery.runtime.low.

In the end, you may not be able to fix it.

If I remember correctly, this is an Eaton that has 1KVA+ capacity. Are all consumer UPS Eatons considered low end?

Can you confirm what NUT does based on the discussion above? Does it send a command to the UPS to shut down its outlets during the shutdown process?

dennypage

@kevindd992002 Please see advice above.

You've configured NUT to initiate shutdown when the battery gets down to 70%. On thing that comes to mind is that the Eaton may simply not support power off at such a high level.

You should research the variables I gave you. Particularly those with battery in their name.

If you aren't able to find a resolution that way, you will probably have to contact Eaton support for help.

kevindd992002

@dennypage said in NUT package:

@kevindd992002 Please see advice above.

You've configured NUT to initiate shutdown when the battery gets down to 70%. On thing that comes to mind is that the Eaton may simply not support power off at such a high level.

You should research the variables I gave you. Particularly those with battery in their name.

If you aren't able to find a resolution that way, you will probably have to contact Eaton support for help.

Yes, I did see your advice above. I just wanted to confirm the confusions we had in the discussion above where we are not sure if NUT can send commands to the UPS or just read data from it. I assume it's the former but I wantes to confirm with you. I guess my question is: when NUT initiates a shutdown, does it only send commands to the local and remote monitors or does it also send a shutdown command to the UPS?

Aha! You're probably right. I did recently change the thresholds to a higher level and the UPS probably doesn't support that. If I remove the thresholds, I should also remove "ignorelb" so that it uses the default thresholds of the UPS, correct?

dennypage

@kevindd992002 I believe the only time you should use ignorelb is if you have a ups that incorrectly asserts low battery immediately when mains fail.

Edit: This is incorrect. Using ignorelb is required in order for battery.charge.low and battery.runtime.low to be checked. Code reference here.

AndreyMoiseev

@dennypage said in NUT package:

Can't tell anything without log info. Have you tried restarting the service? (first icon on the Status line)
reboot didn't help , unfortunately now there is no time for experiments, I replaced ippon with Apc, everything worked without problems

kevindd992002

@dennypage said in NUT package:

@kevindd992002 I believe the only time you should use ignorelb is if you have a ups that incorrectly asserts low battery immediately when mains fail.

If I remember correctly, when we were discussing this a few years ago, we came to a conclusion that ignorelb is needed for the warning/low battery thresholds override to work. Is this not the case anymore?

And my pending question again: "when NUT initiates a shutdown, does it only send commands to the local and remote monitors or does it also send a shutdown command to the UPS?"

Gertjan

@kevindd992002 said in NUT package:

when NUT initiates a shutdown, does it only send commands to the local and remote monitors or does it also send a shutdown command to the UPS?

NUT, the server, the one that is connected to one or multiple 'local' UPS, polls the status of the UPS. Somthing like every 5 seconds : the UPS doesn't send anything.

The NUT server doesn't send to it's clients. The clients do the same thing : they poll the NUT server every xx seconds.

So, NUT as a process can have a 'local UPS' or a remote, other NUT process as a souce. This means that the code for NUT as a server or as a client is the same. Only the source changes.

A local UPS isn't necessary a local serial or USB device, it can even be a IP based SNMP or any other remotely access type of thing.

I've read a lot of the manual yesterday ( the source code, as it is available for everybody ) and i'm pretty sure NUT server and client are only polling the 'upstream' info. This can be another NUT installation, or an UPS.
( still not 100 % convinced, though )

Btw : NUT won't shut down the UPS.
The UPS exposes variables in it's status record that indicate what the battery situation is ( a percentage) and a value where it goes "down". That will be the moment that the outlets go down.
Up to NUT to to it's thing : shutting down locally.
So, NUT itself will never 'see' that the UPS goes down. As it shuts down the device it's running ion => it shuts down itself ;)

Also : NUT is not a program, it a collections of device and interface dependant drivers, server & client programs, tools etc.

And again : I'm not a NUT specialist at all, but I have a couple of UPSs around me for ages now. @work and @home.

kevindd992002

@gertjan said in NUT package:

@kevindd992002 said in NUT package:

when NUT initiates a shutdown, does it only send commands to the local and remote monitors or does it also send a shutdown command to the UPS?

NUT, the server, the one that is connected to one or multiple 'local' UPS, polls the status of the UPS. Somthing like every 5 seconds : the UPS doesn't send anything.

The NUT server doesn't send to it's clients. The clients do the same thing : they poll the NUT server every xx seconds.

So, NUT as a process can have a 'local UPS' or a remote, other NUT process as a souce. This means that the code for NUT as a server or as a client is the same. Only the source changes.

A local UPS isn't necessary a local serial or USB device, it can even be a IP based SNMP or any other remotely access type of thing.

I've read a lot of the manual yesterday ( the source code, as it is available for everybody ) and i'm pretty sure NUT server and client are only polling the 'upstream' info. This can be another NUT installation, or an UPS.
( still not 100 % convinced, though )

Btw : NUT won't shut down the UPS.
The UPS exposes variables in it's status record that indicate what the battery situation is ( a percentage) and a value where it goes "down". That will be the moment that the outlets go down.
Up to NUT to to it's thing : shutting down locally.
So, NUT itself will never 'see' that the UPS goes down. As it shuts down the device it's running ion => it shuts down itself ;)

Also : NUT is not a program, it a collections of device and interface dependant drivers, server & client programs, tools etc.

And again : I'm not a NUT specialist at all, but I have a couple of UPSs around me for ages now. @work and @home.

I see, thanks for the detailed explanation. So as soon as pfsense (my NUT server), shuts down, it's all up to the UPS when to shut off power from its outlets? When does this usually happen? If I remember correctly, the UPS does not wait for its battery to be depleted before doing this. If that's the case, how does the UPS know when to do this? At what point, after a blackout, does it happen?

Gertjan

@kevindd992002 said in NUT package:

it's all up to the UPS when to shut off power from its outlets?

Yep.
Mostly to protect battery, as discharging it to much will affect the life span.
I guess there isn't even a relay or switch on the outlets of an UPS, the DC to AC convert, just before the transfo, it just stops.

@kevindd992002 said in NUT package:

When does this usually happen?

The internal logic of the UPS maintains a table of battery tension and current over time. This tables helps to predict the battery behaviour over time. Everything is done to get a maximum of load unload cycles out of a battery.
Classic Lead accid batteries have far less then 1000 cycles.

@kevindd992002 said in NUT package:

not wait for its battery to be depleted before doing this

Noop. The UPS protects it's own battery for doing that. Depending on the type of battery used, depleting it will kill it.

dennypage

@gertjan At the end of the shutdown process, the NUT master does send a command to the UPS to kill the power (after a delay).

Gertjan

@dennypage said in NUT package:

At the end of the shutdown process, the NUT master does send a command to the UPS to kill the power (after a delay).

Aha : I hope there is some known delay then, because when NUT sends that command, the host system is still running.
System can take their time to shut down (to really power or cold mode).

Thanks, because you said so, my brain started to help me to finding this : https://networkupstools.org/docs/user-manual.pdf page 35 and further on "NUT outlets management".

dennypage

@kevindd992002 said in NUT package:

If I remember correctly, when we were discussing this a few years ago, we came to a conclusion that ignorelb is needed for the warning/low battery thresholds override to work.

You are correct. Thank you for reminding me.

Ignorelb code reference here.

kevindd992002

@gertjan said in NUT package:

@dennypage said in NUT package:

At the end of the shutdown process, the NUT master does send a command to the UPS to kill the power (after a delay).

Aha : I hope there is some known delay then, because when NUT sends that command, the host system is still running.
System can take their time to shut down (to really power or cold mode).

Thanks, because you said so, my brain started to help me to finding this : https://networkupstools.org/docs/user-manual.pdf page 35 and further on "NUT outlets management".

Yes, that's the ups.shutdown.delay that you see in your post above. It's also what @dennypage mentioned to me in the past.

Your APC and both my APC and EATON UPS'es have a default value of 20 seconds.

kevindd992002

@dennypage so I can write to the ups.delay.shutdown variable with upsrw but the values do not persist after a NUT service restart. Do you know of a away to have the modification persist?

Also, I tried removing all the overrides + ignorelb and let the UPS use its default low battery threshold of 60%. At 60% it sent an FSD and waited for 3 minutes of HOSTSYNC. Then it waited for the default 20 secs ups.delay.shutdown. As soon as it turned off power from its outlets, it starting back up again even though it still in battery mode (no mains) and beeping.

Gertjan

@kevindd992002 said in NUT package:

so I can write to the ups.delay.shutdown variable with upsrw but the values do not persist after a NUT service restart.

Same observation here.

When I set "ups.shutdown.delay" to 25 (it was 20) the UPS (on pfSense) show the new value :

But : did it get 'written' to the UPS ??

Restaring NUT gve me the old '20'. back.
Like NUT 'resets' the UPS to default values ?
No a big issue, I guess it's possible to send values to the UPS on "NUT" start-up.

I know now I can actually send commands to the UPS :

[2.5.2-RELEASE][admin@pfsense.local.net]/root: upscmd ups test.battery.start.quick
Username (root): admin
Password:
OK

The UPS did the battery test.

I wonder if :

upscmd ups load.off

works ;)

kevindd992002

@gertjan said in NUT package:

@kevindd992002 said in NUT package:

so I can write to the ups.delay.shutdown variable with upsrw but the values do not persist after a NUT service restart.

Same observation here.

When I set "ups.shutdown.delay" to 25 (it was 20) the UPS (on pfSense) show the new value :

But : did it get 'written' to the UPS ??

Restaring NUT gve me the old '20'. back.
Like NUT 'resets' the UPS to default values ?
No a big issue, I guess it's possible to send values to the UPS on "NUT" start-up.

I know now I can actually send commands to the UPS :
[2.5.2-RELEASE][admin@pfsense.local.net]/root: upscmd ups test.battery.start.quick
Username (root): admin
Password:
OK
The UPS did the battery test.

I wonder if :
upscmd ups load.off
works ;)

Yeah, it looks like upsrw only writes at runtime. Not sure how to make it persist.

For upcsmd and load.off, that command is considered "dangerous" as per the statement here :)

hda

@kevindd992002 said in NUT package:

@dennypage so I can write to the ups.delay.shutdown variable with upsrw but the values do not persist after a NUT service restart. Do you know of a away to have the modification persist?

My UPS is EATON Ellipse PRO 650

Settings master pfSense-box Advanced Additional upsmon.conf:
HOSTSYNC 60

Settings master pfSense-box Advanced Additional ups.conf:
(this makes settings permanent for UPS, ondelay >> offdelay)
offdelay 30
ondelay 60

Setting client/slave XP-box upsmon.conf:
shutdown -s -t 0

Simulation:
/usr/local/sbin/upsmon -c fsd

Reality in action:

1 s shutdown action start master pfSense-box & HOSTSYNC(max.60)
1 s announce shutdown on slave XP-box & shutdown action start XP-box

(23 s) pfSense-box kill-signal to UPS, start UPS offdelay(30)
(23 s) pfSense-box Halted, waiting on HOSTSYNC(max.60) for XP-box

40 s XP-box power-off, therefore takes 40 s
43 s pfSense-box power-off (XP-box signals DEAD to pfSense-box)

53 s UPS power-off, offdelay(30) expired (started when pfSense-box Halted)

83 s UPS power-on, ondelay(60) expired, booting pfSense-box etc.

Source of more information:
"https://dan.langille.org/2020/09/07/monitoring-your-ups-using-nut-on-freebsd/"

kevindd992002

@hda said in NUT package:

@kevindd992002 said in NUT package:

@dennypage so I can write to the ups.delay.shutdown variable with upsrw but the values do not persist after a NUT service restart. Do you know of a away to have the modification persist?

My UPS is EATON Ellipse PRO 650

Settings master pfSense-box Advanced Additional upsmon.conf:
HOSTSYNC 60

Settings master pfSense-box Advanced Additional ups.conf:
(this makes settings permanent for UPS, ondelay >> offdelay)
offdelay 30
ondelay 60

Setting client/slave XP-box upsmon.conf:
shutdown -s -t 0

Simulation:
/usr/local/sbin/upsmon -c fsd

Reality in action:

1 s shutdown action start master pfSense-box & HOSTSYNC(max.60)
1 s announce shutdown on slave XP-box & shutdown action start XP-box

(23 s) pfSense-box kill-signal to UPS, start UPS offdelay(30)
(23 s) pfSense-box Halted, waiting on HOSTSYNC(max.60) for XP-box

40 s XP-box power-off, therefore takes 40 s
43 s pfSense-box power-off (XP-box signals DEAD to pfSense-box)

53 s UPS power-off, offdelay(30) expired (started when pfSense-box Halted)

83 s UPS power-on, ondelay(60) expired, booting pfSense-box etc.

Source of more information:
"https://dan.langille.org/2020/09/07/monitoring-your-ups-using-nut-on-freebsd/"

Thanks for that information!

Why does the offdelay start at 23s?

For the upsmon fsd simulation test, does the UPS automatically turn its outlets ON immediately after turning them off?

dennypage

@kevindd992002 said in NUT package:

Also, I tried removing all the overrides + ignorelb and let the UPS use its default low battery threshold of 60%.

60%? That doesn't make much sense to me. Default low battery is usually 5-15%.

You probably should reach out to the UPS manufacturer to discuss their intended behavior.