NUT Package (2.8.1 and above)

dennypage

@NinthWave You probably had wrong file permissions. Be sure to match permissions as previous. Should look like this:

[23.09.1-RELEASE][root@fw]/root: ls -l /usr/local/sbin/upsmon* /usr/local/libexec/nut/usbhid-ups*
-rwxr-xr-x  1 root wheel 333728 Dec 27 10:14 /usr/local/libexec/nut/usbhid-ups
-rwxr-xr-x  1 root wheel 287088 Nov  1 00:57 /usr/local/libexec/nut/usbhid-ups.org
-rwxr-xr-x  1 root wheel  87760 Dec 27 10:13 /usr/local/sbin/upsmon
-rwxr-xr-x  1 root wheel  68904 Nov  1 00:57 /usr/local/sbin/upsmon.org
[23.09.1-RELEASE][root@fw]/root:

NinthWave

@dennypage said in NUT Package (2.8.1 and above):

@NinthWave You probably had wrong file permissions. Be sure to match permissions as previous. Should look like this:

[23.09.1-RELEASE][root@fw]/root: ls -l /usr/local/sbin/upsmon* /usr/local/libexec/nut/usbhid-ups*
-rwxr-xr-x  1 root wheel 333728 Dec 27 10:14 /usr/local/libexec/nut/usbhid-ups
-rwxr-xr-x  1 root wheel 287088 Nov  1 00:57 /usr/local/libexec/nut/usbhid-ups.org
-rwxr-xr-x  1 root wheel  87760 Dec 27 10:13 /usr/local/sbin/upsmon
-rwxr-xr-x  1 root wheel  68904 Nov  1 00:57 /usr/local/sbin/upsmon.org
[23.09.1-RELEASE][root@fw]/root:

First time I did it, I forgot to stop the service before copying the files. It might have been that.

Also, I did not check the file permissions the first time so I used 777

Now that I hed reinstalled the package and that you showed the original file permission, I used 755

The daemon restarted correctly.

Thanks

markster

@dennypage Thanks. I have SB-5100 so I would like to deploy the files you provided to mitigate my issues.
Can you help me out and describe how to perform actual replacement? Appreciate your help.

dennypage

@markster See the previous few post in this thread. There is also another thread on the topic.

NinthWave

@markster
How to patch NUT until next package availabe, in order to avoid unexpected shutdown

In short:

dowload those two archives on your PC or your pfSense directly (from SSH)
@dennypage said in NUT Package (2.8.1 and above):

For those of you that are on amd64 based systems (Intel or AMD), and are severely affected by the shutdown on calibration/self-test issue, attached are replacement versions of upsmon and usbhid-ups that you can use until the update is published.

Note that these files are for amd64 systems only. I do not have a build/test system for arm. Sorry!

upsmon.gz

usbhid-ups.gz

If download to PC, copy them to pfSense using |Diagnostics|Command Prompt|Upload file or WinSCP or else
Extract the archives. You will have two files: "usbhid-ups" and "upsmon"
Stop your NUT service from within pfSense GUI
Move the files in appropriate directories and make sure to have appropriate file permission or CHMOD 755
@dennypage said in NUT Package (2.8.1 and above):

[23.09.1-RELEASE][root@fw]/root: ls -l /usr/local/sbin/upsmon* /usr/local/libexec/nut/usbhid-ups*
-rwxr-xr-x 1 root wheel 333728 Dec 27 10:14 /usr/local/libexec/nut/usbhid-ups
-rwxr-xr-x 1 root wheel 287088 Nov 1 00:57 /usr/local/libexec/nut/usbhid-ups.org
-rwxr-xr-x 1 root wheel 87760 Dec 27 10:13 /usr/local/sbin/upsmon
-rwxr-xr-x 1 root wheel 68904 Nov 1 00:57 /usr/local/sbin/upsmon.org
[23.09.1-RELEASE][root@fw]/root:

Restart service and voilà

markster

@NinthWave Thank you for your assistance and help. Just did update and things started fine.
Will monitor. Hopefully we soon get official patch.

pfpv

@dennypage, I wonder why it's taking so long to update the official package when this case is so straightforward. I know you are not the one who can do it.

GeneralGresi

Hi guys,

I have a very similar issue, however, not during self tests or something like that, but during "normal" powerfailures too.
I get about the same error messages though, and I can confirm that the UPS is in good state, and the batteries are well too, I also tried with a different OS, with an older NUT Version, where the problem does not occur.
Think is, once power from the wall gets lost, I get the following messages, before the pfsense and all slaves immediately shut down +- 30 Seconds (with >2h of runtime left according to the UPS!)

Jan 25 18:33:08 snmp-ups 53155 Requesting UPS [Eaton_5PX2200] to power off, as/if handled by its driver by default (may exit), due to socket protocol request
Jan 25 18:33:03 upsmon 89442 Auto logout and shutdown proceeding
Jan 25 18:33:03 upsmon 89442 Executing automatic power-fail shutdown
Jan 25 18:33:03 upsd 22669 Client local-monitor@127.0.0.1 set FSD on UPS [Eaton_5PX2200]
Jan 25 18:33:03 upsmon 89442 UPS [Eaton_5PX2200] is reported as (administratively) OFF
Jan 25 18:33:03 upsmon 89442 UPS Eaton_5PX2200: administratively OFF or asleep
Jan 25 18:33:03 upsmon 89442 UPS Eaton_5PX2200 on battery

The UPS Logs are as follows:
2024/01/25 18:32:52 Normal AC NOK
2024/01/25 18:32:52 ABM state resting
2024/01/25 18:32:52 UPS on battery
2024/01/25 18:32:53 System shutdown in 2 h 17 mn 16 s
2024/01/25 18:32:53 Outlet group 1 shutdown in 2 h 49 mn 35 s
2024/01/25 18:32:53 Outlet group 2 shutdown in 2 h 49 mn 35 s
2024/01/25 18:32:53 ABM state Off
2024/01/25 18:32:53 Normal AC voltage out of tolerance

I have replaced the two files mentioned in the above posts, however, I don't think that solved the issue, cause this is not during self test mode nor calibration mode, this is a quite normal condition.

In addition: I updated my pfsense box from 2.7.0 to 2.7.2 just one or two weeks ago, about 3 weeks before I had a poweroutage of about 2 hours, which was handled as usual, no unexpected shutdowns, everything ran through.
Now with that version, it takes under a minute and everything wents black.

I ran upsc on a different machine, and it seems like during a power Outage, the Eaton 5PX Reports the Status "OFF".
ups.status: OB OFF OB

Any ideas?

dennypage

@GeneralGresi said in NUT Package (2.8.1 and above):

Jan 25 18:33:08 snmp-ups 53155 Requesting UPS [Eaton_5PX2200] to power off, as/if handled by its driver by default (may exit), due to socket protocol request
Jan 25 18:33:03 upsmon 89442 Auto logout and shutdown proceeding
Jan 25 18:33:03 upsmon 89442 Executing automatic power-fail shutdown
Jan 25 18:33:03 upsd 22669 Client local-monitor@127.0.0.1 set FSD on UPS [Eaton_5PX2200]
Jan 25 18:33:03 upsmon 89442 UPS [Eaton_5PX2200] is reported as (administratively) OFF
Jan 25 18:33:03 upsmon 89442 UPS Eaton_5PX2200: administratively OFF or asleep
Jan 25 18:33:03 upsmon 89442 UPS Eaton_5PX2200 on battery

This appears to be the failsafe when loosing connection to the UPS while on battery. This could either be a communication issue between the host and the UPS, or a bug in the snmp-ups driver.

I haven't seen the bug with the snmp-ups driver, but of course that doesn't mean it isn't there. There is a pending update to the NUT package. If you want to try an updated file in the interim, I've attached a newer snmp-ups to this post.

snmp-ups.gz

GeneralGresi

@dennypage

Thank's for the fast reply.
Replaced the new snmp-ups file, and restarted the service, however still the same issue.
I unplug the UPS for testing purposes, wait about 30-60 Seconds and the machine shuts down.
I was able to gather a screenshot from the status page:

This looks nearly the same as with a nut 2.8.0 Installation on a Bebian machine during a power loss situation, however, without the forced shutdown Flag.
(nut 2.8.0: ups.status: OB OFF OB)
I would say as this happens concurrently for 5 testruns now, I would rule out the communication loss theory, especially because everything else, including the Debian boxes stay alive - even if I let a longer time pass (as it should be).

I don't know much about the underlyings of nut, however I'd say it's the OFF Flag coming from the UPS via SNMP in a power loss situation?
As the UPS doesn't report any error in it's webinterface, I would not see the UPS itself as the reason.

dennypage

@GeneralGresi said in NUT Package (2.8.1 and above):

I would say as this happens concurrently for 5 testruns now, I would rule out the communication loss theory, especially because everything else, including the Debian boxes stay alive - even if I let a longer time pass (as it should be).

I don't know much about the underlyings of nut, however I'd say it's the OFF Flag coming from the UPS via SNMP in a power loss situation?

You cannot put too much into the Debian boxes not noticing or caring. They may still be on NUT 2.7.4. NUT 2.8.0 and above are much more attentive and sensitive to some UPS notifications.

Only other thing I can think of is to explore the new OFFDURATION configuration option. This would go into the Additional configuration lines for upsmon.conf section.

hbastasic

@dennypage said in NUT Package (2.8.1 and above):

For those of you that are on amd64 based systems (Intel or AMD), and are severely affected by the shutdown on calibration/self-test issue, attached are replacement versions of upsmon and usbhid-ups that you can use until the update is published.

Note that these files are for amd64 systems only. I do not have a build/test system for arm. Sorry!

upsmon.gz

usbhid-ups.gz

Having the shutdown bc. calibration issue, i guess, on APC Back-UPS ES 850G2

Event log:
Jan 27 19:40:41 localhost usbhid-ups[580]: ups_status_set: seems that UPS [ups] is in OL+DISCHRG state now. Is it calibrating or do you perhaps want to set 'onlinedischarge' option? Some UPS models (e.g. CyberPower UT series) emit OL+DISCHRG when offline.
Jan 27 19:40:43 localhost usbhid-ups[580]: ups_status_set: seems that UPS [ups] is in OL+DISCHRG state now. Is it calibrating or do you perhaps want to set 'onlinedischarge' option? Some UPS models (e.g. CyberPower UT series) emit OL+DISCHRG when offline.
...
Jan 27 19:40:46 localhost upsmon[95193]: Host sync timer expired, forcing shutdown
Jan 27 19:40:46 localhost upsmon[95193]: Executing automatic power-fail shutdown
Jan 27 19:40:46 localhost upsmon[95193]: Auto logout and shutdown proceeding
...
Jan 27 19:40:51 localhost shutdown[20307]: power-down by root:
Jan 27 19:40:51 localhost usbhid-ups[580]: sock_connect: enabling asynchronous mode (auto)
Jan 27 19:40:51 localhost kernel:
Jan 27 19:40:51 localhost kernel: Netgate pfSense Plus is now shutting down ...

@dennypage
Should i use the replacement files?

dennypage

@hbastasic said in NUT Package (2.8.1 and above):

Should i use the replacement files?

Up to you. If it were my system, I would.

pfguy2018

@NinthWave said in NUT Package (2.8.1 and above):

Move the files in appropriate directories and make sure to have appropriate file permission or CHMOD 755
@dennypage said in NUT Package (2.8.1 and above):

[23.09.1-RELEASE][root@fw]/root: ls -l /usr/local/sbin/upsmon* /usr/local/libexec/nut/usbhid-ups*
-rwxr-xr-x 1 root wheel 333728 Dec 27 10:14 /usr/local/libexec/nut/usbhid-ups
-rwxr-xr-x 1 root wheel 287088 Nov 1 00:57 /usr/local/libexec/nut/usbhid-ups.org
-rwxr-xr-x 1 root wheel 87760 Dec 27 10:13 /usr/local/sbin/upsmon
-rwxr-xr-x 1 root wheel 68904 Nov 1 00:57 /usr/local/sbin/upsmon.org
[23.09.1-RELEASE][root@fw]/root:

Restart service and voilà

I have been waiting for the "official" patch to be released, but am experiencing frequent shutdowns as a result of this issue, so I would like to update manually. But I am not at all versed in how to accomplish the 2 steps outlined above ("Move the files in appropriate directories and make sure to have appropriate file permission or CHMOD 755") (I have been able to download and extract the archives). Would it be possible for someone to provide the specific commands needed, and where exactly to enter them in pfSense, to accomplish these steps?

Thanks in advance!

hbastasic

@dennypage said in NUT Package (2.8.1 and above):

@hbastasic said in NUT Package (2.8.1 and above):

Should i use the replacement files?

Up to you. If it were my system, I would.

Thanks @dennypage for prompt reply, replaced the files, NUT started and everything works ok. Now just have to wait for the APC's next battery test.

btw, is there a way to invoke a battery test from pfSense, shell command or something?

pfpv

@hbastasic said in NUT Package (2.8.1 and above):

is there a way to invoke a battery test from pfSense, shell command or something?

Usually you can initiate it by pressing the physical power button on the UPS for 10 seconds or until the UPS switches to battery mode indicating the test has started. This is kind of dangerous. If you release the button earlier the UPS will turn off. This can happen unintentionally - if the button is wearing off or if you flinch. Some info: https://www.apc.com/us/en/faqs/FA405317/

I also found this from 2019, I don't know if it still works:

@dennypage said in NUT package (2.8.0 and below):

You can initiate a battery test via nut, but you will have to use the command line. Log into the system and use
upscmd -l ups
to see what commands are available. Look for commands that begin "test.battery..." Start with a quick test if available, then proceed to a deep test.

WARNING if the battery is defective, running these tests can cause the ups to cut power to the load (your pfSense box). Use at your own risk!

hbastasic

@pfpv said in NUT Package (2.8.1 and above):

@hbastasic said in NUT Package (2.8.1 and above):

is there a way to invoke a battery test from pfSense, shell command or something?

Usually you can initiate it by pressing the physical power button on the UPS for 10 seconds or until the UPS switches to battery mode indicating the test has started. This is kind of dangerous. If you release the button earlier the UPS will turn off. This can happen unintentionally - if the button is wearing off or if you flinch. I also found this from 2019, I don't know if it still works:

well, thanks, forgot to check the manual, APC FAQ, although they mention only SmartUPS line nothing on BackUPS

@dennypage said in NUT package (2.8.0 and below):
You can initiate a battery test via nut, but you will have to use the command line. Log into the system and use
upscmd -l ups
to see what commands are available. Look for commands that begin "test.battery..." Start with a quick test if available, then proceed to a deep test.

WARNING if the battery is defective, running these tests can cause the ups to cut power to the load (your pfSense box). Use at your own risk!

command line seems more convenient, thanks for this find, waiting for wife's all clear to start screwing with internet ...

pfpv

@hbastasic, find a manual for your particular model. I found this for two random APC Back-UPS's:

Press and hold the POWER ON/OFF button for 4 to 8 seconds to initiate the UPS Self Test.
During On Line Mode, a longer press of the POWER button until three beeps are heard will perform a manual battery self-test. Then, the LED will flash and UPS will enter self-test mode. Note: This will happen only when battery is fully charged in On Line Mode.

pfguy2018

@pfguy2018 said in NUT Package (2.8.1 and above):

@NinthWave said in NUT Package (2.8.1 and above):

Move the files in appropriate directories and make sure to have appropriate file permission or CHMOD 755
@dennypage said in NUT Package (2.8.1 and above):

[23.09.1-RELEASE][root@fw]/root: ls -l /usr/local/sbin/upsmon* /usr/local/libexec/nut/usbhid-ups*
-rwxr-xr-x 1 root wheel 333728 Dec 27 10:14 /usr/local/libexec/nut/usbhid-ups
-rwxr-xr-x 1 root wheel 287088 Nov 1 00:57 /usr/local/libexec/nut/usbhid-ups.org
-rwxr-xr-x 1 root wheel 87760 Dec 27 10:13 /usr/local/sbin/upsmon
-rwxr-xr-x 1 root wheel 68904 Nov 1 00:57 /usr/local/sbin/upsmon.org
[23.09.1-RELEASE][root@fw]/root:

Restart service and voilà

I have been waiting for the "official" patch to be released, but am experiencing frequent shutdowns as a result of this issue, so I would like to update manually. But I am not at all versed in how to accomplish the 2 steps outlined above ("Move the files in appropriate directories and make sure to have appropriate file permission or CHMOD 755") (I have been able to download and extract the archives). Would it be possible for someone to provide the specific commands needed, and where exactly to enter them in pfSense, to accomplish these steps?

Thanks in advance!

I have tried to figure out how to do this. I think I have moved the files into the correc directories, but I have no idea how to run "CHMOD 755" and now NUT won't start. Can someone please walk me through this?

hbastasic

@pfpv said in NUT Package (2.8.1 and above):

@hbastasic, find a manual for your particular model. I found this for two random APC Back-UPS's:

Press and hold the POWER ON/OFF button for 4 to 8 seconds to initiate the UPS Self Test.

During On Line Mode, a longer press of the POWER button until three beeps are heard will perform a manual battery self-test. Then, the LED will flash and UPS will enter self-test mode. Note: This will happen only when battery is fully charged in On Line Mode.

yep, found it, thanks
RTFM is still current, and somehow comforting or ... shaming

will report after self-test, just for reference