-
This post is deleted! -
@Unoptanio said in NUT Package (2.8.1 and above):
NUT 2.8.1
PFSENSE 2.7.1Riello Sentinel Pro 2200 with USB cable
NUT 2.8.2
PFSENSE 2.7.2Riello Sentinel Pro 2200 with USB cable
From December 9th to today December 17th it has no longer been disconnected.
In the meantime, however, I updated to pfsense 2.7.2 -
I have a UPS connected to my Synology NAS via USB port. Synology NAS acts as a UPS Server, which in turn is connected to the pfSense NUT service, 2.8.2, all on the same network. All is mostly working well with status displayed on both systems.
Recently, though, I had an automatic shutdown of my pfSense, which is good, but I want to have this automatic shutdown delayed by 10 minutes. I have ~1-2x/month a very brief power outage, only 1-2 seconds. It's enough to trigger the UPS and make me reset some clocks though!
Here are the pfsense logs implementing the shutdown process. Note, this was another brief power outage of a few seconds. This was not a long outage. How can I delay the pfSense shutdown process by 10 minutes, so the UPS recovers its normal operating status?
-
@BaseBallHat said in NUT Package (2.8.1 and above):
Recently, though, I had an automatic shutdown of my pfSense
I came here to post about the exact same problem. I have the same setup - an APC UPS connected to Synology that serves as a UPS server. I've had this setup for a couple of years without issues. I recently updated pfSense Plus to 23.09.1 and I guess this updated the nut package to 2.8.2.
Today my pfSense suddenly shut down. I got a message in Pushover "Auto logout and shutdown proceeding". I didn't understand what was going on as there was no power failure. I ran to the basement and everything was on except pfSense. The UPS's fan was on, so I guess the UPS did a self-test. I noticed before that my Back-UPS RS 1500G and other similar units do self-tests from time to time and those that have fans turn them on for some time. This never caused a problem.
I logged into Synology and there was no sign of any UPS event in the logs or anywhere, it was actually happily doing data scrubbing (and successfully finished it later). I went to check pfSense's logs and found what you found:
UPS ups@192.168.0.61: administratively OFF or asleepAnd later:
Executing automatic power-fail shutdown Auto logout and shutdown proceeding halt by root:I've never seen that "administratively OFF or asleep" message before but according to man for upsmon it's a valid message. But why does it cause shutdown? Logically it shouldn't. If the UPS is off it's too late to shutdown. Also, I wonder why Synology doesn't see such messages or maybe it ignores them.
If there are any developers here, please look into it. It started happening since nut was updated to 2.8.2. I think I would have to disable it until this is looked into. This message shouldn't cause shutdown.
P.S. Found posts about this issue with unraid: https://forums.unraid.net/topic/60217-plugin-nut-v2-network-ups-tools/page/40/
See the quoted post. Seems to be version-related. When rolled back to 2.8.0 the issue goes away. Is there any way to roll back to the previous version of NUT? It seems this may not be easy as there is something about SSL certificates and the recent update of OpenSSL. Can anyone advise? -
@BaseBallHat By default, NUT initiates a shutdown when the UPS declares a low battery situation. It's best to stay with this default behavior.
You may have gone into your Synology and set "Customize time" in the UPS configuration. If you have, I would suggest returning it to the default of "Until low battery" to restore the default NUT behavior.
-
@pfpv said in NUT Package (2.8.1 and above):
I've had this setup for a couple of years without issues. I recently updated pfSense Plus to 23.09.1 and I guess this updated the nut package to 2.8.2.
Today my pfSense suddenly shut down. I got a message in Pushover "Auto logout and shutdown proceeding". I didn't understand what was going on as there was no power failure. I ran to the basement and everything was on except pfSense. The UPS's fan was on, so I guess the UPS did a self-test. I noticed before that my Back-UPS RS 1500G and other similar units do self-tests from time to time and those that have fans turn them on for some time. This never caused a problem.
A shutdown being initiated during a self test usually indicates that the batteries are failing. How old are the batteries? UPS batteries need to be replaced every three years or so.
-
@dennypage You misunderstood the issue. Please re-read the posts and the link I provided. There was no shutdown initiated. The batteries are only about 1 year old. The Synology didn't even log anything.
Have you seen this message from a UPS before "administratively OFF or asleep"? In the link I posted it is explained that sometimes APC UPS's send an OFF message during self tests and at the same time they send NOTOFF. Before NUT 2.8.2 the OFF message was ignored. Now it causes a shutdown.
Interestingly, NUT is only at 2.8.1 version itself: https://github.com/networkupstools/nut/releases Where did the 2.8.2 come from?
We need to roll back to avoid APC UPS unexpected shutdowns. Is there a way to roll back?
-
@pfpv I you would like my recommendation on a solution, the first thing I would do is connect the UPS to a system other than a Synology as Synology has a rather messed up NUT implementation, and only support Synology systems as clients.
In my opinion, it would be much better to use pfSense or a linux system as the server and have the Synology be a client. FWIW, after doing this I would also recommend a run-time calibration, regardless of how old your batteries are.
In answer to your questions about versions: The version of NUT is actually FreeBSD's nut-devel-2023.10.07_1, which is based on git just prior to the 2.8.1 release. You can see this on the Installed Packages screen. The upper level package, pfSense-pkg-nut-2.8.2, is labeled as 2.8.2 because of a mistake being made at the moment of package publication. This is mentioned earlier in this thread. The published package should have been pfSense-pkg-nut-2.8.1_1, but once it was published as 2.8.2 the version number could not be downgraded. It doesn't affect anything other than human perception. If you really want to know more, you can read the discussions in the Redmine and Git PRs.
-
D dennypage referenced this topic on
-
Have read through all the config recommendations above, and still not working.
pfSense+ 23.09-RELEASE running on Netgate 8200
Trying to connect Home Assistant to NUT. Eaton UPS already connected to NUT running locally on pfSense with no issues. NUT package 2.8.0_2. Quirk installed to make NUT work properly, but don't remember now what I did.
Changed nut.conf MODE=netserver
upsd.users: added a new user and password, upsmon slave
Changed upsd.conf. Added LISTEN <ipOfPFsense> 3493Attempt to login to NUT from Home Assistant with:
Host: <ip of pFsense>
Port: 3493
Username: new user added above to upsd.users
Password: password for new user in upsd.usersHome Assistant always reports Failed to Connect to NUT Server.
Have tried many variations on the above, but no success. Restarted NUT after each attempted config change.
Home Assistant and pfSense are on the same network, so I don't think this is a firewall issue. Default allow LAN to any rule is present.
-
@hspindel said in NUT Package (2.8.1 and above):
Changed nut.conf MODE=netserver
upsd.users: added a new user and password, upsmon slave
Changed upsd.conf. Added LISTEN <ipOfPFsense> 3493nut.conf is not used on pfSense. You are not hand editing the ups config files are you? If you are, then that's not the right way to configure NUT on pfSense--every time you restart they will be overwritten.
Please post the configuration you have in Services / UPS / UPS Settings.
-
@dennypage said in NUT Package (2.8.1 and above):
nut.conf is not used on pfSense. You are not hand editing the ups config files are you? If you are, then that's not the right way to configure NUT on pfSense--every time you restart they will be overwritten.
Please post the configuration you have in Services / UPS / UPS Settings.
nut.conf is the only file I hand edited because I couldn't see how to do it in the GUI. The rest of the files I used the GUI to modify.
What is the right way to specify MODE=netserver?
My config page looks like this:
UPS Type: Local USB
UPS Name: EatonUPS
Enable notifications: yes
Driver: usbhid
No extra arguments to driver
No extra arguments to upsmon.conf
ups.conf: pollinterval=10
upsd.conf: LISTEN pfSenseIP 3493 (also tried LISTEN 0.0.0.0 3493)
upsd.users:
[howard]
password=xxxxxxx
upsmon slaveThank you!
-
@hspindel said in NUT Package (2.8.1 and above):
What is the right way to specify MODE=netserver?
There isn't one. The file nut.conf is unused in pfSense, and most other places. It is intended to be used by generic OS distributions, but I haven't really seen much uptake on it for the systems I work with.
Returning to the attempt to connect remotely... Have you checked the firewall log? Are you able to successfully telnet to port 3493 from the LAN?
-
@dennypage said in NUT Package (2.8.1 and above):
There isn't one. The file nut.conf is unused in pfSense, and most other places. It is intended to be used by generic OS distributions, but I haven't really seen much uptake on it for the systems I work with.
Okay, then NUT on pfSense is automatically configured as remotely accessible? I haven't found a setting to enable/disable it.
Returning to the attempt to connect remotely... Have you checked the firewall log? Are you able to successfully telnet to port 3493 from the LAN?
Nothing in the firewall logs. Yes, I can telnet in. There is no banner message, but if I type Hello I get a response ERR UNKNOWN-COMMAND. Tried to guess at a valid command. "LIST UPS" returned a good looking response. Couldn't guess any other commands.
So perhaps this is more a problem in Home Assistant.
-
@hspindel said in NUT Package (2.8.1 and above):
Okay, then NUT on pfSense is automatically configured as remotely accessible? I haven't found a setting to enable/disable it.
It is not remotely accessible by default. You are making it remotely accessible by addition of the LISTEN directive in the Extra arguments to upsd.conf.
Yes, I can telnet in. There is no banner message, but if I type Hello I get a response ERR UNKNOWN-COMMAND. Tried to guess at a valid command. "LIST UPS" returned a good looking response. Couldn't guess any other commands.
Try these commands:
USERNAME howard PASSWORD xxxxxxx LOGIN EatonUPSIf you receive "OK" in response to the commands it will confirm your setup.
Then you can then move on to beating your head against HomeAssistant.
Never done it myself, but...
If you are using the hass add-on, I assume you have all of these set, yes?
mode: netclient remote_ups_host: pfSense_IP_Addr remote_ups_name: EatonUPS remote_ups_user: howard remote_ups_password: xxxxxxxNo leading or trailing spaces, yes? No '#' or other special characters in the password, yes?
If it is still failing, you will have to go into the HomeAssistant container for NUT and look at the logs there to see what's happening.
-
@dennypage said in NUT Package (2.8.1 and above):
Try these commands:
USERNAME howard PASSWORD xxxxxxx LOGIN EatonUPSAll of these responded with OK.
Then you can then move on to beating your head against HomeAssistant.
Time to do that!
If you are using the hass add-on, I assume you have all of these set, yes?
mode: netclient remote_ups_host: pfSense_IP_Addr remote_ups_name: EatonUPS remote_ups_user: howard remote_ups_password: xxxxxxxI'm using a Synology DS1522 container for Home Assistant. It doesn't appear to have any of these settings - it just set things up automagically when I installed it.
If it is still failing, you will have to go into the HomeAssistant container for NUT and look at the logs there to see what's happening.
The log isn't very helpful (at least to me). It says:
ERROR (SyncWorker-2) [homeassistant.components.nut] Failure getting NUT ups alias, socket error.[0mI will pursue this with the HomeAssistant people. My gut-level feeling right now is that this is some kind network problem with a container connecting to pfSense.
Thank you for your help.
-
I am using the NUT package to connect to an Eaton UPS via SNMP v3.
Unfortunately I found I was not able to get the configuration as needed by using the pfSense UI to add additional items to the various configuration files, so having saved the configuration with no extra settings in the UI, I edited the configuration files directly in /usr/local/etc/nut to contain the entries required. Once this had been done, NUT started correctly and communicated with the Eaton UPS successfully for about 12 days. After that I started receiving emails that the UPS had disconnected. On checking, I found that the configuration files had become reset and lost all the settings I had added. I restored a backup copy of the configuration files, restarted the NUT service and it connected immediately.
Is there something I am missing here - do the configuration files get overwritten on a periodic basis, or how can I prevent a re-occurrance.
-
@simon_hp said in NUT Package (2.8.1 and above):
I found I was not able to get the configuration as needed by using the pfSense UI to add additional items to the various configuration files, so having saved the configuration with no extra settings in the UI, I edited the configuration files directly in /usr/local/etc/nut to contain the entries required.
What were you not able to configure in the UI?
-
@dennypage said in NUT Package (2.8.1 and above):
@pfpv I you would like my recommendation on a solution, the first thing I would do is connect the UPS to a system other than a Synology as Synology has a rather messed up NUT implementation, and only support Synology systems as clients.
In my opinion, it would be much better to use pfSense or a linux system as the server and have the Synology be a client. FWIW, after doing this I would also recommend a run-time calibration, regardless of how old your batteries are.
In answer to your questions about versions: The version of NUT is actually FreeBSD's nut-devel-2023.10.07_1, which is based on git just prior to the 2.8.1 release. You can see this on the Installed Packages screen. The upper level package, pfSense-pkg-nut-2.8.2, is labeled as 2.8.2 because of a mistake being made at the moment of package publication. This is mentioned earlier in this thread. The published package should have been pfSense-pkg-nut-2.8.1_1, but once it was published as 2.8.2 the version number could not be downgraded. It doesn't affect anything other than human perception. If you really want to know more, you can read the discussions in the Redmine and Git PRs.
Hi @dennypage, I respectfully disagree about Synology. It's just a server that passes UPS messages. It's very stable, does it properly, has worked for years and I haven't seen any complaints. I chose to connect my UPS to Synology because in my opinion it is the most critical piece of equipment to be properly shut down, and it provides a NUT server for other equipment. The messages are up to NUT on the client system to interpret. Synology itself didn't bork unlike pfSense.
I can assure you that equipment is not at fault. And here is the proof:
https://github.com/networkupstools/nut/issues/2104
https://github.com/networkupstools/nut/pull/2108The NUT 2.8.1 release notes have the following now: "Also upsmon should now recognize OFF and BYPASS flags in ups.status and report that these states begin or end. The OFF state usually means than an administrative action happened to power off the load, but the UPS device is still alive and communicating (USB, SNMP, etc.); corresponding MONITOR'ed amount of power sources are considered not being "fed" for the power value calculation purposes. The BYPASS state is now treated similarly to ONBATT: currently this UPS "feeds" its load, but if later communications fail, it is considered dead. This may have unintended consequences for devices (or NUT drivers) that do not report these modes correctly (e.g. an APC calibration routine seems to start with a few seconds of "OFF" state), so the reported status is only considered as a loss of feed if it persists for more than OFFDURATION seconds. [#2044, #2104]"
Link: https://networkupstools.org/historic/v2.8.1/docs/release-notes.chunked/NUT_Release_Notes.html#_release_notes_for_nut_2_8_1_what_8217_s_new_since_2_8_0I read your notes about the release of your 2.8.2 pfSense NUT package. You used nut-devel as there was no indication the a new NUT release was coming. But NUT 2.8.1 came out shortly after and it included a fix for the described faulty behavior.
Since APC UPS's have bi-monthly self-test routines I predict that forums will soon be flooded with messages like mine about "administratively OFF or asleep" and pfSense equipment suddenly shutting down for no reason.
Please, please, please update the pfSense NUT package to 2.8.1 NUT release. It will mitigate the problem.
-
@pfpv said in NUT Package (2.8.1 and above):
I respectfully disagree about Synology. It's just a server that passes UPS messages. It's very stable, does it properly, has worked for years and I haven't seen any complaints. I chose to connect my UPS to Synology because in my opinion it is the most critical piece of equipment to be properly shut down, and it provides a NUT server for other equipment.
I don't disagree that there is a bug in NUT, and I will be looking at that shortly. That said...
I'm in my third generation of Synology equipment. Fifteen plus years. I have handled a number of support issues arising from Synology's NUT implementation, both mine and others. Their NUT implementation started out very straight forward, but over time it has evolved, becoming more and more specialized and complex. With DSM 7.2, it's gotten to the point that I don't consider it to be completely reliable, and view it as a primary of last resort.
I will point out one thing that you said that you may wish to reconsider. You indicate that the NAS is the most important thing to have a proper shutdown. I agree with this general sentiment. However, by running the NAS as the NUT primary, you are actually incurring higher risk for the NAS rather than less.
The NUT primary does not initiate a shutdown until all the associated secondaries have logged out of the primary. Assuming a default polling interval of 5 seconds, a pfSense or Linux system will take something on the order of 10-15 seconds before they log out, and another 30-90 seconds to complete a shutdown. This means that the NAS will not begin its shutdown until 10-15 seconds after the UPS declares a low battery. Depending upon configuration and current activity, a Synology usually takes over 2 minutes to complete a shutdown. If the UPS is off in calculating remaining runtime, you run the risk of exhausting the battery before the NAS has completed its shutdown.
If you reverse this situation and use pfSense or a Linux system as the primary, then the NAS will begin its shutdown within 5 seconds. Not only does this give a wider margin of safety for the NAS, it can give an increased margin of safety for the other systems as well. When the NAS shuts down, there is suddenly a lot less load on the UPS, which gives more time for the other systems to complete their shutdown even if the estimated remaining runtime was incorrect.
The relevant passage from upsmon.conf:
# Also, since the primary system stays up the longest, it suffers higher risks # of ungraceful shutdown if the estimation of remaining runtime (or of the # time it takes to shut down this system) was guessed wrong. By consequence, # the "secondary" systems typically monitor the power environment state # through the 'upsd' processes running on the remote (often "primary") systems # and do not directly interact with an UPS (no local NUT drivers are running # on the secondary systems). As such, secondaries typically shut down as # soon as there is a sufficiently long power outage, or a low-battery alert # from the UPS, or a loss of connection to the primary while the power was # last known to be missing.As a general rule, you want to have systems that represent the highest UPS load and/or longest shutdown time as secondaries, and a system that represents lower load and is fast to shut down as the primary.
-
@hspindel said in NUT Package (2.8.1 and above):
I'm using a Synology DS1522 container for Home Assistant. It doesn't appear to have any of these settings - it just set things up automagically when I installed it.
If you don't have those settings, then I assume you are not using the hass add-on. What then are you using? And how are you configuring it?
As an aside, if you have HomeAssistant running in a container on your Synology, why are you running NUT inside the container? I assume it is not for shutting down the container which would be handled by the Synology. Are you running HA automation based on the UPS values?
