NUT package (2.8.0 and below)
-
@teken If you wanting to stay on 2.4.5, you need to explicitly set the branch in System -> Update to "Previous stable version (2.4.5 DEPRECATED)". Following that, re-install the NUT package via System -> Package Manager.
-
@dennypage If it hasn't been stated today your continued support and insight in the forums with respect to NUT is greatly appreciated!
Followed your steps and had to reboot a couple of times for the system to allow me to install NUT.
Your Rock & Know It . . .
Thank You ~ Sir!
-
@dennypage said in NUT package:
Strange. You didn't update or install any new package since February 15th?
No I haven't done either of those. The only thing that changed in my pfSense configuration was adding two extra dynamic DNS providers last week. The errors just started suddenly yesterday with no changes made.
-
@teken Very kind of you to say. Thank you.
-
wow, I just checked my pfsense setup and all traces of the nut package are gone. Bet this happened with the 2.5 branch snafu. great.
-
@duruser Prior to following @dennypage I had the NUT service stopped. Upon completing the steps as outlined above I found that NUT was missing?!?
As noted up above the solution was to reboot twice and reinstall NUT again. One thing I was surprised to see is locking the system to Previous Stable actually made the system download and install 2.4.5 P1?!?
I understand the reasons for P1 but the system should have just installed EXACTLY what was on the machine prior to this crazy auto snafu!
Regardless, I'm back in action and must again state much thanks to @dennypage!
-
@teken
Same here. Nut was no longer installed. I had to re-install the package from package manager and reboot. Luckily all the settings were still there. All working good now. -
I'm trying to troubleshoot why my Synology NAS (as a NUT client) is always giving me an improper shutdown message when power comes back in our area. Here are the logs on my pfsense box:
Jul 17 06:09:38 shutdown 24254 power-down by root: Jul 17 06:09:36 upsd 18585 User monuser@192.168.10.10 logged into UPS [ups] Jul 17 06:09:33 upsmon 83642 Auto logout and shutdown proceeding Jul 17 06:09:33 upsmon 83642 Executing automatic power-fail shutdown Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups] Jul 17 06:08:24 php 53870 nut_email.php: Message sent to kevindd992002@yahoo.com OK Jul 17 06:08:13 upsd 18585 Client local-monitor@::1 set FSD on UPS [ups] Jul 17 06:08:13 upsmon 83642 UPS ups battery is low Jul 17 06:04:15 php 36392 nut_email.php: Message sent to kevindd992002@yahoo.com OK Jul 17 06:04:02 upsmon 83642 UPS ups on battery
I have the settings in the NUT server:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
-
@kevindd992002 said in NUT package:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
HOSTSYNC is the longest time that NUT will wait for all remote monitors to gracefully disconnect before continuing the shutdown sequence. The logs show a remote logging out:
Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups]
Once the last remote monitor has logged out, NUT is free to continue the local shutdown sequence. This sequence ends with a command to the UPS to turn off the load. The UPS does this after a short, and usually configurable, delay.
The issue is that the Synology is taking too long from the point that it shuts down NUT to the point that it actually powers off. To address this, you need to lengthen the amount of time the UPS delays before turning off the load. How to do this varies with UPS models and driver. If it is directly configurable via NUT, it will be variable "ups.delay.shutdown" (see upsc and upsrw doc).
If it is not directly configurable via NUT, consult your UPS documentation for info on how to configure the delay.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
HOSTSYNC is the longest time that NUT will wait for all remote monitors to gracefully disconnect before continuing the shutdown sequence. The logs show a remote logging out:
Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups]
Once the last remote monitor has logged out, NUT is free to continue the local shutdown sequence. This sequence ends with a command to the UPS to turn off the load. The UPS does this after a short, and usually configurable, delay.
The issue is that the Synology is taking too long from the point that it shuts down NUT to the point that it actually powers off. To address this, you need to lengthen the amount of time the UPS delays before turning off the load. How to do this varies with UPS models and driver. If it is directly configurable via NUT, it will be variable "ups.delay.shutdown" (see upsc and upsrw doc).
If it is not directly configurable via NUT, consult your UPS documentation for info on how to configure the delay.
I see, what you're saying. I guess I had a partial understanding of HOSTSYNC. How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server? Is it right after the successfully receive the FSD from the NUT server and initiate their shutdown sequences?
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
-
@kevindd992002 said in NUT package:
How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Each NUT monitor has a TCP connection to the server. When the monitor "logs out" it closes its TCP connection.
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server?
Implementation dependent. Usually receipt of the FSD causes the standard system shutdown sequence to begin, which terminates various processes including the NUT monitor process. The order in which the shutdown sequence stops processes is outside of NUT's control.
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
Note that the delay isn't something you set in the pfSense NUT config. You will need to use upsrw to update the delay setting in the UPS itself.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Each NUT monitor has a TCP connection to the server. When the monitor "logs out" it closes its TCP connection.
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server?
Implementation dependent. Usually receipt of the FSD causes the standard system shutdown sequence to begin, which terminates various processes including the NUT monitor process. The order in which the shutdown sequence stops processes is outside of NUT's control.
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
Note that the delay isn't something you set in the pfSense NUT config. You will need to use upsrw to update the delay setting in the UPS itself.
So as long as there are no more TCP connections to the NUT server, the server can freely start tye local shutdown sequence, got it.
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
-
@kevindd992002 said in NUT package:
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually powers off.
I’ve seen a lot of variation in shutdown time on the Synology depending upon what’s running (packages, docker containers, VMs). FWIW, DSM 7 seems to speed things up a bit.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually powers off.
I’ve seen a lot of variation in shutdown time on the Synology depending upon what’s running (packages, docker containers, VMs). FWIW, DSM 7 seems to speed things up a bit.
I see. But manually shutting down a Synology is not the same as it doing a "safe shutdown". A safe shutdown happens when an FSD is received and it doesn't actually power off the NAS but only dismounts the drives and turns off the services IIRC. As for the time difference between these two events, I'm not sure. I don't know how to accurately time a safe shutdown because the lights in the NAS are physically on when it happens.
-
@kevindd992002 There's no magic in NUT. A NUT "safe shutdown" is usually invoking a standard system command. For example, "/bin/shutdown -p +0" is used on pfSense.
I've recently upgraded from DSM6 to DSM7. I don't have a DSM6 system to examine, and I haven't had an opportunity to look at NUT in DSM7 to see how it is configured.
-
@dennypage said in NUT package:
@kevindd992002 There's no magic in NUT. A NUT "safe shutdown" is usually invoking a standard system command. For example, "/bin/shutdown -p +0" is used on pfSense.
I've recently upgraded from DSM6 to DSM7. I don't have a DSM6 system to examine, and I haven't had an opportunity to look at NUT in DSM7 to see how it is configured.
That's not entirely true. Have you recently noticed how a Synology NAS acts when it receives an FSD from the NUT server? It only does a safe shutdown, not a complete shutdown. I'm a 100% sure of this as I've been working in the past with Synology support regarding this. I didn't understand why it wasn't turning off completely so I thought I asked them. They said that it is by design that when you set a Synology NAS to shutdown with the "same as server" option, it only does a safe shutdown. It dismounts the drives and kills the services. When that process happens and power is cutoff by the UPS, then it's the same effect as a graceful/complete shutdown. I can vouch for this in DSM 6. I recently upgraded to DSM 7 but I'm not sure if it'll still work that way.
-
I think they're also calling it safe mode instead of shutdown. See post of Ramzi in this thread:
https://community.synology.com/enu/forum/1/post/121370?page=2&sort=oldest
In short, when Synology receives an FSD signal, it goes to safe mode. It does not shut down at all. It only gets turned off when the UPS cuts off power to it.
-
@kevindd992002 I don't think that some of the people writing in that thread know how NUT works.
The concept of a mode where the system is quiesced but is listening to the UPS and will start back up if the UPS signals that mains have returned is not supported with NUT. There is no concept of shutdown being started and subsequently aborted--when shutdown is started, is it expected to be completed through to the point of having the UPS cut the load.
Specifically to the Ramzi post (Synology support), he is saying that there is no way to have the Synology shut down but to leave the UPS powered. He is 100% correct in what he says. As noted above, this is exactly how NUT works. Once NUT shutdown is started, the UPS must be told to cut the load and remove power to the Synology.
If you prefer, you can use this phrasing instead of the prior:
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually enters safe mode.
This will probably be about a 1/4 second less than a power off test.
-
@dennypage said in NUT package:
@kevindd992002 I don't think that some of the people writing in that thread know how NUT works.
The concept of a mode where the system is quiesced but is listening to the UPS and will start back up if the UPS signals that mains have returned is not supported with NUT. There is no concept of shutdown being started and subsequently aborted--when shutdown is started, is it expected to be completed through to the point of having the UPS cut the load.
Specifically to the Ramzi post (Synology support), he is saying that there is no way to have the Synology shut down but to leave the UPS powered. He is 100% correct in what he says. As noted above, this is exactly how NUT works. Once NUT shutdown is started, the UPS must be told to cut the load and remove power to the Synology.
If you prefer, you can use this phrasing instead of the prior:
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually enters safe mode.
This will probably be about a 1/4 second less than a power off test.
Right. I completely understand that this is not supported because I've experienced this myself. Sometimes, people in the house turn on the generator (when there's a blackout) a few seconds after the FSD is sent by NUT to the remote monitors. This messes up everything.
I think we're pertaining to the same thing but different wordings :)
I use "safe mode + Synology waits for UPS to cut power" which is exactly what the Synology support person was saying. If safe mode is just 1/4 second less than a full power off test, then doing a manual power off test would then be an accurate test. I did do this in the past and since I also have an expansion unit in the mix, I remember that both units turned off in the span of two to three minutes.
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
-
@kevindd992002 said in NUT package:
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
You’ll have to time it. Difference between NUT log off and entry to safe mode. My guess (hope) would be that it is significantly less 60 seconds, but that’s just a guess. Note that you’ll need to recheck whenever you make significant changes to the Synology.
Just out of curiosity, are you running DSM 6 or 7?
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
You’ll have to time it. Difference between NUT log off and entry to safe mode. My guess (hope) would be that it is significantly less 60 seconds, but that’s just a guess. Note that you’ll need to recheck whenever you make significant changes to the Synology.
Just out of curiosity, are you running DSM 6 or 7?
When I manually shutdown the NAS, its remote monitor will also log off from the NUT server, correct? So difference between that time and when the NAS is completely shutdown (no lights and all).
Inwas running DSM 6 when I timed it to be 3 minutes from the time I initiated a shutdown sequence up to when the units are completely shutdown. Last week, I upgraded to DSM 7 and haven't checked since.
-
@kevindd992002 said in NUT package:
Last week, I upgraded to DSM 7 and haven't checked since.
I haven't timed a lot of things, but DSM 7 seems a lot faster across the board. I'm guessing that shutdown/safe mode will be faster as well. Your Mileage May Very.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Last week, I upgraded to DSM 7 and haven't checked since.
I haven't timed a lot of things, but DSM 7 seems a lot faster across the board. I'm guessing that shutdown/safe mode will be faster as well. Your Mileage May Very.
I'll try and time it today. For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
-
@kevindd992002 said in NUT package:
For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
You're looking for the amount of time between the point the Synology disconnects from pfSense NUT and the time it has fully entered safe mode.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
You're looking for the amount of time between the point the Synology disconnects from pfSense NUT and the time it has fully entered safe mode.
What I meant is when does the ups.delay.shutdown countdown start? When does the UPS start counting down before it cuts off power to its loads?
-
@kevindd992002 said in NUT package:
What I meant is when does the ups.delay.shutdown countdown start? When does the UPS start counting down before it cuts off power to its loads?
It starts when the UPS receives a shutdown command.
-
Do you have any ideas why is it that when I connect to my APU2C4 via a serial port, the NUT package cannot detect my USB UPS?
As soon as I remove the serial cable connection and restart NUT, it works just fine.
-
@kevindd992002 said in NUT package:
I connect to my APU2C4 via a serial port, the NUT package cannot detect my USB UPS?
Serial or USB ?
As soon as I remove the serial cable connection and restart NUT, it works just fine.
? Without the cable, NUT works ?
-
@gertjan said in NUT package:
@kevindd992002 said in NUT package:
I connect to my APU2C4 via a serial port, the NUT package cannot detect my USB UPS?
Serial or USB ?
As soon as I remove the serial cable connection and restart NUT, it works just fine.
? Without the cable, NUT works ?
APC UPS is connected via USB and I'm trying to keep a serial connection alive but when I do that it kills the APC USB connection. It only happens for the APC USB connection, not with other USB devices.
Yes, it works without the serial connection.
-
@kevindd992002 This is a basic NUT issue rather than a package issue. You'll need to diagnose this at the NUT level, and probably follow up with the NUT forum. I'd suggest you start by examining the log entries for NUT...
-
So I am able to monitor and query the values of the two ups that power my firewall, by adding a second one at Advance Settings under ups.conf
Any tips on what files, and their location the status widget uses to be able to modify and display data for each of the ups. (Hopefully something as simple as adding a loop or hardcoding a link for the second entry).
-
@josepr The pfSense NUT package is designed to supervise/monitor a single UPS only.
I'm curious as to why you have two UPSs on your firewall... does you firewall hardware have dual power supplies connected to separate UPSs?
-
@dennypage
Two ups each ones feeds a pdu which feeds a dual power supply host and other dual power equipment on a small 12U rack.
I am comfortable with programming and some html so wanted to see if could modify my copy of the package locally on my machine and make it work. -
@josepr
So was able to find the widget php and for now hardcode the second ups, havent had the chance to play with php before but will try to make it so that it can read all ups from the ups.conf and do something like the traffic or gateway widget were i can select which to display, get a chance to learn php.
-
@josepr said in NUT package:
Two ups each ones feeds a pdu which feeds a dual power supply host and other dual power equipment on a small 12U rack.
I see. Old school cool. Given the lower cost of hardware these days, I don't see that much anymore. These days I usually see dual systems, each with their own UPS, set up as a fault tolerant or load balanced pair. But you probably have that as well.
Unfortunately, while NUT itself has support for multiple power sources, the pfSense NUT package does not support that configuration. The pfSense NUT package is single UPS only. It would take quite a bit of work to make it support multiples in the various (extensive) configurations, and there really hasn't been any interest. You are actually the first to ask about it.
Probably your best bet is to use one of your other systems to run NUT on, and have that system initiate a remote shutdown of pfSense when the time comes. Sorry.
-
@josepr
Well I have it working for at least a nice widget displaying all the info for the attached ups, which was my main goal so i can get an idea of their status in one page, dont really have it shutting down anything at the moment maybe will get to that later.For anyone that is interested all I did was:
Modified '/usr/local/pkg/nut/nut.inc'
Added two functions:function nut_conf() { global $config; $conf = &$config['installedpackages']['nut']['config'][0]; return $conf; } function list_ups() { $upss = array(); $config = nut_conf(); if (isset($config['name'])) { $upss[] = $config['name']; } if (isset($config['ups_conf'])) { $conf = parse_ini_string(base64_decode($config['ups_conf']), true); foreach ($conf as $name => $_conf) { $upss[] = $name; } } return $upss; }
and modified the start of the nut_ups_status function
function nut_ups_status($name = null) { global $config; $status = array(); $nut_config = &$config['installedpackages']['nut']['config'][0]; if (isset($nut_config['type'])) { $type = $nut_config['type']; } else { $type = 'disabled'; } if ($type == 'disabled') { $status['_summary'] = "Monitoring is not enabled"; return $status; } $ups = $name ?: $nut_config['name'];
Then i modified /usr/local/www/widgets/widgets/nut_status.widget.php
<?php require_once("/usr/local/www/widgets/include/nut_status.inc"); require_once("/usr/local/pkg/nut/nut.inc"); if ($_REQUEST && $_REQUEST['ajax']) { print_all_table(); #Get multiple tables instead of just one. exit; } function print_row($desc, $value) { #didnt touch } function print_row_pct($desc, $value) { #didnt touch } function print_table($name) { $status = nut_ups_status($name); ##Only added the $name parameter, the rest stayed the same } #New Function to print a table per UPS function print_all_table() { print '<table class="table table-bordered table-dark"> <tbody>'; foreach (list_ups() as $name) { print '<tr class="panel-title"><td><b>' . $name . '</b></td></tr>'; print '<tr><td><div">'; print '<table class="table table-striped table-hover table-condensed">'; print '<tbody>' . print_table($name) . '</tbody>'; print '</table>'; print '</div></td></tr>'; } print '</tbody></table>'; } ?> <div class="table-responsive" id = "nuttable"> <?php print_all_table() ?> </div> <script type="text/javascript"> #Didnt touch </script>
And then add my second ups under the ups.conf advanced settings.
-
@josepr It might be a good idea to focus on correct NUT configuration before worrying about the monitoring widget. It appears that you've set up a situation in which the pfSense box will shutdown if either UPS fails, which I'm pretty sure is not what you want. For the work that you're doing, it would be simpler to abandon the pfSense NUT package and just configure NUT directly. Of course, you'll loose the pretty widget, but it would be a lot safer.
The APC UPSs are SNMP based, and also have their own web interface. Is there a reason you are focused on having status displayed in pfSense widgets? Instead of using their built-in web monitoring? Or an SNMP web based monitor such as Librenms or the like?
-
@dennypage
Actions are not taken for the device configured at ups.conf, this is only for the upsd service which setups communications with devices and can also provide access to this devices to other remote services such as a remote upsmon instance in another server.
upsmon is the service that actually enforces action and it has its own configuration which the package only generates an entry for the UPS configured the normal way, I did add a two liner to the Advance part so that it acts when both UPS are down.Auto generated by the package for UPSMON
MONITOR ups-2 1 local-monitor **** master SHUTDOWNCMD "/sbin/shutdown -p +0" POWERDOWNFLAG /etc/killpower
With addition to the upsmon.conf at advance in gui
MONITOR ups-2 1 local-monitor **** master SHUTDOWNCMD "/sbin/shutdown -p +0" POWERDOWNFLAG /etc/killpower ###ADDED LINES upsmon.conf#### MONITOR ups-1 1 local-monitor2 **** master MINSUPPLIES 1
Did also have to add an extra user in upsd.users as the package generates a random password everytime it saves for local-monitor
[local-monitor2] password=**** upsmon master
This tells upsmon that it should monitor the 2 ups, that both UPS provide power for one PSU in the server and that it needs a minimum of one power supply to operate, so it is now fault tolerant (Will only conduct a shutdown if both UPS are down/critical power).
The reason i add them to PFSENSE is that it depends on both UPS so I wanted a way to make sure it reflected this and didnt try anything drastic when one UPS is offline.
It is my always on device and the last device to be shutdown and can it act as a ups provider to any other nut clients in other devices. -
@josepr said in NUT package:
This tells upsmon that it should monitor the 2 ups, that both UPS provide power for one PSU in the server and that it needs a minimum of one power supply to operate, so it is now fault tolerant (Will only conduct a shutdown if both UPS are down/critical power).
MINSUPPLIES was not shown in the configuration you posted previously.
-
@dennypage
Yes, but i also hadnt added
MONITOR ups-1 1 local-monitor2 **** master
to upsmon so it was not watching/monitoring the second ups.