-
I'm trying to troubleshoot why my Synology NAS (as a NUT client) is always giving me an improper shutdown message when power comes back in our area. Here are the logs on my pfsense box:
Jul 17 06:09:38 shutdown 24254 power-down by root: Jul 17 06:09:36 upsd 18585 User monuser@192.168.10.10 logged into UPS [ups] Jul 17 06:09:33 upsmon 83642 Auto logout and shutdown proceeding Jul 17 06:09:33 upsmon 83642 Executing automatic power-fail shutdown Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups] Jul 17 06:08:24 php 53870 nut_email.php: Message sent to kevindd992002@yahoo.com OK Jul 17 06:08:13 upsd 18585 Client local-monitor@::1 set FSD on UPS [ups] Jul 17 06:08:13 upsmon 83642 UPS ups battery is low Jul 17 06:04:15 php 36392 nut_email.php: Message sent to kevindd992002@yahoo.com OK Jul 17 06:04:02 upsmon 83642 UPS ups on batteryI have the settings in the NUT server:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
-
@kevindd992002 said in NUT package:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
HOSTSYNC is the longest time that NUT will wait for all remote monitors to gracefully disconnect before continuing the shutdown sequence. The logs show a remote logging out:
Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups]Once the last remote monitor has logged out, NUT is free to continue the local shutdown sequence. This sequence ends with a command to the UPS to turn off the load. The UPS does this after a short, and usually configurable, delay.
The issue is that the Synology is taking too long from the point that it shuts down NUT to the point that it actually powers off. To address this, you need to lengthen the amount of time the UPS delays before turning off the load. How to do this varies with UPS models and driver. If it is directly configurable via NUT, it will be variable "ups.delay.shutdown" (see upsc and upsrw doc).
If it is not directly configurable via NUT, consult your UPS documentation for info on how to configure the delay.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
So at 6:08:13, the NUT server set an FSD signal to its clients. From what I understand from our past discussions, HOSTSYNC 300 will make the NUT server wait for 5 minutes before it actually shuts down itself. If that's the case, why do the logs say that the shutdown process started at 6:09:33? That's a mere 1 minute 20 seconds from the time it sent an FSD and I don't think this is enough time for the Synology to initiate that shutdown sequence.
HOSTSYNC is the longest time that NUT will wait for all remote monitors to gracefully disconnect before continuing the shutdown sequence. The logs show a remote logging out:
Jul 17 06:09:33 upsd 18585 User monuser@192.168.10.10 logged out from UPS [ups]Once the last remote monitor has logged out, NUT is free to continue the local shutdown sequence. This sequence ends with a command to the UPS to turn off the load. The UPS does this after a short, and usually configurable, delay.
The issue is that the Synology is taking too long from the point that it shuts down NUT to the point that it actually powers off. To address this, you need to lengthen the amount of time the UPS delays before turning off the load. How to do this varies with UPS models and driver. If it is directly configurable via NUT, it will be variable "ups.delay.shutdown" (see upsc and upsrw doc).
If it is not directly configurable via NUT, consult your UPS documentation for info on how to configure the delay.
I see, what you're saying. I guess I had a partial understanding of HOSTSYNC. How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server? Is it right after the successfully receive the FSD from the NUT server and initiate their shutdown sequences?
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
-
@kevindd992002 said in NUT package:
How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Each NUT monitor has a TCP connection to the server. When the monitor "logs out" it closes its TCP connection.
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server?
Implementation dependent. Usually receipt of the FSD causes the standard system shutdown sequence to begin, which terminates various processes including the NUT monitor process. The order in which the shutdown sequence stops processes is outside of NUT's control.
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
Note that the delay isn't something you set in the pfSense NUT config. You will need to use upsrw to update the delay setting in the UPS itself.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
How does the NUT server know which remote monitors to wait for before it actually considers all remote monitors already logged out? Does it have some kind of a table?
Each NUT monitor has a TCP connection to the server. When the monitor "logs out" it closes its TCP connection.
Also, at what point exactly do remote monitors (Synology in this case) gracefully disconnect from the NUT server?
Implementation dependent. Usually receipt of the FSD causes the standard system shutdown sequence to begin, which terminates various processes including the NUT monitor process. The order in which the shutdown sequence stops processes is outside of NUT's control.
As for "ups.delay.shutdown", I just checked and yes the NUT GUI in pfsense does show that ups.delay.shutdown = 20 for both my Eaton and APC UPS'es, so I should be able to configure it to a longer delay.
Note that the delay isn't something you set in the pfSense NUT config. You will need to use upsrw to update the delay setting in the UPS itself.
So as long as there are no more TCP connections to the NUT server, the server can freely start tye local shutdown sequence, got it.
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
-
@kevindd992002 said in NUT package:
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually powers off.
I’ve seen a lot of variation in shutdown time on the Synology depending upon what’s running (packages, docker containers, VMs). FWIW, DSM 7 seems to speed things up a bit.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Ok, I'll check out the delay. Do you have good increased value recommendation to start with?
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually powers off.
I’ve seen a lot of variation in shutdown time on the Synology depending upon what’s running (packages, docker containers, VMs). FWIW, DSM 7 seems to speed things up a bit.
I see. But manually shutting down a Synology is not the same as it doing a "safe shutdown". A safe shutdown happens when an FSD is received and it doesn't actually power off the NAS but only dismounts the drives and turns off the services IIRC. As for the time difference between these two events, I'm not sure. I don't know how to accurately time a safe shutdown because the lights in the NAS are physically on when it happens.
-
@kevindd992002 There's no magic in NUT. A NUT "safe shutdown" is usually invoking a standard system command. For example, "/bin/shutdown -p +0" is used on pfSense.
I've recently upgraded from DSM6 to DSM7. I don't have a DSM6 system to examine, and I haven't had an opportunity to look at NUT in DSM7 to see how it is configured.
-
@dennypage said in NUT package:
@kevindd992002 There's no magic in NUT. A NUT "safe shutdown" is usually invoking a standard system command. For example, "/bin/shutdown -p +0" is used on pfSense.
I've recently upgraded from DSM6 to DSM7. I don't have a DSM6 system to examine, and I haven't had an opportunity to look at NUT in DSM7 to see how it is configured.
That's not entirely true. Have you recently noticed how a Synology NAS acts when it receives an FSD from the NUT server? It only does a safe shutdown, not a complete shutdown. I'm a 100% sure of this as I've been working in the past with Synology support regarding this. I didn't understand why it wasn't turning off completely so I thought I asked them. They said that it is by design that when you set a Synology NAS to shutdown with the "same as server" option, it only does a safe shutdown. It dismounts the drives and kills the services. When that process happens and power is cutoff by the UPS, then it's the same effect as a graceful/complete shutdown. I can vouch for this in DSM 6. I recently upgraded to DSM 7 but I'm not sure if it'll still work that way.
-
I think they're also calling it safe mode instead of shutdown. See post of Ramzi in this thread:
https://community.synology.com/enu/forum/1/post/121370?page=2&sort=oldest
In short, when Synology receives an FSD signal, it goes to safe mode. It does not shut down at all. It only gets turned off when the UPS cuts off power to it.
-
@kevindd992002 I don't think that some of the people writing in that thread know how NUT works.
The concept of a mode where the system is quiesced but is listening to the UPS and will start back up if the UPS signals that mains have returned is not supported with NUT. There is no concept of shutdown being started and subsequently aborted--when shutdown is started, is it expected to be completed through to the point of having the UPS cut the load.
Specifically to the Ramzi post (Synology support), he is saying that there is no way to have the Synology shut down but to leave the UPS powered. He is 100% correct in what he says. As noted above, this is exactly how NUT works. Once NUT shutdown is started, the UPS must be told to cut the load and remove power to the Synology.
If you prefer, you can use this phrasing instead of the prior:
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually enters safe mode.
This will probably be about a 1/4 second less than a power off test.
-
@dennypage said in NUT package:
@kevindd992002 I don't think that some of the people writing in that thread know how NUT works.
The concept of a mode where the system is quiesced but is listening to the UPS and will start back up if the UPS signals that mains have returned is not supported with NUT. There is no concept of shutdown being started and subsequently aborted--when shutdown is started, is it expected to be completed through to the point of having the UPS cut the load.
Specifically to the Ramzi post (Synology support), he is saying that there is no way to have the Synology shut down but to leave the UPS powered. He is 100% correct in what he says. As noted above, this is exactly how NUT works. Once NUT shutdown is started, the UPS must be told to cut the load and remove power to the Synology.
If you prefer, you can use this phrasing instead of the prior:
Start a shutdown on the Synology and check the amount of time between the log off message in pfSense and the time the Synology actually enters safe mode.
This will probably be about a 1/4 second less than a power off test.
Right. I completely understand that this is not supported because I've experienced this myself. Sometimes, people in the house turn on the generator (when there's a blackout) a few seconds after the FSD is sent by NUT to the remote monitors. This messes up everything.
I think we're pertaining to the same thing but different wordings :)
I use "safe mode + Synology waits for UPS to cut power" which is exactly what the Synology support person was saying. If safe mode is just 1/4 second less than a full power off test, then doing a manual power off test would then be an accurate test. I did do this in the past and since I also have an expansion unit in the mix, I remember that both units turned off in the span of two to three minutes.
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
-
@kevindd992002 said in NUT package:
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
You’ll have to time it. Difference between NUT log off and entry to safe mode. My guess (hope) would be that it is significantly less 60 seconds, but that’s just a guess. Note that you’ll need to recheck whenever you make significant changes to the Synology.
Just out of curiosity, are you running DSM 6 or 7?
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Would that value be a good value for the power off delay of the UPS then? I just need to make sure that the UPS has still enough juice to wait for that long.
You’ll have to time it. Difference between NUT log off and entry to safe mode. My guess (hope) would be that it is significantly less 60 seconds, but that’s just a guess. Note that you’ll need to recheck whenever you make significant changes to the Synology.
Just out of curiosity, are you running DSM 6 or 7?
When I manually shutdown the NAS, its remote monitor will also log off from the NUT server, correct? So difference between that time and when the NAS is completely shutdown (no lights and all).
Inwas running DSM 6 when I timed it to be 3 minutes from the time I initiated a shutdown sequence up to when the units are completely shutdown. Last week, I upgraded to DSM 7 and haven't checked since.
-
@kevindd992002 said in NUT package:
Last week, I upgraded to DSM 7 and haven't checked since.
I haven't timed a lot of things, but DSM 7 seems a lot faster across the board. I'm guessing that shutdown/safe mode will be faster as well. Your Mileage May Very.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
Last week, I upgraded to DSM 7 and haven't checked since.
I haven't timed a lot of things, but DSM 7 seems a lot faster across the board. I'm guessing that shutdown/safe mode will be faster as well. Your Mileage May Very.
I'll try and time it today. For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
-
@kevindd992002 said in NUT package:
For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
You're looking for the amount of time between the point the Synology disconnects from pfSense NUT and the time it has fully entered safe mode.
-
@dennypage said in NUT package:
@kevindd992002 said in NUT package:
For the UPS delay, when does that delay start? Is it as soon as I see the auto shutdown message in the pfsense NUT?
You're looking for the amount of time between the point the Synology disconnects from pfSense NUT and the time it has fully entered safe mode.
What I meant is when does the ups.delay.shutdown countdown start? When does the UPS start counting down before it cuts off power to its loads?
-
@kevindd992002 said in NUT package:
What I meant is when does the ups.delay.shutdown countdown start? When does the UPS start counting down before it cuts off power to its loads?
It starts when the UPS receives a shutdown command.
-
Do you have any ideas why is it that when I connect to my APU2C4 via a serial port, the NUT package cannot detect my USB UPS?
As soon as I remove the serial cable connection and restart NUT, it works just fine.
