-
@dennypage Right, and that doesn’t surprise me. What I can’t figure out is why reloading the network interfaces causes a connection error for a USB-connected UPS.
-
@mlake said in NUT Package (2.8.1 and above):
Dec 3 10:17:29 upsmon 37971 Signal 15: exiting
Dec 3 10:17:29 upsd 22980 User local-monitor@127.0.0.1 logged out from UPS [ups]
Dec 3 10:17:29 upsd 22980 mainloop: Interrupted system call
Dec 3 10:17:29 upsd 22980 Signal 15: exiting
Dec 3 10:17:29 usbhid-ups 54011 Signal 15: exiting
Dec 3 10:17:29 php-fpm 405 /rc.start_packages: Starting service nut
Dec 3 10:17:29 upsmon 94129 Startup successful
Dec 3 10:17:29 upsmon 94715 UPS [ups]: connect failed: Connection failure: Connection refused
Dec 3 10:17:29 upsmon 94715 Communications with UPS ups lost
Dec 3 10:17:29 usbhid-ups 3404 Startup successful
Dec 3 10:17:30 upsd 64401 listening on 127.0.0.1 port 3493
Dec 3 10:17:30 upsd 64401 listening on ::1 port 3493
Dec 3 10:17:30 upsd 64401 not listening on 127.0.0.1 port 3493
Dec 3 10:17:30 upsd 64401 listening on 172.17.37.1 port 3493
Dec 3 10:17:30 upsd 64401 Connected to UPS [ups]: usbhid-ups-ups
Dec 3 10:17:30 upsd 64401 Found 1 UPS defined in ups.conf
Dec 3 10:17:30 upsd 64746 Startup successfulNUT is being explicitly stopped, as noted by the "Signal 15: exiting" messages. Signal 15 is SIGTERM, which is the default signal. See the rc_stop() function in /usr/local/etc/rc.d/nut.sh.
-
I managed to figure out my remaining connection lost messages after the package update. It's related to a Suricata issue I'm having. The interfaces are going down and up briefly for Suricata restarting interfaces, and is causing the connection to go down long enough to send 2 connection lost messages followed by a connection established.
Each time this consistently happens over a 9 second period from first lost to the established message. Is there a way to tune nut to continue trying the connection for say 15 seconds before calling it a lost connection? This would eliminate these problems of interfaces going down briefly for an interface restart.
Otherwise over the last week or so I haven't seen a single instance of the connection lost message that wasn't correlated directly to a suricata interface going down and then up. Nut is doing what it is supposed to, so I can't complain, but it would be ideal if there is a way to slow down nut a little before firing off the logs/notices for a connection lost. Nut catches it quickly!
-
@sgnoc said in NUT Package (2.8.1 and above):
The interfaces are going down and up briefly for Suricata restarting interfaces, and is causing the connection to go down long enough to send 2 connection lost messages followed by a connection established.
If the interface is being restarted, It's not just that you are loosing a connection, but that NUT is being completely restarted, yes? Just like the issue posted immediately prior to yours?
-
@dennypage Thanks, I hadn't connected the dots even after I read that post earlier. I went and checked previous logs for connection lost messages and the timestamps do add up. What's odd is I don't remember receiving messages for interfaces going down/up, although I hadn't previously had another issues causing a core dump and a forced reload of the interfaces, so I may just have not had cause for those messages to be generated previously. If that is expected behavior as stated, then all should be well. I guess it is a good thing to get a connection lost message indicating that there was a suricata core dump causing the interfaces to reload. Thanks and sorry I didn't connect the dots reading the above message!
-
@sgnoc I believe there have been a few comments about Suricata flapping interfaces recently...
Edit: here is one: https://forum.netgate.com/post/1138736
-
@dennypage Is NUT supposed to be restarted when an interface changes? Or is the Signal 15 exit unexpected? Something definitely changed between pfSense and NUT after updating to 2.8.2 because I never got the connection errors before and now I get them once or twice a day (when the remote VPN renegotiates).
-
@mlake said in NUT Package (2.8.1 and above):
@dennypage Is NUT supposed to be restarted when an interface changes? Or is the Signal 15 exit unexpected? Something definitely changed between pfSense and NUT after updating to 2.8.2 because I never got the connection errors before and now I get them once or twice a day (when the remote VPN renegotiates).
All pfSense packages are restarted when an interface changes state. This is outside of NUT’s control.
There may have been a change in how OpenVPN manages interfaces in recent releases 23.09/2.7.1, but I don’t know enough about OpenVPN to say. You might ask about interference handling changes in the OpenVPN forum.
-
@dennypage Thanks for the clarification. However, the over-notification problem appears to be a NUT/pfSense issue and doesn't seem to have anything to do with OpenVPN specifically. I disabled my OpenVPN connections and am still able to get the the "Communications with UPS ups lost" / "Communications with UPS ups established" emails by simply opening WAN interface page, clicking Save without changing anything, and then clicking Apply Changes.
Did something change in NUT's monitoring/polling system between 2.7.x and 2.8.x? It seems like NUT is now detecting a 'process restart' as a 'UPS disconnect'.
-
@mlake said in NUT Package (2.8.1 and above):
I disabled my OpenVPN connections and am still able to get the the "Communications with UPS ups lost" / "Communications with UPS ups established" emails by simply opening WAN interface page, clicking Save without changing anything, and then clicking Apply Changes.
This is the same thing as above, however in this case you are restarting the WAN interface instead of the OpenVPN interface.
-
@mlake said in NUT Package (2.8.1 and above):
Did something change in NUT's monitoring/polling system between 2.7.x and 2.8.x? It seems like NUT is now detecting a 'process restart' as a 'UPS disconnect'.
Another item of note is that email notifications for non root processes (such as NUT) were broken in 23.05/2.7.0 which would have suppressed the emails. Pushover (and others) worked though.
-
@dennypage Curious - this may speak to the reason why I wasn't seeing the emails when a quick change happened in 23.05, but I certainly did get emails when I had a power outage.
For now, I'm going to disable the email notifications because they're not helpful and are just reporting false-positives.
It seems that when restarted by the system, there is a race condition and NUT should probably withhold sending notifications until the system has had time to bring interfaces back up.
Thanks for responding to these and at least clarifying that some of the underlying behavior is expected.
-
@mlake Do me a favor?
Try with this patch and let me know if it suppresses the problem when your OpenVPN connection flaps.
You will need to go into UPS -> settings and press Save in order to activate the change.
--- /usr/local/pkg/nut/nut.inc.org 2023-11-17 05:42:10.000000000 -0800 +++ /usr/local/pkg/nut/nut.inc 2023-12-05 15:20:38.575637000 -0800 @@ -82,14 +82,19 @@ $start .= "\n /usr/bin/killall -q -9 $driver"; } - /* Service status keys off upsmon, so start it first. */ - $start .= "\n /usr/local/sbin/upsmon"; if (isset($driver)) { $start .= "\n /usr/local/sbin/upsdrvctl start &"; - /* Since we are starting the driver in backgroud, give it a moment to start. */ - $start .= "\n sleep 1"; $start .= "\n /usr/local/sbin/upsd -u root"; + + /* + * Since we are starting the driver in backgroud, give + * the driver and upsd a moment to start. + */ + $start .= "\n sleep 1"; } + + /* NB: Service status keys off of upsmon. */ + $start .= "\n /usr/local/sbin/upsmon"; $start .= "\n return 0"; $stop = "echo stopping NUT"; -
@dennypage Wow, thank you! Got the patch applied (made a "nut.inc.org" copy and had to put "/usr" as the Base Directory to get it to take) and the results look promising. I've tried a few interface resets that normally cause the notifications and I've only gotten it to trigger once. Unplugging the UPS and plugging it back in works as expected. I'm going to let this sit for a few days and I'll check back in.
-
@mlake said in NUT Package (2.8.1 and above):
(made a "nut.inc.org" copy and had to put "/usr" as the Base Directory to get it to take) and the results look promising.
Use the patches GUI :
Give it a description, like the URL of this fprum thread.
No URL/Commit ID,
Paste the patch as fond above,
Path strip count : set to 1
Base dir = /
Auto apply : free of choice.
save, and the patch is now ready to be applied.
So Apply.
Restart the UPS service as indicated above. -
@Gertjan Ah thanks, yes! (First time applying a manual patch... now the UI makes more sense).
-
@dennypage The patch seems to have solved my Suricata restarting interfaces too. I applied the patch and restarted Suricata, which would normally generate 2 connection lost and one connect established messages when NUT restarted, but the messages were supressed. The sleep delays seem to have done the trick. Thanks a bunch!
-
Im having issues getting pfsense to conect to the NUT server on my Synology whilst that connection work on other servers on my system:
"ld-elf.so.1: Shared object "libssl.so.30" not found, required by "upsmon"
nut sysutils 2.8.2
PfSense Version 2.7.0-RELEASE (amd64). Installed clean, not upgradedSynology 192.168.2.3
"synoups.conf"ups_enabled="yes" ups_mode="usb" ups_safeshutdown="no" ups_acl="192.168.2.1|192.168.2.12||||""upsd.users"
[monuser] password = slaveHome Assistant connects to this and outputs ;
"domain": "nut", "title": "synology.home.lan:3493", "data": { "host": "synology.home.lan", "port": 3493, "username": "**REDACTED**", "password": "**REDACTED**"Home Assistant Status 192.168.2.12 (excert):
"status": { "battery.capacity": "9.00", "battery.charge": "100", "battery.charge.low": "20", "battery.charge.restart": "0", "battery.energysave": "no", "battery.protection": "yes"PfSense config 192.168.2.1
cat /usr/local/etc/nut/upsmon.conf MONITOR ups@192.168.2.3 1 monuser slave slaveLog output when started manually:
starting NUT ld-elf.so.1: Shared object "libssl.so.30" not found, required by "upsmon" -
@bashers46899
Looked up the error "shared object "libssl.so.30" not found" and can see it was possibly resoloved by an upgrade from 2.7.0 to 2.7.3.
I've done this, reinstalled the NUT package amd it all seems fine now:
-
@bashers46899 said in NUT Package (2.8.1 and above):
Im having issues getting pfsense to conect to the NUT server on my Synology whilst that connection work on other servers on my system:
"ld-elf.so.1: Shared object "libssl.so.30" not found, required by "upsmon"
nut sysutils 2.8.2
PfSense Version 2.7.0-RELEASE (amd64). Installed clean, not upgradedThe version of OpenSSL changed with 23.09/2.7.1. The error you experienced happens when a version of the package that is intended for 23.09/2.7.1 or above is installed on an older version of pfSense (23.05/2.7.0).
NUT 2.8.2 should not have been offered for installation with 23.05/2.7.0. That said, you are not the only person who has experienced this, so I think there is/was a hiccup related to the package repo.
EDIT: there is a note about resolving the package repo issue here.
