-
@dennypage Sorry, it was the issue with the APC BackUPS models causing a shutdown during the selftest.
https://github.com/networkupstools/nut/issues/2104
I know the fix has been applied to upstream NUT, I just wanted to make sure the fix is present in the package, because this was happening in the 2.8.2 version of the package on pfSense 2.7.2. I did just upgrade to 2.8.2_1.
-
@4pack Yes, that is addressed in nut-devel-2024.01.03.
-
I have four Synologies. Three out of four (ds412, two ds120j) connect to my NUT server (running on pfSense) just fine.
My ds1522 cannot connect to the NUT server.
telnet <NUT server> works fine and lets me login. So telnetd is responding correctly on the NUT server.
telnet <NUT server> 3493 says "Unable to connnect to the remote host: Connection refused." The same command on any other Synology connects to the NUT server no problem.
I can't find anything in the firewall on the NUT server that would block the DS1522. I have even tried putting a special firewall rule to allow the DS1522.
-
@hspindel said in NUT Package (2.8.1 and above):
telnet <NUT server> works fine and lets me login. So telnetd is responding correctly on the NUT server.
telnet <NUT server> 3493 says "Unable to connnect to the remote host: Connection refused."
These are two totally different things.
In the first case, you are attempting to connect to whatever process is listening on port 23, which would be telnetd. As an aside, having telnetd enabled on your firewall is very bad practice--you should use ssh instead.
In the second case, you are attempting to connect to whatever process is listening on port 3493, which would be upsd. Telnetd would not be involved in any way.
You are correct to be looking at your firewall rules, however you should also double check the configuration on the Synology: Make sure you are using the IP address rather than a hostname, and If you have the firewall enabled on the Synology, check that as well.
-
@dennypage said in NUT Package (2.8.1 and above):
@hspindel said in NUT Package (2.8.1 and above):
telnet <NUT server> works fine and lets me login. So telnetd is responding correctly on the NUT server.
telnet <NUT server> 3493 says "Unable to connnect to the remote host: Connection refused."
These are two totally different things.
In the first case, you are attempting to connect to whatever process is listening on port 23, which would be telnetd. As an aside, having telnetd enabled on your firewall is very bad practice--you should use ssh instead.
In the second case, you are attempting to connect to whatever process is listening on port 3493, which would be upsd. Telnetd would not be involved in any way.
You are correct to be looking at your firewall rules, however you should also double check the configuration on the Synology: Make sure you are using the IP address rather than a hostname, and If you have the firewall enabled on the Synology, check that as well.
Yes, I understand they are different things. Sorry for confusing things.
All four of my Synologies are using an IP address, not a hostname. telnet <NUT Server IP> 3493 works fine on three of the four Synologies.
I have tried using tcpdump to see how the Synology is talking to the NUT server. Unless I'm using tcpdump wrong, the Synology isn't even trying to connect. tcpdump port 3493 doesn't show a thing on the problem Synology, while it shows traffic on the other Synologies.
I have tried disabling the Synology firewall completely, without effect.
-
@hspindel said in NUT Package (2.8.1 and above):
I have tried using tcpdump to see how the Synology is talking to the NUT server. Unless I'm using tcpdump wrong, the Synology isn't even trying to connect. tcpdump port 3493 doesn't show a thing on the problem Synology, while it shows traffic on the other Synologies.
Just to be clear, tcpdump on the Synology shows no outbound packets? Do you have multiple ethernet interfaces on the Synology?
-
@dennypage said in NUT Package (2.8.1 and above):
@hspindel said in NUT Package (2.8.1 and above):
I have tried using tcpdump to see how the Synology is talking to the NUT server. Unless I'm using tcpdump wrong, the Synology isn't even trying to connect. tcpdump port 3493 doesn't show a thing on the problem Synology, while it shows traffic on the other Synologies.
Just to be clear, tcpdump on the Synology shows no outbound packets? Do you have multiple ethernet interfaces on the Synology?
tcpdump by itself on the questionable Synology shows plenty of packets. tcpdump port 3493 shows nothing.
-
@hspindel said in NUT Package (2.8.1 and above):
tcpdump port 3493 shows nothing.
Port 3493 was implied.
You see no outbound packets even when you do
telnet firewall 3493on the Synology?
You said that you received a connection refused error when doing the telnet. If there are no outbound packets on the interface, then the issue would have to be the Synology itself. Synology's firewall would be my guess. I've never used Synology's firewall, so I really can't help much there.
-
@dennypage Yes, I have further narrowed this down to be a problem on the Synology.
I have not yet been able to figure out why, but when the Synology attempts to contact pfsense (even by using numeric IP instead of DNS) it actually contacts itself (localhost). Since NUT doesn't exist on localhost, that's why I get a connection refused response.
Just FYI, arp cache looks correct and even flushing the arp cache on the Synology didn't help.
-
@dennypage said in NUT Package (2.8.1 and above):
FWIW, I don't currently know of outstanding issues with pfSense-pkg-nut 2.8.2_1 / nut-devel-2024.01.03 other than the tripplite_usb and nutdrv_qx issues discussed over the last week. I'm not currently aware of anything affecting usbhid-ups or upsmon.
I have a Tripp Lite ECO850LCD using the usbhid-ups driver. I am running 2.8.2_1 nut-devel-2024.01.03. I still have the problem that every few days the hid driver looses connection to the Tripp Lite and I have to physically disconnect the USB and restart Nut. Is this problem supposed to be fixed now?
-
@hspindel said in NUT Package (2.8.1 and above):
I have not yet been able to figure out why, but when the Synology attempts to contact pfsense (even by using numeric IP instead of DNS) it actually contacts itself (localhost).
The actual localhost (logical) interface? Or an ethernet (physical) interface? Btw, you didn't answer the question regarding multiple interfaces being present on the Synology.
What do you have in
Control Panel -> Network -> General -> Network Interfaceand
Control Panel -> Network -> General -> Advanced Settings?
If you want me to look at the tcpdump output from the Synology during the connection attempt, please send it via PM.
-
@ghound said in NUT Package (2.8.1 and above):
I still have the problem that every few days the hid driver looses connection to the Tripp Lite and I have to physically disconnect the USB and restart Nut.
I don't know about this issue. What do you see in the logs for kernel USB and usbhid-ups?
When the unit disappears, does it still show up when you do
usbconfig -v?
-
I recently reloaded the driver, so I may have to wait a few days for the problem to show up. In the mean time this post is the problem I thought I was having (even though I have a Tripp Lite not a CyberPower, others reported your devel packaged helped Tripp Lite UPS):
From your Feb 22, 2023 post...
For those of you affected by the CyberPower issue (and those that will be in the future):
First, I want to thank all of you for your help in tracking this down. Particularly @shaffergr, who was kind enough, and trusting enough, to run test builds for me against his CyberPower.
1 tracked this down to a double free by the ush code in nut. FWIW, I don't believe that this issue is completely limited to CyberPower UPSs, but it is probably pretty difficult to encounter with other units as hitting the issue requires a reconnect of the UPS on the bus. CyberPower units are well known to randomly disconnect and reconnect seemingly at random.
For those wondering why this issue did not happen in prior to 23.01: 22.05 and below used nut version 2.7.4. In 2.7.4, nut did not actually close the device when a disconnect happened. Presumably, this may have resulted in a memory leak at shutdown, but I didn't explore enough to confirm. In 2.8.0, when a device disconnects nut actually closes the device when it disconnects. In fact, it closes it twice on all systems other than Linux. There is even a comment in the code noting how the double close would cause corruption on Linux systems. Unfortunately contrary to the code comments it also causes corruption in FreeBSD.
Having traced this down in the 2.8.0 release code, when I went to the current development version I surprised to find that someone had beat me to it, and fixed the issue back in August. It's just not been release yet. :)
To resolve this will require a new version of nut. Either a new release from the nut team, or for the pfSense team to move from the 2.8.0 release version to the current development version of nut. The nut team is looking toward 2.8.1, but it appears that they have a few things they still want to address before putting that out. I will explore the concept of moving to nut-devel with the pfSense team as time permits (I think they are pretty busy right now). But no matter which way it goes, it's going to take some time.
In the interim, the only known work-around using the release code (discovered by @tman222) is to add the line "interruptonly"
to the Extra Arguments to driver section. This will cut down some of the information you can see about your UPS, but the important stuff needed to monitor and shutdown should still be there.
Alternatively, if brave souls are interested, I have a build of usbhid-ups made from the FreeBSD nut-devel package. I do not have a CyberPower available so I haven't been able to directly test it, but I expect that it will work.
If you you decide you would like a copy, reach out to me and I will see about getting it to you. For reference, the shasum and sha256sum checksums are:
49ce9131502bb86789ee97b7fb3fc81fc9f8fff usbhid-ups
999a2653559dbc50ecc8ba592a67587ble307a1495f6e8ebbd3d8e90e3967133 usbhid-ups -
@dennypage said in NUT Package (2.8.1 and above):
@hspindel said in NUT Package (2.8.1 and above):
I have not yet been able to figure out why, but when the Synology attempts to contact pfsense (even by using numeric IP instead of DNS) it actually contacts itself (localhost).
The actual localhost (logical) interface? Or an ethernet (physical) interface? Btw, you didn't answer the question regarding multiple interfaces being present on the Synology.
What do you have in
Control Panel -> Network -> General -> Network Interfaceand
Control Panel -> Network -> General -> Advanced Settings?
If you want me to look at the tcpdump output from the Synology during the connection attempt, please send it via PM.
I can hardly believe it, but I finally fixed this. The problem had nothing to do with NUT, which is why it was tricky to track down.
The problem was that one of the network settings in a disused part of Container Manager added a route to the routing table that prevented correct access to other devices on my local network. Deleting the network in the Container Manager fixed the issue. Once, the DS1522 could correctly contact the local network, it found the UPS/NUT right away.
I do thank you for your help in looking at this. It eventually helped me reach the right answer.
-
@ghound said in NUT Package (2.8.1 and above):
I recently reloaded the driver, so I may have to wait a few days for the problem to show up. In the mean time this post is the problem I thought I was having (even though I have a Tripp Lite not a CyberPower, others reported your devel packaged helped Tripp Lite UPS):
From your Feb 22, 2023 post...
That post is long out of date. The particular issue discussed there was a USB library problem, which was addressed in the October 2023 release.
The recent TrippLite issues introduced in the January 2024 release are particular to the legacy TrippLite protocol (tripplite_usb). No association with HID UPSs (usbhid-ups).
-
@dennypage said in NUT Package (2.8.1 and above):
I don't know about this issue. What do you see in the logs for kernel USB and usbhid-ups?
When the unit disappears, does it still show up when you do
usbconfig -v
?Is there a way to capture when the connection is dropped? The system log is quickly flooded with:
Mar 4 20:06:25 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:06:20 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:06:14 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:06:09 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:06:04 upsmon 54708 UPS TrippLite is unavailable Mar 4 20:06:04 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:05:59 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected Mar 4 20:05:54 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connectedThe output of usbconfig -v is:
ugen0.1: <Generic XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.1.0: uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0300 bDeviceClass = 0x0009 <HUB> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0003 bMaxPacketSize0 = 0x0009 idVendor = 0x0000 idProduct = 0x0000 bcdDevice = 0x0100 iManufacturer = 0x0001 <Generic> iProduct = 0x0002 <XHCI root HUB> iSerialNumber = 0x0000 <no string> bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x001f bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x0040 bMaxPower = 0x0000 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0001 bInterfaceClass = 0x0009 <HUB> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0002 bInterval = 0x00ff bRefresh = 0x0000 bSynchAddress = 0x0000 Additional Descriptor bLength = 0x06 bDescriptorType = 0x30 bDescriptorSubType = 0x00 RAW dump: 0x00 | 0x06, 0x30, 0x00, 0x00, 0x00, 0x00 ugen1.1: <Marvell EHCI root HUB> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) ugen1.1.0: uhub1: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0200 bDeviceClass = 0x0009 <HUB> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0001 bMaxPacketSize0 = 0x0040 idVendor = 0x0000 idProduct = 0x0000 bcdDevice = 0x0100 iManufacturer = 0x0001 <Marvell> iProduct = 0x0002 <EHCI root HUB> iSerialNumber = 0x0000 <no string> bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x0019 bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x0040 bMaxPower = 0x0000 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0001 bInterfaceClass = 0x0009 <HUB> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0008 bInterval = 0x00ff bRefresh = 0x0000 bSynchAddress = 0x0000 ugen1.2: <Tripp Lite ECO850LCD> at usbus1, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0200 bDeviceClass = 0x0000 <Probed by interface class> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x09ae idProduct = 0x3024 bcdDevice = 0x0002 iManufacturer = 0x0003 <retrieving string failed> iProduct = 0x0001 <retrieving string failed> iSerialNumber = 0x0005 <retrieving string failed> bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x0022 bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x00a0 bMaxPower = 0x0032 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0001 bInterfaceClass = 0x0003 <HID device> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Additional Descriptor bLength = 0x09 bDescriptorType = 0x21 bDescriptorSubType = 0x11 RAW dump: 0x00 | 0x09, 0x21, 0x11, 0x01, 0x21, 0x01, 0x22, 0x75, 0x08 | 0x03 Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0040 bInterval = 0x00ff bRefresh = 0x0000 bSynchAddress = 0x0000 -
@ghound said in NUT Package (2.8.1 and above):
Is there a way to capture when the connection is dropped? The system log is quickly flooded with:
Mar 4 20:06:25 upsmon 54708 Poll UPS [TrippLite] failed - Driver not connected
I think I found the place in the system log when the connection was lost:
Mar 3 23:15:06 router01 upsd[53387]: Found 1 UPS defined in ups.conf Mar 3 23:15:07 router01 upsd[53855]: Startup successful Mar 3 23:15:07 router01 usbhid-ups[53729]: using 'battery.charge' to set battery low state Mar 3 23:15:07 router01 usbhid-ups[53729]: using 'battery.runtime' to set battery low state Mar 3 23:15:07 router01 usbhid-ups[54232]: Startup successful Mar 3 23:15:08 router01 upsmon[54510]: Startup successful Mar 3 23:15:08 router01 upsd[53855]: Connected to UPS [TrippLite]: usbhid-ups-TrippLite Mar 3 23:15:08 router01 upsd[53855]: User local-monitor@::1 logged into UPS [TrippLite] ~ ~ ~ Mar 4 14:18:19 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 14:18:19 router01 upsd[53855]: Data for UPS [TrippLite] is stale - check driver Mar 4 14:18:19 router01 upsmon[54708]: Poll UPS [TrippLite] failed - Data stale Mar 4 14:18:19 router01 upsmon[54708]: Communications with UPS TrippLite lost Mar 4 14:18:21 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 14:18:23 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything -
@ghound said in NUT Package (2.8.1 and above):
Is there a way to capture when the connection is dropped? The system log is quickly flooded with…
We only care about things that match “kernel” or “usbhid-ups”…
-
@dennypage said in NUT Package (2.8.1 and above):
We only care about things that match “kernel” or “usbhid-ups”…
Everything was working fine on March 3rd (after restart) but when I checked afternoon of March 4th, Nut reported: Failed to retrieve status.
(Of course there are many "usbhid-ups: Could not open any HID devices" entries between Mar 4 14:18 and Mar 4 15:03.)
Mar 3 23:15:00 router01 kernel: nd6_dad_timer: called with non-tentative address fe80:c::1:1(mvneta0.4092) Mar 3 23:15:07 router01 usbhid-ups[53729]: using 'battery.charge' to set battery low state Mar 3 23:15:07 router01 usbhid-ups[53729]: using 'battery.runtime' to set battery low state Mar 3 23:15:07 router01 usbhid-ups[54232]: Startup successful Mar 4 14:18:19 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 14:18:21 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 14:18:23 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything ~ ~ ~ Mar 4 15:03:26 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 15:03:28 router01 usbhid-ups[54232]: libusb1: Could not open any HID devices: insufficient permissions on everything Mar 4 15:03:28 router01 usbhid-ups[54232]: Failed to init libusb 1.0: Cannot allocate memory Mar 4 15:03:28 router01 kernel: kern.ipc.maxpipekva exceeded; see tuning(7) -
@ghound If I understood you correctly, you had previously said that you had to physically disconnect the USB cable after the UPS disappeared. However the usbconfig output you posted shows that the UPS is still connected. Also, there are not kernel USB messages. Can you clarify please?
When the UPS stops, have you tried restarting the UPS service?
Lastly, can you please post anything you have in the "Extra Arguments to driver" and "Additional configuration lines for ups.conf" sections?
