-
@keyser Maybe try googling the UPS model number and "NUT" or "Network UPS Tools", and/or the error from your logs and see what pops up. It's much more likely that the root cause is in upstream FreeBSD or NUT, like the other two recent issues have been, than in pfSense or this NUT wrapper package. (Assuming the UPS itself isn't being weird, a la CyberPower)
But dennypage is right - there are too many variables to do much troubleshooting without being hands-on with the specific gear.
-
@dennypage i come back after a 2.7 clean install with the same error like before.
upsmon 85035 UPS [APC]: connect failed: Connection failure: Connection refused
I add user=root in ups.conf and also hw.usb.quirk.0="0x051d 0x0003 0x0000 0xffff UQ_HID_IGNORE" on loader.conf.local and it is not working anymore -
@lcbbcl You didn't give detail as to what "not working" means (permission failure, bus error, etc.), but the usual caveats about disconnect/reconnect or reboot following install still apply. The quirk definitely requires a reboot.
I would suggest that you install the updated usbhid-ups and reboot. If that doesn't work, then please post some detail about what you are seeing.
-
@dennypage Done is fixed , i deleted and redo the loader.cond.local
-
Unfortunately this did not work for me. Even worse it seems to have broken it more. I would be able to restart the NUT service and it would work for a while and then crap out, but with the dev usbhid-ups, it won't connect at all.
pfSense+ 23.05.1-RELEASE (amd64)
NUT 2.8.0_2. UPS Type: Local USB. Driver: usbhid
Cyperpower CP1500AVRLCD connected over USB.
Was working fine prior to upgrading from pfsense CE 2.6.0. Already tried reinstalling the NUT package.Tried the below to resolve:
- interruptonly by itself
- user=root by itself
- Use the dev usbhid-ups by itself
- Didn't try any combination of the above together.
Steps taken to use the dev usbhid_ups
- Stopped NUT
- Backed up original usbhid-ups from /usr/local/libexec/nut/
- Removed and uploaded the dev usbhid-ups you provided.
- chmod 755 (the permissions didn't match)
- Ran "logger beforerestart" to have a clear spot in the logs to reference (you'll see that in the log snip below)
- Started NUT. Failed to connect
- Rebooted. Still failed to connect.
- Stopped NUT, removed dev usbhid-ups, and restored original. Fixed permissions
- Started NUT. Successfully connects on startup but disconnects after some time.
Logs from using the dev usbhid-ups
Sep 7 16:01:02 pfbox root[96703]: beforerestart Sep 7 16:01:02 pfbox php-fpm[396]: /nut_settings.php: Configuration Change: admin@10.10.10.123 (Local Database): Updated UPS settings Sep 7 16:01:02 pfbox check_reload_status[435]: Syncing firewall Sep 7 16:01:02 pfbox php-fpm[396]: /nut_settings.php: Stopping service nut Sep 7 16:01:02 pfbox upsmon[21097]: Signal 15: exiting Sep 7 16:01:02 pfbox upsd[22142]: User local-monitor@::1 logged out from UPS [CP1500AVRLCD] Sep 7 16:01:02 pfbox upsd[22142]: mainloop: Interrupted system call Sep 7 16:01:02 pfbox upsd[22142]: Signal 15: exiting Sep 7 16:01:02 pfbox php-fpm[396]: /nut_settings.php: Starting service nut Sep 7 16:01:02 pfbox upsmon[1506]: Startup successful Sep 7 16:01:02 pfbox upsmon[2025]: UPS [CP1500AVRLCD]: connect failed: Connection failure: Connection refused Sep 7 16:01:02 pfbox upsmon[2025]: Communications with UPS CP1500AVRLCD lost Sep 7 16:01:03 pfbox upsd[14606]: listening on ::1 port 3493 Sep 7 16:01:03 pfbox upsd[14606]: listening on 127.0.0.1 port 3493 Sep 7 16:01:03 pfbox upsd[14606]: Can't connect to UPS [CP1500AVRLCD] (usbhid-ups-CP1500AVRLCD): Connection refused Sep 7 16:01:03 pfbox upsd[14784]: Startup successfulOn the original usbhid-ups, it starts up fine, and then later starts to have the same issue. This is with no additional options configured.
Sep 7 16:05:26 pfbox upsmon[86229]: Startup successful Sep 7 16:05:26 pfbox upsmon[86705]: UPS [CP1500AVRLCD]: connect failed: Connection failure: Connection refused Sep 7 16:05:26 pfbox upsmon[86705]: Communications with UPS CP1500AVRLCD lost Sep 7 16:05:26 pfbox usbhid-ups[87908]: Startup successful Sep 7 16:05:27 pfbox upsd[88031]: listening on ::1 port 3493 Sep 7 16:05:27 pfbox upsd[88031]: listening on 127.0.0.1 port 3493 Sep 7 16:05:27 pfbox upsd[88031]: Connected to UPS [CP1500AVRLCD]: usbhid-ups-CP1500AVRLCD Sep 7 16:05:27 pfbox upsd[88080]: Startup successful Sep 7 16:05:31 pfbox upsd[88080]: User local-monitor@::1 logged into UPS [CP1500AVRLCD] Sep 7 16:05:31 pfbox upsmon[86705]: Communications with UPS CP1500AVRLCD established ###Gap in time where it was working### Sep 7 17:52:08 pfbox upsd[88080]: Can't connect to UPS [CP1500AVRLCD] (usbhid-ups-CP1500AVRLCD): Connection refused **Sep 7 17:52:08 pfbox kernel: pid 87908 (usbhid-ups), jid 0, uid 316: exited on signal 10** Sep 7 17:52:13 pfbox upsmon[86705]: Poll UPS [CP1500AVRLCD] failed - Driver not connected Sep 7 17:52:13 pfbox upsmon[86705]: Communications with UPS CP1500AVRLCD lost Sep 7 17:52:18 pfbox upsmon[86705]: Poll UPS [CP1500AVRLCD] failed - Driver not connected Sep 7 17:52:23 pfbox upsmon[86705]: Poll UPS [CP1500AVRLCD] failed - Driver not connectedI tried enabling debug in the additional options for NUT based on the NUT documentation but didn't see any debug messaging being output anywhere.
Any wisdom or guidance for me? My telegram bot is annoying the hell out of me
-
@Nuggets-0 said in NUT package:
kernel: pid 87908 (usbhid-ups), jid 0, uid 316: exited on signal 10
I would suggest double checking the file checksums against the alternate usbhid-ups checksums posted back on March 12th.
-
Thanks for the quick response (and for all that you've done). I did do that when I originally did it but I did it again so I could get you the command output below. I use openhashtab which does it for you in the properties tab and confirms if it's a match if you paste in a value to compare it to. I copied the sha1 and sha256 values you provided and confirmed them when I downloaded and extracted. Then when I copied them to the system, I ran sha1 and sha256 on the file and compared it to the confirmed downloaded copy which also match the values you provided.
/usr/local/libexec/nut: sha1 usbhid-ups; sha256 usbhid-ups
SHA1 (usbhid-ups) = 49ce9131502bfb8b789ee97b7fb3fc81fc9f8fff
SHA256 (usbhid-ups) = 999a2653559dbc50ecc8ba592a67587b1e307a1495f6e8ebbd3d8e90e3967133I'm using the usbhid-ups you provided on that post from
July 7thMarch 12th (the one I replied to). Is that the right one or is there an updated one?EDIT:
Just realized you said there was one on March 12th. I'll try that one and report back.
Not sure where I got July 7th, but the March 12th one is indeed the one I tried. SHA1 and SHA256 above match it and I even have it in my notes that it's the one I tried. Just mixed up a date when typing this post. -
@Nuggets-0 Sorry, I mixed up which was dev and which was original. Looking at your logs, the original version is the one that is core dumping (as you would expect).
The logs with the dev version do not show any output from usbhid-ups. You are on an x86_64 system, yes? If so, can you try starting the executable by hand please?
If you are on a ARM instead of an x86_64, you need to use a different executable. I believe that someone else posted an ARM executable. [Edit: On Jul 7, 2023]
-
@dennypage said in NUT package:
@Nuggets-0 Sorry, I mixed up which was dev and which was original. Looking at your logs, the original version is the one that is core dumping (as you would expect).
The logs with the dev version do not show any output from usbhid-ups. You are on an x86_64 system, yes? If so, can you try starting the executable by hand please?
If you are on a ARM instead of an x86_64, you need to use a different executable. I believe that someone else posted an ARM executable. [Edit: On Jul 7, 2023]
Yes, I'm on x86_64. I tried to perform your instructions with the dev usbhid-ups
I tested it both in the CLI and from the Execute Shell Command in pfSense. They both had the same issue
Command:
/usr/local/libexec/nut/usbhid-ups -a CP1500AVRLCDResponse:
Can't chdir to /var/db/nut: Permission denied Network UPS Tools - Generic HID driver 0.49 (11-eol-49673-g687a1b3d4995) USB communication driver (libusb 1.0) 0.45So in my SSH session, I see that /var/db/nut was set to 750. I set it to 777 (for testing) and ran again. This is the response:
Network UPS Tools - Generic HID driver 0.49 (11-eol-49673-g687a1b3d4995) USB communication driver (libusb 1.0) 0.45 libusb1: Could not open any HID devices: no USB buses found No matching HID UPS foundI ran the usbconfig -v command you recommended before and I see the UPS. If I put in the old usbhid-ups without any reboots, it connects just fine (and eventually cores as you mentioned)
ugen0.2: <CPS CP1500AVRLCDa> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)In all of this I can't figure out where the issue is or if I'm running the command incorrect to launch it manually. I've also moved it to multiple USB ports. There are no logs in the syslog as the usbhid-ups never ran successfully. I also never see the pid file being generated in /var/db/nut/ for usbhid-ups. I set /var/db/nut back to 750 as it was originally until I hear back. The original usbhid-ups is functioning again until it cores so I haven't broken anything yet in my troubleshooting
Any suggestions?
Edit: If you need more output from usbconfig -v, let me know. The post was being flagged as spam with the more verbose information I had tried to put in.
-
-
@dennypage
Didn't want to leave you hanging.So I had accidentally started the regular usbhid-ups with the 777 permissions on /var/db/nut. It ran for over 12 hours without a disconnect. I decided to see if I could reproduce with 750 on /var/db/nut. So far it's been running for 5 hours without disconnect. Most of the time it would only last 2-3 hour at max before a disconnect. I didn't do the chown because nut was already owner for both user and group.
I looked at the /var/db/nut/ files beforehand and after changing the permissions to see if a file wasn't getting updated, but all of them were listing pids correctly. The only file I couldn't check the contents of was the socket file. But all files were removed and created with the start and stop of the service with both 750 and 777 on /var/db/nut. So I have no idea if this fixed it or what. I'll come back if I run into any issues.
Thanks again for all your assistant and quick responses.
-
@Nuggets-0 The permissions will be a problem again when you reboot. It's only my list to fix as soon as I can.
-
@dennypage said in NUT package:
Do
chown nut /var/db/nutand restart the service.
So naturally, my previous steps didn't fix it.
Running chown as requested didn't fix it as /var/db/nut was already owned by nut.
However, with the dev usbhid-ups, I set user=nut in ups.conf and it came up. For the record, user=root worked too but I wanted to get it working with the nut user instead of root. Not sure why user=nut works. The daemon is supposed to start as the user nut and nut has all the right properties (owner and 750). But at least it's working on the dev usbhid-ups now.
I'll monitor for a bit and come back.
-
@Nuggets-0 it's a local daemon just run it as root why complicate things
-
@incith said in NUT package:
@Nuggets-0 it's a local daemon just run it as root why complicate things
Both user=root and user=nut worked both the same. I merely tested using user=nut on a whim and it worked.
I have my upsd hosting the NUT server so it's available to other servers running on the same UPS. I want pfsense to be the last server running should everything else need to go down, so it makes sense to host it on pfsense. I have rules in place as well as a user/password to prevent from unfettered access. So having it run as root is a potential risk to me, not so much others.
What I don't understand is WHY it works when using user=nut. usbhid-ups is supposed to start as the nut user anyways. Here usbhid-ups running on the original file:
79502 nut 1 20 0 13M 3420K select 0 0:00 0.01% usbhid-upsUsing user=nut The nut user is the owner and has full access to associated files. user=nut is telling the nut user to act like itself. Of course, this could all be my confusion with what is getting started with nut vs root.
Edit: Forgot to mention, it had been running fine under user=nut for at least 12 hours. Telegram bots had an outage yesterday so it did some funky stuff in pfsense with my notifications, so I restarted the NUT service so it would initiate monitoring again. Will update probably tomorrow to give it more time to run into any issues.
-
@Nuggets-0 The old version wanted to be "uucp" and the new version wants to be "nut", so now there is a disagreement between the NUT executable and the pfSense NUT package.
-
Hello, I recently re-activated the SMTP notifications after 1 year of using only telegram (because of https://redmine.pfsense.org/issues/14031), but I no longer receive NUT notifications on email, only telegram.
On nut settings, the Enable notifications is checked.
I'm running 2.8.0_2.Am I missing something?
-
@Samlink Unfortunately, the fix for the Redmine issue you referred to broke the smtp notification mechanism for non root processes. I am hoping this gets fixed in the next release set.
In the interim, you should be able to work around the issue by adding
RUN_AS_USER rootin the "Additional configuration lines for upsmon.conf" section.
-
So here's another data point:
Finally got around to buying a new UPS for my pfsense box (A 4 core i5-6500 Dell Optiplex)
pfsense 23.05.1-RELEASE and nut 2.8.0_2
CyberPower CST135UC2 $120 from Costco - I really should have known better, but I'd used CPS UPS' with pfsense NUT before with no issues. Sigh.After much futzing and the dreaded CPS disconnect yo-yo and much searching I finally found this thread.
So I now have a stable UPS, yay.
All it took was adding the "user=root" to ups.conf
AND
"interruptonly" to driver arguments.I see far fewer parameters, but I can live with the workarounds. For grins and giggles they are:
Variable Value battery.charge 100 battery.runtime 16525 battery.runtime.low 300 device.mfr CPS device.model CST135UC2 device.serial CDZNP7001732 device.type ups driver.flag.interruptonly enabled driver.name usbhid-ups driver.parameter.pollfreq 30 driver.parameter.pollinterval 2 driver.parameter.port auto driver.parameter.synchronous auto driver.version 2.8.0 driver.version.data CyberPower HID 0.6 driver.version.internal 0.47 driver.version.usb libusb-1.0.0 (API: 0x1000102) ups.beeper.status disabled ups.mfr CPS ups.model CST135UC2 ups.productid 0601 ups.serial CDZNP7001732 ups.status OL ups.vendorid 0764and here's the "usbconfig -v -d ugen0.5 show_ifdrv" output:
ugen0.5: <CPS CST135UC2> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0200 bDeviceClass = 0x0000 <Probed by interface class> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x0764 idProduct = 0x0601 bcdDevice = 0x0200 iManufacturer = 0x0003 <retrieving string failed> iProduct = 0x0001 <CST135UC2> iSerialNumber = 0x0002 <CDZNP7001732> bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x0029 bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x00c0 bMaxPower = 0x0032 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0002 bInterfaceClass = 0x0003 <HID device> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Additional Descriptor bLength = 0x09 bDescriptorType = 0x21 bDescriptorSubType = 0x11 RAW dump: 0x00 | 0x09, 0x21, 0x11, 0x01, 0x21, 0x01, 0x22, 0xd6, 0x08 | 0x02 Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0040 bInterval = 0x0020 bRefresh = 0x0000 bSynchAddress = 0x0000 Endpoint 1 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0002 <OUT> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0040 bInterval = 0x0020 bRefresh = 0x0000 bSynchAddress = 0x0000The really weird part is the id wasn't recognized on some of the ports, which considering they're all ugen0 was a little freaky.
So that's what I get for being lazy and cheap. Sigh again.
Now I just have to remember to remove everything when the nut package eventually gets updated.
Thank you for all the hard work here, and I volunteer as guinea pig for testing patched code if needed. Since the Optiplex is Intel i5, 4 cores, 8GB of RAM and about half a TB of SSD free it's not exactly a constrained test platform. Happy to set up a user for ssh or VPN remote access during test periods My home fiber is 1G/1G uncapped so slinging large files around isn't much of an issue either.
-
@daplumber I would just like to add, that while there are “bad” UPS’s out there, I think the BSD crowd should take a long breath and have a look in its own backyard. I have suffered the disconnect issues on two different pfSenses (x64 and Arm) with two different UPS’s (Eaton and CyperPower), and the only stable solution was to connect the UPSes to my Windows Server or my Linux Raspberry Pi. None of those platforms suffered the disconnect issue and both UPS’s were perfectly stable there.
So I would still argue your issue is just as likely with the USB implementation in BSD.
