-
@lcbbcl I need the messages from the usbhid-ups process. The messages from upsd are expected when usbhid-ups isn't running.
If you don't have any log entries from usbhid-ups, then please post your ups settings (Services / UPS / Settings)
I'll be away for a bit... but will look at it when I'm back.
-
@dennypage
I have no msg/error from usbhid-ups
-
@lcbbcl If you re-save your ups settings, it will restart the processes and you should be able to find usbhid entries.
-
16:07 removed interruptonly and saved settings
On console:
[23.01-RELEASE][X@Y]/root: sysctl kern.corefile=/var/tmp/%N.core
kern.corefile: /root/%N.core -> /var/tmp/%N.coreBroadcast Message from root@gw.fubr.org
(no tty) at 16:07 EST...Communications with UPS CyberPower lost
Broadcast Message from root@gw.fubr.org
(no tty) at 16:07 EST...Communications with UPS CyberPower established
Feb 20 16:07:40 Y php-fpm[83664]: /nut_settings.php: Stopping service nut
Feb 20 16:07:40 Y upsmon[66045]: Signal 15: exiting
Feb 20 16:07:40 Y upsd[97704]: User local-monitor@::1 logged out from UPS [CyberPower]
Feb 20 16:07:40 Y upsd[97704]: mainloop: Interrupted system call
Feb 20 16:07:40 Y upsd[97704]: Signal 15: exiting
Feb 20 16:07:40 Y usbhid-ups[82519]: Signal 15: exiting
Feb 20 16:07:40 Y php-fpm[83664]: /nut_settings.php: Starting service nut
Feb 20 16:07:40 Y upsmon[7684]: Startup successful
Feb 20 16:07:40 Y upsmon[7684]: UPS [CyberPower]: connect failed: Connection failure: Connection refused
Feb 20 16:07:40 Y upsmon[7684]: Communications with UPS CyberPower lost
Feb 20 16:07:40 Y usbhid-ups[8439]: Startup successful
Feb 20 16:07:41 Y upsd[8857]: listening on 192.168.11.1 port 3493
Feb 20 16:07:41 Y upsd[8857]: listening on ::1 port 3493
Feb 20 16:07:41 Y upsd[8857]: listening on 127.0.0.1 port 3493
Feb 20 16:07:41 Y upsd[8857]: Connected to UPS [CyberPower]: usbhid-ups-CyberPower
Feb 20 16:07:41 Y upsd[8857]: Startup successful
Feb 20 16:07:45 Y upsd[8857]: User local-monitor@::1 logged into UPS [CyberPower]
Feb 20 16:07:45 Y upsmon[7684]: Communications with UPS CyberPower establishedFeb 20 17:57:45 Y upsd[8857]: Can't connect to UPS [CyberPower] (usbhid-ups-CyberPower): Connection refused
Feb 20 17:57:45 Y kernel: pid 8439 (usbhid-ups), jid 0, uid 66: exited on signal 10 (core dumped)
Feb 20 17:57:48 Y upsmon[7684]: Poll UPS [CyberPower] failed - Driver not connected
Feb 20 17:57:48 Y upsmon[7684]: Communications with UPS CyberPower lost
Feb 20 17:57:53 Y upsmon[7684]: Poll UPS [CyberPower] failed - Driver not connected
Feb 20 17:57:58 Y upsmon[7684]: Poll UPS [CyberPower] failed - Driver not connectedAlso set about 10 minutes later: sysctl kern.sugid_coredump=1
I have a core dump in /var/tmp/usbhid-ups.core
What do you want me to do with it?
-
@shaffergr said in NUT package:
I have a core dump in /var/tmp/usbhid-ups.core
What do you want me to do with it?Perfect! Compress and email it to me. Thanks!
-
email address?
-
@shaffergr Sent in chat. Look for a quote bubble next to the pic of your icon at the top. Should have an alert there.
-
sent via email. Let me know if you don't get it.
-
@lcbbcl I believe I have found what is causing the issue you are experiencing. To correct the issue will require an update to the pfSense nut package as well as updating the underlying FreeBSD nut package used by pfSense. It will take some time.
In the interim, you can address the issue by adding the line "user=root" to "Additional configuration lines for ups.conf" under Advanced settings.
-
@dennypage
Thank you, now is working -
@dennypage said in NUT package:
@lcbbcl I believe I have found what is causing the issue you are experiencing. To correct the issue will require an update to the pfSense nut package as well as updating the underlying FreeBSD nut package used by pfSense. It will take some time.
In the interim, you can address the issue by adding the line "user=root" to "Additional configuration lines for ups.conf" under Advanced settings.
Your hard and dedicated work on this package is an inspiration. Thank you for the effort :-)
I assume it was an access permissions issue since running it as root solves the issue? Can you comment on why it only happens for some UPS models?
-
@keyser said in NUT package:
I assume it was an access permissions issue since running it as root solves the issue? Can you comment on why it only happens for some UPS models?
Yes, it's a permissions issue. I am hoping it can be addressed in the FreeBSD nut package (or perhaps in kernel config?), but if not I will have to add the user=root to the config for all usb devices.
What models it happens on are dependent upon the quirks defined for that device. If you want to jump down the rabbit hole, run "usbconfig dump_quirk_names" and "usbconfig dump_device_quirks".
NB: This is not related to the signal 10/11 issue being experienced with CyberPower units.
-
I see the github packages have been moved to
https://github.com/pfsense/FreeBSD-ports
But i don't see the nut package in there.
Where is the code for nut, or is it closed source now?
In the upsmon configuration, can we please add two config boxes for the below files.
/usr/local/etc/nut/upssched.conf with permission 644
/usr/local/bin/upssched-cmd with permission 755To use the scheduler to trigger events from upsmon. these files need to be configurable.
Also, there should be a note for using them, the notification box has to be disabled.
here is a sample of working filesupsmon
# upsmon.conf - Configuration for network UPS Tools upsmon # set the path to the notify command processor !!! the UPS notifications option must be unchecked !!! NOTIFYCMD /usr/local/sbin/upssched # Poll the UPS every 60 seconds. POLLFREQ 60 # Wait this many seconds for "Notify Shutdown" messages from slaves POLLFREQALERT 30 # time UPS may go missing before being declared dead DEADTIME 600 # Warn every 12 hours if battery needs to be replaced RBWARNTIME 43200 # Warn every 5 minutes, if UPS is unreachable NOCOMMWARNTIME 300 # Messages for the events %s will be replaced with the UPS system identifier in question NOTIFYMSG ONLINE "UPS %s on line power" NOTIFYMSG ONBATT "UPS %s on battery" NOTIFYMSG LOWBATT "UPS %s battery is low" NOTIFYMSG FSD "UPS %s forced shutdown in progress" NOTIFYMSG COMMOK "UPS %s communications (re-)established" NOTIFYMSG COMMBAD "UPS %s communications lost" NOTIFYMSG SHUTDOWN "Auto logout and shutdown proceeding" NOTIFYMSG REPLBATT "UPS %s battery needs to be replaced" NOTIFYMSG NOCOMM "UPS %s is not reachable" NOTIFYMSG NOPARENT "upsmon parent process died, shutdown impossible" # Change behavior of upsmon on certain events. # Possible values for the flags: # SYSLOG - Write the message in the syslog # WALL - Write the message to all users on the system # EXEC - Execute NOTIFYCMD (see above, in our case see upssched.conf) with the message # IGNORE - Don't do anything # If you use IGNORE, don't use any other flags on the same line. # NOTIFYFLAG <notify type> <flag>[+<flag>][+<flag>] ... NOTIFYFLAG ONLINE SYSLOG NOTIFYFLAG ONBATT SYSLOG NOTIFYFLAG LOWBATT SYSLOG NOTIFYFLAG FSD SYSLOG+WALL+EXEC NOTIFYFLAG COMMOK SYSLOG+EXEC NOTIFYFLAG COMMBAD SYSLOG+WALL+EXEC NOTIFYFLAG SHUTDOWN SYSLOG NOTIFYFLAG REPLBATT SYSLOG NOTIFYFLAG NOCOMM SYSLOG NOTIFYFLAG NOPARENT SYSLOGupssched
# Network UPS Tools - upssched.conf file #Command script to run CMDSCRIPT /usr/local/bin/upssched-cmd # command for pipe and lock files PIPEFN /var/db/nut/upssched.pipe LOCKFN /var/db/nut/upssched.lock # ============================================================================ # AT <notifytype> <upsname> <command> # Define a handler for a specific event <notifytype> on UPS <upsname>. # <upsname> can be the special value * to apply this handler to every # possible value of <upsname>. # Run the command <command> via your CMDSCRIPT when it happens. # Note that any AT that matches both the <notifytype> and the <upsname> # for the current event will be used. # ============================================================================ # Possible AT commands # - START-TIMER <timername> <interval> # Start a timer called <timername> that will trigger after <interval> # seconds, calling your CMDSCRIPT with <timername> as the first # argument. # Example: # Start a timer that'll execute when any UPS (*) has been gone 10 seconds # AT COMMBAD * START-TIMER COMMBAD 1800 # ----------------------------------------------------------------------- # - CANCEL-TIMER <timername> [cmd] # Cancel a running timer called <timername>, if possible. If the timer # has passed then pass the optional argument <cmd> to CMDSCRIPT. # Example: # If a specific UPS (myups@localhost) comes back online, then stop the # timer before it triggers # AT COMMOK * CANCEL-TIMER COMMBAD COMMOK # ----------------------------------------------------------------------- # - EXECUTE <command> # Immediately pass <command> as an argument to CMDSCRIPT. # # Example: # If any UPS (*) reverts to utility power, then execute # 'ups-back-on-line' via CMDSCRIPT. # UPS is online in syslog. we don't need action when no UPS connected # AT ONLINE * CANCEL-TIMER onbatt # UPS on battery in syslog. badcomms will shutdown for us # AT ONBATT * START-TIMER onbatt 300 # UPS low battery is syslog. we should badcomms to shutdown # AT LOWBATT * START-TIMER lowbatt 300 # UPS forced shutdown by master or upsmon AT FSD * EXECUTE powerdown # UPS communications restored AT COMMOK * CANCEL-TIMER badcomms # UPS communications failed. AT COMMBAD * START-TIMER badcomms 600 # UPS shutdown recorded in syslog # AT SHUTDOWN * EXECUTE powerdown # UPS needs replacement battery. Info in syslog # AT REPLBATT * EXECUTE replbatt # UPS is not reachable. Relying on deadtime for shutdown AT NOCOMM * EXECUTE commbad # UPSMON parent process has died - shutdown not possible. # AT NOPARENT * EXECUTE noparentupssched-cmd
#! /bin/sh # # This script should be called by upssched via the CMDSCRIPT directive. # # Here is a quick example to show how to handle a bunch of possible # timer names with the help of the case structure. # # This script may be replaced with another program without harm. # # The first argument passed to your CMDSCRIPT is the name of the timer # from your AT lines. # # case $1 in # upsgone) # logger -t upssched-cmd "The UPS has been gone for awhile" # ;; # *) # logger -t upssched-cmd "Unrecognized command: $1" # ;; # esac case $1 in onbatt) # function not needed as we only use syslog logger -t upssched-cmd "UPS is running on battery." ;; online) # function not needed as we only use syslog logger -t upssched-cmd "UPS back online." ;; lowbatt) # function not needed as we only use syslog logger -t upssched-cmd "UPS has low battery" # we rely on badcomms when no UPS connected # /usr/local/sbin/upsmon -c fsd ;; badcomms) logger -t upssched-cmd "UPS offline too long, forced shutdown." /usr/local/sbin/upsmon -c fsd ;; commok) # function not needed as we only use syslog logger -t upssched-cmd "UPS communications restored." ;; powerdown) logger -t upssched-cmd "UPS forced shutdown started" /usr/local/sbin/upsmon -c fsd ;; replbatt) # function not needed as we only use syslog logger -t upssched-cmd "UPS battery needs replacing" ;; noparent) # function not needed as we only use syslog logger -t upssched-cmd "upsmon parent process died. Shutdown not possible" ;; *) logger -t upssched-cmd "No code defined for command: $1" ;; esac -
@gwaitsi It hasn't moved since I wrote it. sysutils/pfSense-pkg-nut.
-
This solved my issue. Everything is working great. Thank you :)
@dennypage said in NUT package:
@lcbbcl I believe I have found what is causing the issue you are experiencing. To correct the issue will require an update to the pfSense nut package as well as updating the underlying FreeBSD nut package used by pfSense. It will take some time.
In the interim, you can address the issue by adding the line "user=root" to "Additional configuration lines for ups.conf" under Advanced settings.
-
-
@dennypage
Here it isusbconfig -v ugen0.1: <Intel XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.1.0: uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0300 bDeviceClass = 0x0009 <HUB> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0003 bMaxPacketSize0 = 0x0009 idVendor = 0x0000 idProduct = 0x0000 bcdDevice = 0x0100 iManufacturer = 0x0001 <Intel> iProduct = 0x0002 <XHCI root HUB> iSerialNumber = 0x0000 <no string> bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x001f bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x0040 bMaxPower = 0x0000 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0001 bInterfaceClass = 0x0009 <HUB> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0002 bInterval = 0x00ff bRefresh = 0x0000 bSynchAddress = 0x0000 Additional Descriptor bLength = 0x06 bDescriptorType = 0x30 bDescriptorSubType = 0x00 RAW dump: 0x00 | 0x06, 0x30, 0x00, 0x00, 0x00, 0x00 ugen0.2: <American Power Conversion Smart-UPS C 1000 FW:UPS 12.0 / ID1005> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0200 bDeviceClass = 0x0000 <Probed by interface class> bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = 0x051d idProduct = 0x0003 bcdDevice = 0x0106 iManufacturer = 0x0001 <American Power Conversion > iProduct = 0x0002 <Smart-UPS C 1000 FW:UPS 12.0 / ID=1005> iSerialNumber = 0x0003 <AS1506222288 > bNumConfigurations = 0x0001 Configuration index 0 bLength = 0x0009 bDescriptorType = 0x0002 wTotalLength = 0x0029 bNumInterfaces = 0x0001 bConfigurationValue = 0x0001 iConfiguration = 0x0000 <no string> bmAttributes = 0x00e0 bMaxPower = 0x0001 Interface 0 bLength = 0x0009 bDescriptorType = 0x0004 bInterfaceNumber = 0x0000 bAlternateSetting = 0x0000 bNumEndpoints = 0x0002 bInterfaceClass = 0x0003 <HID device> bInterfaceSubClass = 0x0000 bInterfaceProtocol = 0x0000 iInterface = 0x0000 <no string> Additional Descriptor bLength = 0x09 bDescriptorType = 0x21 bDescriptorSubType = 0x00 RAW dump: 0x00 | 0x09, 0x21, 0x00, 0x01, 0x21, 0x01, 0x22, 0x03, 0x08 | 0x02 Endpoint 0 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0081 <IN> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0040 bInterval = 0x0014 bRefresh = 0x0000 bSynchAddress = 0x0000 Endpoint 1 bLength = 0x0007 bDescriptorType = 0x0005 bEndpointAddress = 0x0001 <OUT> bmAttributes = 0x0003 <INTERRUPT> wMaxPacketSize = 0x0040 bInterval = 0x000a bRefresh = 0x0000 bSynchAddress = 0x0000 -
For those of you affected by the APC issue (and those that will be in the future):
First, I believe that the issue only affects recent APC UPS designs. Those in the last couple of years or so. The core issue is that a new device identification number has been introduced by APC, which FreeBSD is not aware of. Since the kernel does not recognize the device as a UPS that should be ignored, the kernel is attaching an internal driver to it. The presence of this internal driver on the device precludes any non root user from attaching to the device.
This cannot be reasonably fixed in the pfSense nut package or the underlying FreeBSD nut package. The permanent solution requires an adjustment to the usb quirk table in the kernel itself. A patch has been established for the kernel, but I don't know how long it will take to flow through.
In the interim, there are two solutions available:
The first solution, as noted previously, is to add the line
user=rootto the "Additional configuration lines for ups.conf" section under Advanced settings. This will cause the usbhid-ups command to be run as root. The disadvantage of this approach is that you are running the driver as root, which may be of concern to some.
The alternative solution is to add the line
hw.usb.quirk.0="0x051d 0x0003 0x0000 0xffff UQ_HID_IGNORE"to file "/boot/loader.conf.local". Note that you may have to create this file. The disadvantage of this approach is that you will have to reboot pfSense before it will work.
If you previously implemented the user=root approach, you can remove the user=root line following the reboot. -
For those of you affected by the CyberPower issue (and those that will be in the future):
First, I want to thank all of you for your help in tracking this down. Particularly @shaffergr, who was kind enough, and trusting enough, to run test builds for me against his CyberPower.
I tracked this down to a double free by the usb code in nut. FWIW, I don’t believe that this issue is completely limited to CyberPower UPSs, but it is probably pretty difficult to encounter with other units as hitting the issue requires a reconnect of the UPS on the bus. CyberPower units are well known to randomly disconnect and reconnect seemingly at random.
For those wondering why this issue did not happen in prior to 23.01: 22.05 and below used nut version 2.7.4. In 2.7.4, nut did not actually close the device when a disconnect happened. Presumably, this may have resulted in a memory leak at shutdown, but I didn’t explore enough to confirm. In 2.8.0, when a device disconnects nut actually closes the device when it disconnects. In fact, it closes it twice on all systems other than Linux. There is even a comment in the code noting how the double close would cause corruption on Linux systems. Unfortunately contrary to the code comments it also causes corruption in FreeBSD.
Having traced this down in the 2.8.0 release code, when I went to the current development version I surprised to find that someone had beat me to it, and fixed the issue back in August. It's just not been release yet. :)
To resolve this will require a new version of nut. Either a new release from the nut team, or for the pfSense team to move from the 2.8.0 release version to the current development version of nut. The nut team is looking toward 2.8.1, but it appears that they have a few things they still want to address before putting that out. I will explore the concept of moving to nut-devel with the pfSense team as time permits (I think they are pretty busy right now). But no matter which way it goes, it’s going to take some time.
In the interim, the only known work-around using the release code (discovered by @tman222) is to add the line
interruptonlyto the Extra Arguments to driver section. This will cut down some of the information you can see about your UPS, but the important stuff needed to monitor and shutdown should still be there.
Alternatively, if brave souls are interested, I have a build of usbhid-ups made from the FreeBSD nut-devel package. I do not have a CyberPower available so I haven't been able to directly test it, but I expect that it will work.
If you you decide you would like a copy, reach out to me and I will see about getting it to you. For reference, the shasum and sha256sum checksums are:
49ce9131502bfb8b789ee97b7fb3fc81fc9f8fff usbhid-ups 999a2653559dbc50ecc8ba592a67587b1e307a1495f6e8ebbd3d8e90e3967133 usbhid-ups -
I'm going to sleep now...
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.