Netgate 1100 with 24.03-RC and APC Smart-UPS 1000

pfsjap · Apr 14, 2024, 1:07 PM

Netgate 1100 with 24.03-RC and nut 2.8.2_2 does not connect to the APC Smart-UPS 1000:

Apr 13 19:36:47	upsmon	49711	UPS qnapups is unavailable
Apr 13 19:36:47	upsmon	49711	Poll UPS [qnapups] failed - Driver not connected
Apr 13 19:36:42	upsmon	49711	Communications with UPS qnapups lost
Apr 13 19:36:42	upsmon	49711	Poll UPS [qnapups] failed - Driver not connected
Apr 13 19:36:42	upsd	49283	User local-monitor@127.0.0.1 logged into UPS [qnapups]
Apr 13 19:36:42	upsmon	49673	Startup successful
Apr 13 19:36:41	upsd	49283	Startup successful
Apr 13 19:36:41	upsd	49042	Found 1 UPS defined in ups.conf
Apr 13 19:36:41	upsd	49042	Can't connect to UPS [qnapups] (usbhid-ups-qnapups): No such file or directory

[24.03-RC][admin@pfSense1100.localdomain]/root: usbconfig -v
ugen1.1: <Marvell EHCI root HUB> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen1.1.0: uhub0: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1>

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0200
  bDeviceClass = 0x0009  <HUB>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0001
  bMaxPacketSize0 = 0x0040
  idVendor = 0x0000
  idProduct = 0x0000
  bcdDevice = 0x0100
  iManufacturer = 0x0001  <Marvell>
  iProduct = 0x0002  <EHCI root HUB>
  iSerialNumber = 0x0000  <no string>
  bNumConfigurations = 0x0001


 Configuration index 0

    bLength = 0x0009
    bDescriptorType = 0x0002
    wTotalLength = 0x0019
    bNumInterfaces = 0x0001
    bConfigurationValue = 0x0001
    iConfiguration = 0x0000  <no string>
    bmAttributes = 0x0040
    bMaxPower = 0x0000

    Interface 0
      bLength = 0x0009
      bDescriptorType = 0x0004
      bInterfaceNumber = 0x0000
      bAlternateSetting = 0x0000
      bNumEndpoints = 0x0001
      bInterfaceClass = 0x0009  <HUB>
      bInterfaceSubClass = 0x0000
      bInterfaceProtocol = 0x0000
      iInterface = 0x0000  <no string>

     Endpoint 0
        bLength = 0x0007
        bDescriptorType = 0x0005
        bEndpointAddress = 0x0081  <IN>
        bmAttributes = 0x0003  <INTERRUPT>
        wMaxPacketSize = 0x0008
        bInterval = 0x00ff
        bRefresh = 0x0000
        bSynchAddress = 0x0000



ugen0.1: <Generic XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.1.0: uhub1: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1>

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0300
  bDeviceClass = 0x0009  <HUB>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0003
  bMaxPacketSize0 = 0x0009
  idVendor = 0x0000
  idProduct = 0x0000
  bcdDevice = 0x0100
  iManufacturer = 0x0001  <Generic>
  iProduct = 0x0002  <XHCI root HUB>
  iSerialNumber = 0x0000  <no string>
  bNumConfigurations = 0x0001


 Configuration index 0

    bLength = 0x0009
    bDescriptorType = 0x0002
    wTotalLength = 0x001f
    bNumInterfaces = 0x0001
    bConfigurationValue = 0x0001
    iConfiguration = 0x0000  <no string>
    bmAttributes = 0x0040
    bMaxPower = 0x0000

    Interface 0
      bLength = 0x0009
      bDescriptorType = 0x0004
      bInterfaceNumber = 0x0000
      bAlternateSetting = 0x0000
      bNumEndpoints = 0x0001
      bInterfaceClass = 0x0009  <HUB>
      bInterfaceSubClass = 0x0000
      bInterfaceProtocol = 0x0000
      iInterface = 0x0000  <no string>

     Endpoint 0
        bLength = 0x0007
        bDescriptorType = 0x0005
        bEndpointAddress = 0x0081  <IN>
        bmAttributes = 0x0003  <INTERRUPT>
        wMaxPacketSize = 0x0002
        bInterval = 0x00ff
        bRefresh = 0x0000
        bSynchAddress = 0x0000

      Additional Descriptor

      bLength = 0x06
      bDescriptorType = 0x30
      bDescriptorSubType = 0x00
       RAW dump:
       0x00 | 0x06, 0x30, 0x00, 0x00, 0x00, 0x00




ugen0.2: <American Power Conversion Smart-UPS 1000 FW:UPS 09.3 / ID18> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0200
  bDeviceClass = 0x0000  <Probed by interface class>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0000
  bMaxPacketSize0 = 0x0040
  idVendor = 0x051d
  idProduct = 0x0003
  bcdDevice = 0x0106
  iManufacturer = 0x0001  <American Power Conversion >
  iProduct = 0x0002  <Smart-UPS 1000 FW:UPS 09.3 / ID=18>
  iSerialNumber = 0x0003  <AS2203240601  >
  bNumConfigurations = 0x0001


 Configuration index 0

    bLength = 0x0009
    bDescriptorType = 0x0002
    wTotalLength = 0x0029
    bNumInterfaces = 0x0001
    bConfigurationValue = 0x0001
    iConfiguration = 0x0000  <no string>
    bmAttributes = 0x00e0
    bMaxPower = 0x0001

    Interface 0
      bLength = 0x0009
      bDescriptorType = 0x0004
      bInterfaceNumber = 0x0000
      bAlternateSetting = 0x0000
      bNumEndpoints = 0x0002
      bInterfaceClass = 0x0003  <HID device>
      bInterfaceSubClass = 0x0000
      bInterfaceProtocol = 0x0000
      iInterface = 0x0000  <no string>

      Additional Descriptor

      bLength = 0x09
      bDescriptorType = 0x21
      bDescriptorSubType = 0x00
       RAW dump:
       0x00 | 0x09, 0x21, 0x00, 0x01, 0x21, 0x01, 0x22, 0x03,
       0x08 | 0x02

     Endpoint 0
        bLength = 0x0007
        bDescriptorType = 0x0005
        bEndpointAddress = 0x0081  <IN>
        bmAttributes = 0x0003  <INTERRUPT>
        wMaxPacketSize = 0x0040
        bInterval = 0x0014
        bRefresh = 0x0000
        bSynchAddress = 0x0000

     Endpoint 1
        bLength = 0x0007
        bDescriptorType = 0x0005
        bEndpointAddress = 0x0001  <OUT>
        bmAttributes = 0x0003  <INTERRUPT>
        wMaxPacketSize = 0x0040
        bInterval = 0x000a
        bRefresh = 0x0000
        bSynchAddress = 0x0000

How can I fix this?

dennypage · Apr 14, 2024, 3:36 PM

@pfsjap There are currently no changes to nut for 24.03-RC from 23.09.1. Yes, the version number of the pfSense package changed, but this was related to icons in the UI.

You didn't post anything your config. Please do.

Please see 3 posts above regarding own ownership of /var/db/nut.

pfsjap · Apr 14, 2024, 4:47 PM

@dennypage

[24.03-RC][admin@pfSense1100.localdomain]/root: ls -l /var/db/nut
total 8
-rw-r--r--  1 root nut 6 Apr 14 19:44 upsd.pid
-rw-r--r--  1 nut  nut 6 Apr 14 19:44 upsmon.pid
[24.03-RC][admin@pfSense1100.localdomain]/root: ls -l /usr/local/etc/nut
total 139
-rw-r--r--  1 root wheel 15136 Apr  8 11:05 cmdvartab
-rw-r--r--  1 root wheel 98236 Apr  8 11:05 driver.list
-rw-r--r--  1 root wheel  1071 Apr  8 11:05 hosts.conf
-rw-r--r--  1 root wheel  1071 Apr  8 11:05 hosts.conf.sample
-rw-r--r--  1 root wheel  4774 Apr  8 11:05 nut.conf
-rw-r--r--  1 root wheel  4774 Apr  8 11:05 nut.conf.sample
-rw-r-----  1 root wheel    38 Apr 14 19:44 ups.conf
-rw-r--r--  1 root wheel  9845 Apr  8 11:05 ups.conf.sample
-rw-r-----  1 root wheel    48 Apr 14 19:44 upsd.conf
-rw-r--r--  1 root wheel  9199 Apr  8 11:05 upsd.conf.sample
-rw-r-----  1 root wheel   169 Apr 14 19:44 upsd.users
-rw-r--r--  1 root wheel  2349 Apr  8 11:05 upsd.users.sample
-rw-r-----  1 root wheel   534 Apr 14 19:44 upsmon.conf
-rw-r--r--  1 root wheel 24735 Apr  8 11:05 upsmon.conf.sample
-rw-r--r--  1 root wheel  4217 Apr  8 11:05 upssched.conf
-rw-r--r--  1 root wheel  4217 Apr  8 11:05 upssched.conf.sample
-rw-r--r--  1 root wheel  1871 Apr  8 11:05 upsset.conf
-rw-r--r--  1 root wheel  1871 Apr  8 11:05 upsset.conf.sample
-rw-r--r--  1 root wheel  6861 Apr  8 11:05 upsstats-single.html
-rw-r--r--  1 root wheel  6861 Apr  8 11:05 upsstats-single.html.sample
-rw-r--r--  1 root wheel  4096 Apr  8 11:05 upsstats.html
-rw-r--r--  1 root wheel  4096 Apr  8 11:05 upsstats.html.sample
[24.03-RC][admin@pfSense1100.localdomain]/root:

dennypage · Apr 14, 2024, 5:03 PM

@pfsjap You missed the '-a' option for the ls of /var/db/nut. Please double check and make sure the directory is owned by nut.

Your configuration looks okay.

Can you run the driver by hand in debug mode via ssh please?

First start the service, and then kill the driver:

killall usbhid-ups

Then, run the driver with debug:

/usr/local/libexec/nut/usbhid-ups -DDD -a qnapups

See what error it shows. If it doesn't show enough you can add more 'D's until it does up to "-DDDDDD".

Post the output with the error. This may be lengthy, so it's okay to eliminate duplicated lines in the middle.

Just FYI, I will be going offline for about 48 hours shortly and will look at your output when I return.

pfsjap · Apr 17, 2024, 1:56 AM

@dennypage /var/db/nut is owned by nut.

The driver starts, but then stops:

[24.03-RC][admin@pfSense1100.localdomain]/root: /usr/local/libexec/nut/usbhid-ups -DDDDDD -a qnapups
Network UPS Tools - Generic HID driver 0.52 (2.8.1.1)
USB communication driver (libusb 1.0) 0.46
   0.000001     [D5] do_upsconf_args: confupsname=qnapups, var=driver, val=usbhid-ups
   0.000088     [D5] do_upsconf_args: call main_arg()
   0.000102     [D3] main_arg: var='driver' val='usbhid-ups'
   0.000118     [D5] do_upsconf_args: not a main_arg()
   0.000131     [D5] do_upsconf_args: this is a 'driver' setting, may we proceed?
   0.000145     [D6] testval_reloadable: var=driver, oldval=usbhid-ups, newval=usbhid-ups, reloadable=0, reload_flag=0
   0.000161     [D6] testval_reloadable: verdict for (re)loading var=driver value: -1
   0.000173     [D5] do_upsconf_args: 'driver' setting already applied with this value
   0.000193     [D5] do_upsconf_args: confupsname=qnapups, var=port, val=auto
   0.000204     [D5] do_upsconf_args: call main_arg()
   0.000215     [D3] main_arg: var='port' val='auto'
   0.000229     [D6] testinfo_reloadable: var=port, infoname=driver.parameter.port, newval=auto, reloadable=0, reload_flag=0
   0.000241     [D6] testinfo_reloadable: verdict for (re)loading var=port value: 1
   0.000290     [D5] send_to_all: SETINFO driver.parameter.port "auto"
   0.000391     [D1] Network UPS Tools version 2.8.1.1 (release/snapshot of 2.8.1.1) built with FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367); Target: aarch64-unknown-freebsd15.0; Thread model: posix and configured with flags: --sysconfdir=/usr/local/etc/nut --program-transform-name= --localstatedir=/var/db/nut --datadir=/usr/local/etc/nut --with-devd-dir=/usr/local/etc/devd --with-drvpath=/usr/local/libexec/nut --with-statepath=/var/db/nut --with-altpidpath=/var/db/nut --with-pidpath=/var/db/nut --with-pkgconfig-dir=/usr/local/libdata/pkgconfig --with-user=nut --with-group=nut --with-python=/usr/local/bin/python3.11 --without-python2 --with-python3=/usr/local/bin/python3.11 --without-nut_monitor --with-ltdl --with-nut-scanner --with-avahi --with-cgi --with-cgipath=/usr/local/www/cgi-bin/nut --with-htmlpath=/usr/local/www/nut --with-gd-includes=-I/usr/local/include --with-gd-libs='-L/usr/local/lib -lgd' --without-dev --without-freeipmi --without-ipmi --with-doc=no --with-modbus --with-neon --without-nss --with-openssl --with-powerman --with-serial --with-snmp --with-usb=auto --prefix=/usr/local --mandir=/usr/local/man --disable-silent-rules --infodir=/usr/local/share/info/ --build=aarch64-portbld-freebsd15.0
   0.000429     [D1] debug level is '6'
   0.000451     [D5] send_to_all: SETINFO driver.debug "6"
   0.000470     [D5] send_to_all: SETFLAGS driver.debug RW NUMBER
   0.002571     [D1] Succeeded to become_user(nut): now UID=316 GID=316
   0.002739     [D5] send_to_all: SETINFO device.type "ups"
   0.002769     [D5] send_to_all: SETINFO driver.state "init.device"
   0.002783     [D1] upsdrv_initups (non-SHUT)...
   0.002801     [D2] Initializing an USB-connected UPS with library libusb-1.0.0 (API: 0x1000102) (NUT subdriver name='USB communication driver (libusb 1.0)' ver='0.46')
   0.003507     [D2] libusb1: No appropriate HID device found
   0.003576     libusb1: Could not open any HID devices: no USB buses found
   0.003590     No matching HID UPS found
   0.003661     [D5] send_to_all: SETINFO driver.state "cleanup.exit"
   0.003682     upsnotify: failed to notify about state 4: no notification tech defined, will not spam more about it
[24.03-RC][admin@pfSense1100.localdomain]/root:

There certainly is USB bus:

[24.03-RC][admin@pfSense1100.localdomain]/root: usbconfig
ugen1.1: <Marvell EHCI root HUB> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.1: <Generic XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <American Power Conversion Smart-UPS 1000 FW:UPS 09.3 / ID18> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)
[24.03-RC][admin@pfSense1100.localdomain]/root:

dennypage · Apr 17, 2024, 3:37 PM

@pfsjap said in Netgate 1100 with 24.03-RC and APC Smart-UPS 1000:

0.002783 [D1] upsdrv_initups (non-SHUT)...
0.002801 [D2] Initializing an USB-connected UPS with library libusb-1.0.0 (API: 0x1000102) (NUT subdriver name='USB communication driver (libusb 1.0)' ver='0.46')
0.003507 [D2] libusb1: No appropriate HID device found
0.003576 libusb1: Could not open any HID devices: no USB buses found

Strange. Immediately following the "Initializing an USB-connected UPS..." line should be a "Checking device 1 of 1" message. It would appear that the USB iterator isn't seeing any USB devices. This could either be an issue with NUT's own usb code, libusb, or a kernel issue.

Were you running 23.09.1 prior to updating to 24.03-RC? Did the UPS work there?

Also, have you tried the other USB port?

pfsjap · Apr 17, 2024, 8:23 PM

@dennypage There has been many changes lately due to another problem (Netgate 1100 not rebooting), but if my memory serves me well, I first tried nut in this device running 23.09.1. It didn't work so then used apcupsd for awhile. I would prefer nut, though.

Yes, have tried both USB ports.

dennypage · Apr 17, 2024, 8:29 PM

@pfsjap So, if I understand you correctly, it currently works with apcupsd under 24.03-RC?

pfsjap · Apr 18, 2024, 9:42 AM

@dennypage Device is currently running 2nd RC version on which I installed both nut and apcupsd: nut did not work (same driver error msg as before) and apcupsd connected to the UPS.

Both are using the same libusb version (libusb.so.3):

[24.03-RC][admin@pfSense1100.localdomain]/root: ldd /usr/local/libexec/nut/usbhid-ups
/usr/local/libexec/nut/usbhid-ups:
        libusb.so.3 => /usr/lib/libusb.so.3 (0x688505092000)
        libm.so.5 => /lib/libm.so.5 (0x6885044f5000)
        libltdl.so.7 => /usr/local/lib/libltdl.so.7 (0x68850627c000)
        libthr.so.3 => /lib/libthr.so.3 (0x688505b3f000)
        libc.so.7 => /lib/libc.so.7 (0x68850729c000)
[24.03-RC][admin@pfSense1100.localdomain]/root:
[24.03-RC][admin@pfSense1100.localdomain]/root: ldd /usr/local/sbin/apcupsd
/usr/local/sbin/apcupsd:
        libwrap.so.6 => /usr/lib/libwrap.so.6 (0x7706afd8a000)
        libusb.so.3 => /usr/lib/libusb.so.3 (0x7706b0243000)
        libc++.so.1 => /lib/libc++.so.1 (0x7706b0714000)
        libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x7706b1454000)
        libm.so.5 => /lib/libm.so.5 (0x7706b2cf9000)
        libthr.so.3 => /lib/libthr.so.3 (0x7706b1548000)
        libc.so.7 => /lib/libc.so.7 (0x7706b1ad1000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7706b340a000)
[24.03-RC][admin@pfSense1100.localdomain]/root:

dennypage · Apr 19, 2024, 7:46 PM

@pfsjap This would generally indicate an issue with NUT's USB library. There is a new version of NUT pending release, although I do not know when it will be pushed. Unfortunately I do not have the resources to do a test arm build.

In the interim, just to make sure it's not a privilege issue, can you test with

user=root

in the Extra Arguments to driver section please?

pfsjap · Apr 20, 2024, 1:30 PM

@dennypage It is somehow related to privileges, after setting user=root, nut connected to UPS successfully.

dennypage · Apr 21, 2024, 4:12 PM

@pfsjap Can you have a look at the USB quirks post and see if you are missing a quirk please? Thanks.

pfsjap · Apr 22, 2024, 3:49 PM

@dennypage I first checked the vendor and product ids and then checked if there was a quirk and there was (VID=0x051d PID=0x0003):

[24.03-RC][admin@pfSense1100.localdomain]/root: usbconfig dump_device_quirks | grep 0x051d
VID=0x051d PID=0x0002 REVLO=0x0000 REVHI=0xffff QUIRK=UQ_HID_IGNORE
VID=0x051d PID=0x0003 REVLO=0x0000 REVHI=0xffff QUIRK=UQ_HID_IGNORE
[24.03-RC][admin@pfSense1100.localdomain]/root:

Then deleted user=root from driver arguments and restarted nut. It connected to UPS successfully, also after disconnecting and connecting USB cable.

So why is it now working ok? When testing nut and apcupsd above, both were installed, but only one was enabled at a time. After setting user=root and nut working ok, I deinstalled apcupsd. Maybe apcupsd was the culprit, even having been disabled? If that is the case, then I'm sorry and thank you for going through this. ** Edit: or maybe having driver running as root somehow made it work also without it. **

dennypage · Apr 22, 2024, 6:12 PM

@pfsjap Only thing I can think of is that the quirk isn't being recognized and that the kernel is attaching a driver (uhid0). Running the driver as root allows it to detach the driver, which is required in order to access it as a generic USB device. Once the driver is detached, it will always work.

it fails again following a reboot, check to see if the kernel has attached a driver by using show_ifdrv. If it has, then the quirk isn't being applied for some reason.

pfsjap · Apr 22, 2024, 8:06 PM

@dennypage After my previous post I already did think about the quirk possibly not being there after reboot, so did shut down 1100 and restarted it. When booted up, nut had connected to the UPS. Quirk being applied is also verified by this:

[24.03-RC][admin@pfSense1100.localdomain]/root: usbconfig
ugen1.1: <Marvell EHCI root HUB> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.1: <Generic XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <American Power Conversion Smart-UPS 1000 FW:UPS 09.3 / ID18> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)
[24.03-RC][admin@pfSense1100.localdomain]/root: usbconfig -d ugen0.2 show_ifdrv
ugen0.2: <American Power Conversion Smart-UPS 1000 FW:UPS 09.3 / ID18> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (2mA)
[24.03-RC][admin@pfSense1100.localdomain]/root:

Since I didn't add the quirk, it did come with nut package, right?

dennypage · Apr 22, 2024, 9:38 PM

@pfsjap said in Netgate 1100 with 24.03-RC and APC Smart-UPS 1000:

Since I didn't add the quirk, it did come with nut package, right?

Quirks in FreeBSD are part of the kernel itself. NUT distributes a bunch of quirks, but they are not used in FreeBSD.

I'm glad that it works following reboot. However, this leaves me without any way to explain why it didn't work originally.