NUT package (2.8.0 and below)

[whoami TM]

@dennypage said in NUT package:

After loosing power and restarting, what do you see if you read the ups.delay.shutdown with upsrw?

ups.delay.shutdown goes back to the default of 20... Even without losing power and restarting ups.delay.shutdown goes back to the default of 20 after an hour or so.

[dennypage]

@whoami-tm Okay, now add

offdelay=60
ondelay=90

(and only that) in the Extra Arguments to driver section and repeat the test.

[whoami TM]

@dennypage I removed all settings, which only consisted of ignorelb and override.battery.charge.low = 20 in the Extra Arguments to driver field, and started with a blank slate.

I added offdelay=60 and ondelay=90 to the Extra Arguments to driver field and saved. On the UPS Status page it added

Using the command upsrw tripplite the response was

[ups.delay.shutdown]
Interval to wait after shutdown with delay command (seconds)
Type: STRING
Maximum length: 10
Value: 60

I let about 20mins pass and checked the UPS Status page again and ups.delay.shutdown reverted back to the default ups.delay.shutdown 20. I also checked by looking using the command upsrw tripplite and the response was

[ups.delay.shutdown]
Interval to wait after shutdown with delay command (seconds)
Type: STRING
Maximum length: 10
Value: 20

but it still displayed

I went back into UPS settings and saved again and again got the results posted above of ups.delay.shutdown 60 in both the status page and using cmd upsrw tripplite.
The UPS said it had 60mins of run time. I killed power to the main... The UPS instantly dropped run time down to 30mins... The UPS passed 50% which should have been the Smart1500LCD default, pfSense did not shut down and the Smart1500 did nothing. I let the UPS go all the way down to 0% and it continued for another 30-45 mins before it didnt have enough power to push my Netgate box and both my Netgate box and Unifi POE switch shut off from the Tripplite losing power then a few seconds later my Unifi switch started up again with more power now that the Netgate box was offline. Then I gave power back to the main, pulled the plug on my Netgate box and let the power drain from the PSU so it would start again, then I had to back up my pfSense config because the config.xml got wiped.

I'm convinced there is something wrong with this Tripplite UPS amazon sent me. I just bought an APC Back-UPS Pro 1500VA Gaming UPS to run a Windows PC that runs my BlueIris NVR software and DeepStack AI server to process images sent by BlueIris and it also uses about 100 watts and says it has 60 mins of runtime but when I cut the main it doesn't drop the runtime in half like the Tripplite.

As a test, I've connected this headless Windows PC running BlueIris to the Tripplite UPS and used the PowerAlert software to change settings. For one I could only see half of the words in the PowerAlert software, possibly because I'm remoting into the windows PC from a laptop with 4k screen ¯_(ツ)_/¯. I was able to set offdelay = 60 and ondelay = 120 and clicked execute to test and it actually worked. But just like the networking gear, the runtime dropped in half when I cut the main. I have yet to do a full test with the windows pc and poweralert software letting the ups run down battery past the default 50% to see what would happen.

[dennypage]

@whoami-tm said in NUT package:

I let about 20mins pass and checked the UPS Status page again and ups.delay.shutdown reverted back to the default ups.delay.shutdown 20.

Yea, this should not happen. Something is wrong.

@whoami-tm said in NUT package:

The UPS said it had 60mins of run time. I killed power to the main... The UPS instantly dropped run time down to 30mins... The UPS passed 50% which should have been the Smart1500LCD default, pfSense did not shut down and the Smart1500 did nothing.

The sudden drop of runtime is is not completely abnormal for a standby UPS. To fix this, you need would need to run a deep test while the load is enabled.

The default battery.charge.low on the TrippLite is 10% rather than 50%. This is settable via upsrw.

Seems like your UPS may have some nvram problems though...

[whoami TM]

@dennypage said in NUT package:

The default battery.charge.low on the TrippLite is 10% rather than 50%. This is settable via upsrw.

I'm a little confused. upsrw -h says:

Network UPS Tools upsrw 2.7.4

usage: upsrw [-h]
       upsrw [-s <variable>] [-u <username>] [-p <password>] <ups>

Demo program to set variables within UPS hardware.

  -h            display this help text
  -s <variable>	specify variable to be changed
		use -s VAR=VALUE to avoid prompting for value
  -u <username> set username for command authentication
  -p <password> set password for command authentication

  <ups>         UPS identifier - <upsname>[@<hostname>[:<port>]]

Call without -s to show all possible read/write variables.

upsrw tripplite returns:

[ups.delay.shutdown]
Interval to wait after shutdown with delay command (seconds)
Type: STRING
Maximum length: 10
Value: 20

Doesn't that mean that the only option upsrw supports on this UPS is ups.delay.shutdown

upsrw -s battery.charge.low=20 -u admin -p ea798a2ea466af5134e4 tripplite@localhost returns and error message:

Unexpected response from upsd: ERR VAR-NOT-SUPPORTED

[dennypage]

@whoami-tm I'm pretty sure I was mistaken about you having the same UPS I did. Mine was definitely a standby unit, but looking at the Tripp-Lite site, that model shows as a line interactive UPS.

Just to be clear, this is the UPS you have, correct?

Rack version
Tower version

The NUT report for the SMART1500LCD does show battery.charge.low as being settable, but your output definitely indicates that it is not.

Not sure what to think think about that.

[whoami TM]

@dennypage said in NUT package:

Just to be clear, this is the UPS you have, correct?

Rack version

Yes, the Rack Version.

There is a lot of conflicting info. Tripplite says on its website its compatible with Windows and Mac OS X. But in the manual it says Windows/Mac/Linux... and if you follow links from the website into the PowerAlert software page it also says its compatible with NUT.

NUT website says its compatible here. The link to the NUT report you posted is for the Tower which has a different product ID of 3016 and I'm guessing protocol too since the SMART1500LCD I have has product ID and protocol number of 2009.

I called Tripplite support and they said since its not listed as NUT compatible on the direct website they couldn't help me. But I was able to verify that the default ```
battery.charge.low=10

[dennypage]

@whoami-tm said in NUT package:

I called Tripplite support and they said since its not listed as NUT compatible on the direct website they couldn't help me.

Rather unfortunate...

[whoami TM]

I figured I'd give a follow up for anyone in the future who's reading my posts on the Tripplite SmartPro1500LCD aka SMART1500LCD trying to figure out why it wont work.

Personally I ended up giving up and opening a return before my return window closed. I just wanted a cheap rack mountable UPS that had a depth less than 16" to power my network on power failure but the Tripplite Smart1500LCD isn't it. After spending well over 40hrs trying to figure out why the Smart1500LCD wouldn't play nice with NUT I ended up buying a APC BackUps Pro 1500 gaming bgm1500 and a 1U rack shelf. It took me about a whole 2 hrs to get the APC working with apcuspd and do a full test to verify it would accomplish what I was trying to achieve.

If I was to guess why the Smart1500LCD doesn't work with the NUT package in pfSense I think the driver is sending the reboot command to the UPS.

I assume that NUT sends the available instant commands with usbhid-ups based on what is available for productid=2012 to the UPS to get it to respond.

upscmd -l tripplite returns

Instant commands supported on UPS [tripplite]:

beeper.disable - Disable the UPS beeper
beeper.enable - Enable the UPS beeper
beeper.mute - Temporarily mute the UPS beeper
beeper.off - Obsolete (use beeper.disable or beeper.mute)
beeper.on - Obsolete (use beeper.enable)
load.off - Turn off the load immediately
load.off.delay - Turn off the load with a delay (seconds)
reset.watchdog - Reset watchdog timer
shutdown.reboot - Shut down the load briefly while rebooting the UPS
shutdown.stop - Stop a shutdown in progress
test.battery.start.deep - Start a deep battery test
test.battery.start.quick - Start a quick battery test
test.battery.stop - Stop the battery test

I can verify that beeper.enable & beeper.disable work and the setting holds.

So I'm guessing that ups.delay.shutdown is not written to the actual UPS it's self until the final shutdown command is issued and the timer is set in load.off.delay. I'm thinking this is why setting ups.delay.shutdown with upsrw wouldn't hold and kept reverting back to the default of 20 after a period of time. I'm also thinking setting offdelay=60 in the driver config wouldn't work because the driver is actually sending shutdown.reboot to the UPS instead of load.off.delay with the time specified by offdelay= on fsd which is why the UPS would shutdown within 20 seconds killing pfSense in mid shutdown then waiting about 20-30 seconds before restoring power even though it was on battery and the main was off.

Maybe by reverse engineering and figuring out exactly what commands the driver is sending to the UPS, then editing the driver, you might be able to get this to work.

[whoami TM]

Two months later....
I finely got everything working as intended. Returned the Tripplite Smart1500LCD, got a APC BackUps Pro 1500 BGM1500 and a rack shelf. I initially tested the APC with apcupsd and thought I had everything working until the battery charged to 100%. Once the BGM1500 battery was fully charged and the status changed to online, apcupsd couldn't recognize the status and displayed it as Unknown (ONBATT) which prevented it from sending me notifications on powerloss.
I tried testing with NUT after deleting the apcupsd pkg but couldn't get NUT's USBHID driver to detect the UPS. After a lot of google searches I found someone on TrueNAS forum that got the BGM1500 working with the USBHID driver which gave me the epiphany that I needed to reboot pfSense after deleting apcupsd pkg to get NUT to recognize the UPS.
Now everything works as expected plus the bonus of PushOver notifications.

Extra Arguments to driver:

ignorelb
override.battery.charge.low = 5
override.battery.runtime.low = 180
offdelay=60
pollfreq=2

[dennypage]

@whoami-tm said in NUT package:

After a lot of google searches I found someone on TrueNAS forum that got the BGM1500 working with the USBHID driver which gave me the epiphany that I needed to reboot pfSense after deleting apcupsd pkg to get NUT to recognize the UPS.

This is one of the common issues with USB serial ports under FreeBSD. Mentioned a number of times earlier times earlier in this thread. After installing NUT (or anything else that futzes with USB serial ports), you need to either reboot the OS or unplug and replug the USB device. The reason for this is that the device initialization scripts that set ownership and permissions on the device file only run at initial USB connect. It's annoying as hell, but it's a one-time problem.

[joesl]

Hello everyone,

I have noticed that when there are multiple power losses over a long period of time, sooner or later, there will be a safe shutdown sent by my pfsense running NUT to all the devices listening in. Power comes back up and devices start again. Often this repeat 2-3 times. Finally UPS comes back up, devices start, but then the power goes out again while I suppose the UPS is almost dead, UPS gives out and turns off and thus I get an improper shutdown.

Is there a parameter setting to avoid automatic restart after safe shutdown? i.e. if NUT says to safe-shutdown, then a person will have to come and turn on the devices again?

Or is there an alternative approach?

Thanks!

[dennypage]

@joesl said in NUT package:

Is there a parameter setting to avoid automatic restart after safe shutdown? i.e. if NUT says to safe-shutdown, then a person will have to come and turn on the devices again?

A couple of approaches...

You can configure the BIOS to not automatically start when power is restored. This will mean that human intervention will always be required for startup.

A much better option, if available, is to configure the UPS to not restore load power until there is sufficient battery available. Use upsrw and see if you have a variable called "battery.charge.restart".

NUT variable documentation is here.

[joesl]

@dennypage Thanks a lot!

I ran upsrw upsname and I don't see that variable. Am I safe to assume it's not available?
Upon further reading and thanks to your leads, I think the issue is on my devices after all due to a poor software handling. It seems they keep going on/off until pfsense goes into FSD, then they don't receive anymore info from the UPS and just die when the UPS dies.

[Patch]

@joesl said in NUT package:

Or is there an alternative approach?

Configure controlled devices to both

1. Not automatically starting on power supplied
2. But instead wake on LAN

[dennypage]

@joesl said in NUT package:

I ran upsrw upsname and I don't see that variable. Am I safe to assume it's not available?

Yes, it's safe to assume that the UPS does not support configuring the minimum battery level before start. Your UPS probably uses a fixed value like 3%.

Upon further reading and thanks to your leads, I think the issue is on my devices after all due to a poor software handling. It seems they keep going on/off until pfsense goes into FSD, then they don't receive anymore info from the UPS and just die when the UPS dies.

Based on your description, the UPS is starting the load before there is enough battery for all attached systems to complete a full startup, including NUT connections, before entering a low battery condition. So if mains power goes out before the NUT server fully starts and the NUT clients all reconnect to the server, the server will initiate shutdown and the clients will never know. This is not a software problem per se.

The only way to address the issue is to ensure that the UPS has enough battery to fully bring the load system(s) up to a fully functional state before reentering a low battery condition. Since the UPS has no idea what's connected to it, the configuration needs to be either minimum percentage of battery charge before start, or wait X number of minutes before start.

If I were in your situation, I would consider these options:

• Replace the batteries in the UPS if they are more than 24 months old.
• Run a full battery calibration test to ensure that the UPS has a correct view of battery %.
• Get a larger UPS that has more battery capacity.
• Move some of the client systems off the UPS to reduce the load (which will increase the effective reserve).

You could also look at getting a network (SNMP) based UPS so that the individual hosts all speak directly to the UPS. While this feature alone may not address your issue, network UPSs usually have larger capacities and more configurability, so your issue would likely be solved one way or another.

[dennypage]

@patch said in NUT package:

Configure controlled devices to both

1. Not automatically starting on power supplied
2. But instead wake on LAN

Hmm... I don't personally know of any system were a WOL packet will trigger the BIOS to boot an operating system.

[joesl]

@dennypage Thanks a lot for your whole writeup!

I think you are probably correct on what happened (though I'm still a little confused). This has happened 2-3 times in the past couple of years. I actually have a substantial UPS for my load which was able to go on for almost 6 hours of these on/off conditions (always happens during the night and eventually gives up in the early morning). I use BackUPS Pro with the external battery pack and only have a 150W load on the main UPS. All other devices on secondary UPS.

I captured some logs if you are interested in this post. I can't understand the logic for the devices (I have multiple and all behave in the same way).. 7:16 devices go to a "kind of shutdown" state, 7:29 pfsense sends a "we are doomed" signal and turns off, 7:34 the devices decide to turn on again (probably coz power went back to UPS briefly) and at this stage there is no more NUT (pfsense is off) then UPS dies and so do the devices. I would expect a permanent shutdown on the devices like pfsense does when we reach the LOWBATT/FSD state sent by pfsense but they just keep doing the "safe state" instead. Moreover the devices have a "restart automatically when power supply issue is fixed" flag set to OFF, but still they turned on.. Opened a ticket with vendor.

As of now I did an override on NUT to consider the low battery state to be 30% and disabled the "safe state" on the devices with a different logic. The hope is that they go in shutdown/safestate together (pfsense and devices) and the issue is just the out-of-sync dance.

Thanks again for your help!

[missionit.tech]

Im having a problem with a Tripplite Internet550U. I am using the USB cable that came with the UPS. The UPS is brand new. I installed the NUT package on my Netgate SG2100 and then connected my Tripplite to the SG2100 USB port. Then I go to the Services>UPS and then the UPS settings. I select Local USB, then I give the USB a name "INTERNET550U" and then I select the driver USBHID (which is the driver that it says should be compatible with that UPS on the NUT compatibility website). I click Save and then wait and eventually get the "Status Alert: The UPS requires attention." Then in the System log I get "Can't connect to UPS [INTERNET550U] (usbhid-ups-INTERNET550U): No such file or directory" then "Startup successful" then "User local-monitor@::1 logged into UPS [INTERNET550U]" then "UPS INTERNET550U is unavailable" and finally "Poll UPS [INTERNET550U] failed - Driver not connected" every 5 seconds forever.
Things I have already tried:

1. rebooting UPS by holding down the power button and waiting until it turns off, then waiting 10 seconds and turning the UPS back on.
2. Unplugged and replugged the USB cable multiple times at both the UPS side and the Netgate side
3. rebooted the SG2100 multiple times
4. uninstalled NUT package, rebooted SG2100, reinstalled NUT package, rebooted SG2100
5. Tried Tripplite driver instead of USBHID driver
6. Attempted to add "user=root" in the ups.conf advanced settings (per https://forum.netgate.com/topic/117510/pfsense-2-3-4-with-nut-2-7-4_4-and-tripp-lite-smx500rt1u-usb-no-matching-hid-ups/8)

Any assistance with this issue would be greatly appreciated! Thank you!

[dennypage]

@missionit-tech said in NUT package:

Then in the System log I get "Can't connect to UPS [INTERNET550U] (usbhid-ups-INTERNET550U): No such file or directory" then "Startup successful" then "User local-monitor@::1 logged into UPS [INTERNET550U]" then "UPS INTERNET550U is unavailable" and finally "Poll UPS [INTERNET550U] failed - Driver not connected" every 5 seconds forever.

Please

1. reboot pfSense with the UPS attached
2. post a screen capture of your UPS settings (Services -> UPS -> UPS Settings)
3. post system log entries matching 'usb' (Status -> System Logs -> System -> General, click on wrench icon and type 'usb' in the Message field, click Apply Filter).