NUT package (2.8.0 and below)
-
So i ran a deep battery test, and it passes every time. Is there anything diagnostic wise i could do to see what's going on at the time it reports LOWBATT, because as far as i can tell its never low. This just started happening after I upgraded pfSense to the latest version 2.4.4, If i remember, i was on 2.3.2. Something has changed in the way pfsense is handling this across these versions.
-
This just started happening after I upgraded pfSense to the latest version 2.4.4, If i remember, i was on 2.3.2. Something has changed in the way pfsense is handling this across these versions.
Can you clarify the version information please? The latest version of pfSense is 2.3.4. Version 2.4 of pfSense is still in development (beta). The latest version of the nut package is 2.7.4_4, which uses underlying nut version 2.7.4_1. Thanks.
Btw, unless you were previously running pfSense 2.2.X or below, I'm guessing that the "something changed with nut" may be coincidental. This stuff hasn't changed much in the last year. The underlying release of nut for pfSense, version 2.7.4, was introduced in pfSense 2.3.0. The current generation of the nut package, version 2.7.4, was introduced shortly thereafter. Note that the prior generation nut package, 2.3.X, didn't have the ability to send email notifications so you would have only noticed LOWBATT messages if you scanned the system log.
-
This just started happening after I upgraded pfSense to the latest version 2.4.4, If i remember, i was on 2.3.2. Something has changed in the way pfsense is handling this across these versions.
Can you clarify the version information please? The latest version of pfSense is 2.3.4. Version 2.4 of pfSense is still in development (beta). The latest version of the nut package is 2.7.4_4, which uses underlying nut version 2.7.4_1. Thanks.
Btw, unless you were previously running pfSense 2.2.X or below, I'm guessing that the "something changed with nut" may be coincidental. This stuff hasn't changed much in the last year. The underlying release of nut for pfSense, version 2.7.4, was introduced in pfSense 2.3.0. The current generation of the nut package, version 2.7.4, was introduced shortly thereafter. Note that the prior generation nut package, 2.3.X, didn't have the ability to send email notifications so you would have only noticed LOWBATT messages if you scanned the system log.
Ok sorry, let me clear that up, I'm on the latest version now of 2.3.4. Previously on 2.3.2, never had a single LOWBATT alert. Back checked email, July 9th I upgrade to 2.3.4, alerts start rolling in since, randomly. Is something polling differently now?
-
Btw, unless you were previously running pfSense 2.2.X or below, I'm guessing that the "something changed with nut" may be coincidental. This stuff hasn't changed much in the last year. The underlying release of nut for pfSense, version 2.7.4, was introduced in pfSense 2.3.0. The current generation of the nut package, version 2.7.4, was introduced shortly thereafter. Note that the prior generation nut package, 2.3.X, didn't have the ability to send email notifications so you would have only noticed LOWBATT messages if you scanned the system log.
Ok sorry, let me clear that up, I'm on the latest version now of 2.3.4. Previously on 2.3.2, never had a single LOWBATT alert. Back checked email, July 9th I upgrade to 2.3.4, alerts start rolling in since, randomly. Is something polling differently now?
The only scenario I can think of would be having an earlier version of pfSense and the nut package installed, then installing 2.3.2 and not reinstalling the packages until the upgrade to 2.3.4.
If this was the case, there would have been two changes. First would have been the underlying nut version going from 2.7.3 to 2.7.4. Second would have been the nut package going from 2.3.X to 2.7.4. If this happened, it should have been pretty obvious because the nut package would have put up a banner about settings migration and forced the migrated settings to be explicitly saved before it would start. Nut email notifications are also off by default, so at some point they had to be explicitly enabled as well.
For nut, the release notes are here. Nothing in particular jumps out at me, but I'm not a nut developer. For the nut package, there hasn't been much since the introduction of 2.7.4, which is when the ability to configure email was introduced. Overall, my best guess is that the LOWBATT thing has been happening all along and it wasn't noticed in the system logs until email notifications were enabled because it didn't impair operation.
So I guess the key questions to think about are: Was there a nut configuration migration performed? If so, when? And when were email notifications enabled?
-
Btw, unless you were previously running pfSense 2.2.X or below, I'm guessing that the "something changed with nut" may be coincidental. This stuff hasn't changed much in the last year. The underlying release of nut for pfSense, version 2.7.4, was introduced in pfSense 2.3.0. The current generation of the nut package, version 2.7.4, was introduced shortly thereafter. Note that the prior generation nut package, 2.3.X, didn't have the ability to send email notifications so you would have only noticed LOWBATT messages if you scanned the system log.
Ok sorry, let me clear that up, I'm on the latest version now of 2.3.4. Previously on 2.3.2, never had a single LOWBATT alert. Back checked email, July 9th I upgrade to 2.3.4, alerts start rolling in since, randomly. Is something polling differently now?
The only scenario I can think of would be having an earlier version of pfSense and the nut package installed, then installing 2.3.2 and not reinstalling the packages until the upgrade to 2.3.4.
If this was the case, there would have been two changes. First would have been the underlying nut version going from 2.7.3 to 2.7.4. Second would have been the nut package going from 2.3.X to 2.7.4. If this happened, it should have been pretty obvious because the nut package would have put up a banner about settings migration and forced the migrated settings to be explicitly saved before it would start. Nut email notifications are also off by default, so at some point they had to be explicitly enabled as well.
For nut, the release notes are here. Nothing in particular jumps out at me, but I'm not a nut developer. For the nut package, there hasn't been much since the introduction of 2.7.4, which is when the ability to configure email was introduced. Overall, my best guess is that the LOWBATT thing has been happening all along and it wasn't noticed in the system logs until email notifications were enabled because it didn't impair operation.
So I guess the key questions to think about are: Was there a nut configuration migration performed? If so, when? And when were email notifications enabled?
Email notifications have been on all along, I'd get OL and OB notifications during switches, but never random LOWBATTs until after the upgrade, nut package has been on 2.7.4. I just don't see why its sending LOWBATT, when it never comes off 100%, LOWBATT shouldn't even come out until past 70% or so and if it was actually discharging, it sure wouldn't come back to 100% that quick. I'm going to try moving NUT master servers and see if it continues, before trying a rollback as another experiment.
-
Do you have another UPS to test with by chance?
-
Email notifications have been on all along, I'd get OL and OB notifications during switches, but never random LOWBATTs until after the upgrade, nut package has been on 2.7.4. I just don't see why its sending LOWBATT, when it never comes off 100%, LOWBATT shouldn't even come out until past 70% or so and if it was actually discharging, it sure wouldn't come back to 100% that quick.
Okay, so no changes in the nut version or the nut package. Presuming the same UPS the entire time.
Just for completeness, can you post output from the following two commands?
upsc <upsname>upsrw <upsname>Thanks</upsname></upsname>
-
Email notifications have been on all along, I'd get OL and OB notifications during switches, but never random LOWBATTs until after the upgrade, nut package has been on 2.7.4. I just don't see why its sending LOWBATT, when it never comes off 100%, LOWBATT shouldn't even come out until past 70% or so and if it was actually discharging, it sure wouldn't come back to 100% that quick.
Okay, so no changes in the nut version or the nut package. Presuming the same UPS the entire time.
Just for completeness, can you post output from the following two commands?
upsc <upsname>upsrw <upsname>Thanks</upsname></upsname>
battery.charge: 100
battery.charge.low: 50
battery.charge.warning: 20
battery.mfr.date: CPS
battery.runtime: 960
battery.runtime.low: 600
battery.type: PbAcid
battery.voltage: 24.0
battery.voltage.nominal: 24
device.mfr: CPS
device.model: OR1500PFCRT2U
device.type: ups
driver.flag.ignorelb: enabled
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: /dev/ugen0.6
driver.parameter.synchronous: no
driver.version: 2.7.4
driver.version.data: CyberPower HID 0.4
driver.version.internal: 0.41
input.transfer.high: 139
input.transfer.low: 88
input.voltage: 118.0
input.voltage.nominal: 120
output.voltage: 118.0
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.load: 58
ups.mfr: CPS
ups.model: OR1500PFCRT2U
ups.productid: 0601
ups.realpower.nominal: 900
ups.status: OL
ups.test.result: Done and passed
ups.timer.shutdown: -60
ups.timer.start: -60
ups.vendorid: 0764[input.transfer.high]
High voltage transfer point (V)
Type: STRING
Maximum length: 10
Value: 139[input.transfer.low]
Low voltage transfer point (V)
Type: STRING
Maximum length: 10
Value: 88[ups.delay.shutdown]
Interval to wait after shutdown with delay command (seconds)
Type: STRING
Maximum length: 10
Value: 20[ups.delay.start]
Interval to wait before (re)starting the load (seconds)
Type: STRING
Maximum length: 10
Value: 30 -
battery.charge.low: 50
battery.charge.warning: 20I didn't look at all the variables, but this one is an issue: battery.charge.warning should be higher than battery.charge.low.
What is being set in the Extra Arguments to driver section? Anything in the Advanced section?
-
battery.charge.low: 50
battery.charge.warning: 20I didn't look at all the variables, but this one is an issue: battery.charge.warning should be higher than battery.charge.low.
What is being set in the Extra Arguments to driver section? Anything in the Advanced section?
Driver Extra Args:
port = /dev/ugen0.6
ignorelb
override.battery.charge.low = 50
override.battery.runtime.low = 600upsmon.conf
RUN_AS_USER root
upsd.conf
LISTEN 10.10.0.1
upsd.users
[monitor]
password = *************
upsmon slave -
Hello,
Is it possible to have the NUT daemon monitor and manage two UPSes?
I have a CyberPower1500 and a CyberPower1350 in my set-up and the pfSense firewall is the only bare-metal box (everything else is an ESXi server). I want to have each of the other systems query the NUT daemon on the pfSense for the status of the UPS that host is connected to. (the UPSes are on different circuits so there are scenarios where one goes down but not both)
What I don't know is how to set these configurations. I see there are a number of advanced options in the UI, I'm a little lost as to where to start. Has anyone done this before? I'd like to do it via the pfSense UI so my configs will survive upgrades and reinstalls, rather than being hidden down in config files.
To make this work I figure I need to pass options for additional configurations here so that the port of each UPS is listed explicitly:
[2.3.4-RELEASE][admin@pfsense.thirtynineohsix]/root: cat /usr/local/etc/nut/ups.conf [CyberPower1500] driver=usbhid-ups port=auto…and here so that each UPS has an explicit TCP port for communication with the remote servers.:
[2.3.4-RELEASE][admin@pfsense.thirtynineohsix]/root: cat /usr/local/etc/nut/upsd.conf LISTEN 127.0.0.1 LISTEN ::1… I'm really not sure what I need to do or what will happen automatically here:
[2.3.4-RELEASE][admin@pfsense.thirtynineohsix]/root: cat /usr/local/etc/nut/upsmon.conf MONITOR CyberPower1500 1 monuser f0e07165bec481f09a93 master SHUTDOWNCMD "/sbin/shutdown -p +0" POWERDOWNFLAG /etc/killpower NOTIFYCMD /usr/local/pkg/nut/nut_email.php NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC NOTIFYFLAG LOWBATT SYSLOG+WALL+EXEC NOTIFYFLAG FSD SYSLOG+WALL+EXEC NOTIFYFLAG COMMOK SYSLOG+WALL+EXEC NOTIFYFLAG COMMBAD SYSLOG+WALL+EXEC NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC NOTIFYFLAG REPLBATT SYSLOG+WALL+EXEC NOTIFYFLAG NOCOMM SYSLOG+WALL+EXEC NOTIFYFLAG NOPARENT SYSLOG+WALL+EXECHere's the USB information .. the difference I can see is the "ugen1.[2|3]" but everything else is the same? How do I identify/label the port?
[2.3.4-RELEASE][admin@pfsense.thirtynineohsix]/root: usbconfig dump_device_desc ugen0.1: <uhci root="" hub="" intel="">at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=SAVE (0mA) [ details from four root hubs elided ] ugen1.2: <cp 1500c="" cps="">at usbus1, cfg=0 md=HOST spd=LOW (1.5Mbps) pwr=ON (50mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0110 bDeviceClass = 0x0000 <probed by="" interface="" class="">bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0008 idVendor = 0x0764 idProduct = 0x0501 bcdDevice = 0x0001 iManufacturer = 0x0003 <cps>iProduct = 0x0001 < CP 1500C> iSerialNumber = 0x0000 <no string="">bNumConfigurations = 0x0001 ugen1.3: <cp 1350c="" cps="">at usbus1, cfg=0 md=HOST spd=LOW (1.5Mbps) pwr=ON (50mA) bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0110 bDeviceClass = 0x0000 <probed by="" interface="" class="">bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0008 idVendor = 0x0764 idProduct = 0x0501 bcdDevice = 0x0001 iManufacturer = 0x0003 <cps>iProduct = 0x0001 < CP 1350C> iSerialNumber = 0x0000 <no string="">bNumConfigurations = 0x0001</no></cps></probed></cp></no></cps></probed></cp></uhci>I'm okay if only one shows up in the UPS widget, it's more important that the respective machines know how to validate their power source.
Thanks for any insights …
--jg3
-
Driver Extra Args:
port = /dev/ugen0.6
ignorelb
override.battery.charge.low = 50
override.battery.runtime.low = 600The ups is at 58% load, or approximately 610 watts. Expected runtime of 14-15 min according to Cyber. So the shutdown should be triggered approximately 5 min after mains go offline.
One of the notes in the nut documentation on ignorelb is that some UPSs generate a LB indication almost immediately after loss of mains. In the context of general Cyberpower flakiness, I can see this happening either from loss of mains, high/low transfer, or maybe even a UPS initiated self test. Even though it would not initiate a shutdown with your configuration, nut would still generate a notification if the UPS sent a LB alert. Is it possible this is happening?
In terms of timing, did the load level change around the time of the upgrade? Or Driver Extra arguments?
-
@jg3:
Is it possible to have the NUT daemon monitor and manage two UPSes?
There is no way to do this in the UI. Sorry.
-
Driver Extra Args:
port = /dev/ugen0.6
ignorelb
override.battery.charge.low = 50
override.battery.runtime.low = 600The ups is at 58% load, or approximately 610 watts. Expected runtime of 14-15 min according to Cyber. So the shutdown should be triggered approximately 5 min after mains go offline.
One of the notes in the nut documentation on ignorelb is that some UPSs generate a LB indication almost immediately after loss of mains. In the context of general Cyberpower flakiness, I can see this happening either from loss of mains, high/low transfer, or maybe even a UPS initiated self test. Even though it would not initiate a shutdown with your configuration, nut would still generate a notification if the UPS sent a LB alert. Is it possible this is happening?
In terms of timing, did the load level change around the time of the upgrade? Or Driver Extra arguments?
Not sure i'm following completely. As far as I know, the main is never lost, i never get any switch alerts, and never see any power surge/ flicker etc if that's what you're asking? The only thing that changed a while back was a 8port POE switch added to the load. Don't see how going from 52% load to 58% should say LOWBATT? Just had an alert come out now about 10minutes ago, checked the UPS immediately and its 100% charge. Driver Args and all config stayed the same. Is there something i can tweak / change to try to eliminate this.
-
As far as I know, the main is never lost, i never get any switch alerts, and never see any power surge/ flicker etc if that's what you're asking?
Yep. Or load surges, mains frequency change, UPS initiated self tests, etc.
In general, I doubt that nut is "making up" the event, and believe that the Cyberpower is actually issuing a LB signal which nut is dutifully logging. You can get definitive verification by running usbdump to capture all the traffic involving the ups, but I'm not sure it's worth it to delve this deeply though. :)
The ups model doesn't have a modifiable battery.charge.low, so I don't know of anything you can tweak with nut. You might search/ask in the nut users mailing list, but I expect that they will view it as a UPS problem. About the only other diagnostic recommendation I can think of is to change the load characteristics and see if it affects the issue. Temporarily removing the PoE switch perhaps.
Sorry I don't have a better recommendation.
-
As far as I know, the main is never lost, i never get any switch alerts, and never see any power surge/ flicker etc if that's what you're asking?
Yep. Or load surges, mains frequency change, UPS initiated self tests, etc.
In general, I doubt that nut is "making up" the event, and believe that the Cyberpower is actually issuing a LB signal which nut is dutifully logging. You can get definitive verification by running usbdump to capture all the traffic involving the ups, but I'm not sure it's worth it to delve this deeply though. :)
The ups model doesn't have a modifiable battery.charge.low, so I don't know of anything you can tweak with nut. You might search/ask in the nut users mailing list, but I expect that they will view it as a UPS problem. About the only other diagnostic recommendation I can think of is to change the load characteristics and see if it affects the issue. Temporarily removing the PoE switch perhaps.
Sorry I don't have a better recommendation.
Alright, appreciate the help. the UPS still has warranty, I just dunno exactly how to pick that battle with CP, they will say its WOD because self test passes. Fact of matter is, I've almost had this UPS for a year with flawless operation, and now this just started July 9th and coincidence or not with the upgrade. It would be a shame if I can't run 58% load on a UPS, i would think 80% would be where you don't exceed.
I called CyberPower support, and of course, they just brush it off as a NUT issue since the battery never drops charge so its not really low. He mentioned something about if your runtime goes below a certain value which can be specified if using "their power software" it would set an error. Is there something equivalent in NUT? Looking at the output again, i can change the battery.charge.low and .warning. I set charge.low = 40 and charge.warning = 50. battery.runtime.low is set at 600, so load would need to increase to the point where runtime drops below 600? It currently sits around 840.
-
Looking at the output again, i can change the battery.charge.low and .warning. I set charge.low = 40 and charge.warning = 50. battery.runtime.low is set at 600, so load would need to increase to the point where runtime drops below 600? It currently sits around 840.
Yes, you do want the warning to happen before the low. With your runtime number and low setting, you will actually initiate shutdown ~4 minutes after loosing mains.
I would tend to set all of them a little lower than what you have, but it depends upon your situation. I.E. how long it takes your systems to shut down. On my general purpose systems I use:
override.battery.charge.warning = 25
override.battery.charge.low = 20
override.battery.runtime.low = 300The reason I set these variables is because I have several servers that can take 2-3 minutes to perform a clean shutdown. Certainly longer than the 60 seconds or so warning I get from the LB signal.
For the firewall I use more aggressive values because pfSense shuts down quickly:
override.battery.charge.warning = 25
override.battery.charge.low = 10
override.battery.runtime.low = 180Truth be told, the LB signal on the UPS probably does a fine job here, and these settings are really not necessary. I just set them out of habit.
Your mileage may vary.
-
I plugged the UPS into a 2K12 server and installed CPs PowerPanel software, let it run for few days and didn't get any LOWBATT messages, then plugged it into my FreeNAS server which is using driver 2.7.3, internal driver .39. and no LOWBATT messages. Here is the output running on the FreeNAS box. So, something is not the same after the upgrade on pfsense because i dont see the LOWBATT message on 2 other boxes. Going to try mirroring the freenas NUT configs, (being doubtful that will matter) then going to have to find some time to reinstall pfsense to 2.3.2 and go from there).
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 20
battery.mfr.date: CPS
battery.runtime: 960
battery.runtime.low: 300
battery.type: PbAcid
battery.voltage: 24.0
battery.voltage.nominal: 24
device.mfr: CPS
device.model: OR1500PFCRT2U
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: /dev/ugen0.6
driver.parameter.synchronous: no
driver.version: 2.7.3
driver.version.data: CyberPower HID 0.4
driver.version.internal: 0.39
input.transfer.high: 139
input.transfer.low: 88
input.voltage: 120.0
input.voltage.nominal: 120
output.voltage: 120.0
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.load: 57
ups.mfr: CPS
ups.model: OR1500PFCRT2U
ups.productid: 0601
ups.realpower.nominal: 900
ups.status: OL
ups.test.result: Done and passed
ups.timer.shutdown: -60
ups.timer.start: -60
ups.vendorid: 0764 -
I plugged the UPS into a 2K12 server and installed CPs PowerPanel software, let it run for few days and didn't get any LOWBATT messages, then plugged it into my FreeNAS server which is using driver 2.7.3, internal driver .39. and no LOWBATT messages.
So, something is not the same after the upgrade on pfsense because i dont see the LOWBATT message on 2 other boxes. Going to try mirroring the freenas NUT configs, (being doubtful that will matter) then going to have to find some time to reinstall pfsense to 2.3.2 and go from there).
When you put it back on the pfSense box did the LOWBATT messages immediately return?
-
I plugged the UPS into a 2K12 server and installed CPs PowerPanel software, let it run for few days and didn't get any LOWBATT messages, then plugged it into my FreeNAS server which is using driver 2.7.3, internal driver .39. and no LOWBATT messages.
So, something is not the same after the upgrade on pfsense because i dont see the LOWBATT message on 2 other boxes. Going to try mirroring the freenas NUT configs, (being doubtful that will matter) then going to have to find some time to reinstall pfsense to 2.3.2 and go from there).
When you put it back on the pfSense box did the LOWBATT messages immediately return?
No
