NUT package (2.8.0 and below)

dennypage

Is there anyway to silence the NUT error reporting I'm seeing?

To my knowledge, you cannot silence the error. You are attempting to set a variable in the UPS snmp module that you do not have permission to change (battery.charge.low).

Remove

ignorelb
override.battery.charge.low = 50

from the advanced parameters.

For snmp, you generally have to set variables in the ups using either upsrw (with snmp write access) or a proprietary application. Occasionally you can also use http/https.

You can learn more in the archive of the nut users mailing list.

q54e3w

That fixed that error, thank you, I can configure those values through my Eaton web configuration page as you suggested.
I'm seeing different behavior with this Eaton 9px than my previous APC. Once the battery level falls below my predetermined level (87% for testing) the ups.status turns to OB OFF OB, ESXi shuts down VMs then itself, but pfSense doesn't ever initiate its own shut down. I have 'FINALDELAY 600' in my upsmon.conf additional parameters to give ESXi time to shutdown, however there no progress from pfSense reporting UPS UPS on battery.

upsrw ups
[battery.charge.low]
Remaining battery level when UPS switches to LB (percent)
Type: STRING
Maximum length: 2
Value: 87

[outlet.desc]
Outlet description
Type: STRING
Maximum length: 20
Value: Main Outlet

[ups.delay.shutdown]
Interval to wait after shutdown with delay command (seconds)
Type: STRING
Maximum length: 6
Value: 20

[ups.delay.start]
Interval to wait before (re)starting the load (seconds)
Type: STRING
Maximum length: 6
Value: 30

[ups.start.auto]
UPS starts when mains is (re)applied
Type: ENUM
Option: "yes" SELECTED
Option: "no"



upsc ups
ambient.humidity: 47.40
ambient.temperature: 11.7
battery.charge: 70.00
battery.charge.low: 87
battery.runtime: 12652.00
battery.runtime.low: 13000.00
battery.voltage: 49.00
device.mfr: Eaton
device.model: Eaton 9PX
device.serial: GA18H32185
device.type: ups
driver.name: snmp-ups
driver.parameter.pollinterval: 2
driver.parameter.port: 192.168.10.52
driver.parameter.synchronous: no
driver.version: 2.7.4
driver.version.data: mge MIB 0.5
driver.version.internal: 0.97
input.phases: 1.00
input.transfer.reason: input voltage out of range
outlet.desc: Main Outlet
outlet.id: 0
output.current: 2.00
output.frequency: 60.00
output.phases: 1.00
output.voltage: 119.00
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.firmware: 01.14.4257
ups.firmware.aux: LB
ups.load: 21.00
ups.mfr: Eaton
ups.model: Eaton 9PX
ups.serial: 111122223333
ups.start.auto: yes
ups.status: OB OFF OB
ups.test.result: done and passed
ups.timer.reboot: -1.00
ups.timer.shutdown: -1.00
ups.timer.start: -1.00

dennypage

@q54e3w Unless you are running NUT on pfSense as a NUT master with slaves on other systems, you should not be setting FINDELAY on pfSense. Also, 10 minutes is an incredibly long time...

I'm not very knowledgable about Eaton, but you may also need to set some variables for the snmp driver. See the snmp-ups man page for detail on what is available. Perhaps the polling frequency is too short to constantly catch a shutdown pending situation.

q54e3w

Thanks for the pointers @dennypage, my problem appears to be resolved although I still need to tweak some final timings. Sharing here for any one else who runs across this issue.
The solution was that the snmp drivers 'auto' detection mode resulted in a less than optimal mibs parameter causing improper device reporting. The Eaton manual states that the Network-MS card is compatible with following MIBs:

MIB II (RFC 1213)
Internet Engineering Task Force (IETF) Standard UPS MIB (RFC 1628)
EATON Pulsar MIB (ex MGE) V1.7
EATON Powerware MIB (PowerMib)

Selecting the 'pw' mib allowed me to access the correct parameter and using a SNMP v1 write enabled passphrase allowed adjusting the shutdown parameters (although I've reverted to setting these in the UPS web UI). Adding 'overidelb' was needed to prod the correct OB/LB needed to initiative the shutdown procedure.

mibs = pw
pollfreq = 15
community = private
ignorelb

The 10 minute FINALDELAY I had used was because pfSense in my master node and I want pfSense to stay up whilst my ESXi hosts shutdown the various VMs to ensure routing across VLANs for storage requirements is available.

thanks again for your support and maintenance of this package.

dennypage

@q54e3w I'm glad you got everything straightened out. And I'd like to give a big thank you for posting details on the resolution--I'm sure this will help someone in the future.

doxymoron

Hello, I'm trying to do a couple different things. But I am not sure where I can set these options in. The first thing I want to do is modify the ups.delay.shutdown from 20 to 120. The second thing, is it possible to completely disable shutting down the UPS itself? Thanks.

dennypage

@doxymoron The power off delay, ups.delay.shutdown, is a variable stored in the ups. If it is settable in your ups, you would set this via upsrw.

As to disabling shutdown completely, it is possible, however I would strongly recommend against doing that. If you are looking to maximize your runtime, look at adjusting the battery low settings instead. But be careful of going to far... an extra minute of runtime isn't really worth the corruption risk of an ungraceful shutdown.

doxymoron

@dennypage Ok thanks, I will check if those options are available. I have 2 ESXi servers that I have with the Nut client installed and talking to the pfsense package. But when I did a test letting the battery run down I don't think the ESXi servers had enough time to shut down gracefully. Once my UPS hit the low battery threshold I set to 30%, it shut the UPS completely off 20 seconds later. I've been trying to figure out the best way to set this up, and I think I have to increase the shutdown delay. The ESXi servers only get triggered once the low battery threshold is hit I believe, then will probably take at least a minute to shutdown.

doxymoron

Ok I was able to run this command successfully: upsrw -s ups.delay.shutdown=120 -u admin -p **** CyberPower-01@localhost
OK
So am I correct that this means once my battery.charge.low is hit, it will wait 120 seconds and then issue a shutdown command? This shuts down pfsense and then the UPS itself?

dennypage

@doxymoron Unfortunately the term "shutdown" is a little overloaded. Is used to refer to initiating a shutdown of the operating system, and also to refer to sending a command to the ups to kill the mains.

When a low battery condition is determined to exist, nut initiates a shutdown of the operating system. The os shutdown can take seconds or minutes depending upon the operating system and what is running on it. As one of the final steps of the shutdown, the os calls back into nut which issues the kill command to the ups. This is where the delay described by ups.delay.shutdown comes into effect--the ups waits this amount of time before killing the mains. Usually the 20 seconds is sufficient because the os isn't doing much other than syncing disk buffers for a few seconds followed by a halt.

Compared to pfSense, ESXi can take a very long time to shut down. To get a better handle on this, you might want to consider running a nut slave on the ESXi server. Yes, surprisingly there is one available. A quick write up using FreeNAS can be found here. Using a remote client may give you a little more time and flexibility in how the ESXi shutdown works. Note that I do not have direct experience with the ESXi client and cannot answer questions about it, but there may be others here who have experience with it...

The pfSense information needed for enabling remote clients can be found in the second post of this thread.

doxymoron

Thanks so much for helping me, I understand a bit more. I do actually have the ESXi client installed that should properly shut down the host and all VMs. I think the problem I ran into was that from the time the low battery condition hit, to when the Pfsense box shutdown, and then the UPS itself shutdown, was just too fast. The ESXi server did not have time to do any of the proper shutting down before it lost power. I will test tomorrow to see if increasing the ups.delay.shutdown will help. If it delays the shutting down of the UPS, I think it should.

dennypage

@doxymoron You may also wish to look at the HOSTSYNC variable, which would be set in the upsmon.conf section of the advanced settings.

lesodm

Hi, every time I restart my pfsense box, I need to manually run:
/usr/local/sbin/upsdrvctl -u root start
To get the UPS driver to start.
I noticed it has the below in /usr/local/etc/rc.d/nut.sh
/usr/local/sbin/upsdrvctl start & sleep 1
But editing nut.sh just gets reset after a reboot. Any suggestions please?

dennypage

@lesodm Need more information to help.

What type of ups?
What ups settings do you have you in Services / UPS / Settings?
Following reboot, what entries are there in the system log (Status / System Logs / System / General) matching "ups" or "nut"?
How are you determining that you need to run upsdrvctl?

Gertjan

@lesodm said in NUT package:

I noticed it has the below in /usr/local/etc/rc.d/nut.sh

So it's run (started) when you boot.
Check dmesg and other logs why it fails.

edit :
If iI start in manually :

[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/usr/local/pkg/nut: /usr/local/sbin/upsdrvctl start
Network UPS Tools - UPS driver controller 2.7.4
Network UPS Tools - Generic HID driver 0.41 (2.7.4)
USB communication driver 0.33
Duplicate driver instance detected! Terminating other driver!

Broadcast Message from root@pfsense.brit-hotel-fumel.net
        (no tty) at 11:21 CET...

Communications with UPS ups lost

Using subdriver: APC HID 0.96

Broadcast Message from root@pfsense.brit-hotel-fumel.net
        (no tty) at 11:21 CET...

Communications with UPS ups established

which means : it was already running - it switches driver instance, the USB connection got kicked, NUT connects again => all is well.

lesodm

@dennypage,
*Local USB
*Settings: Driver blazer and listen port settings in upsd.conf and user settings in upsd.users
*There is this error every 5 seconds in the system log:
41044 Poll UPS [UPS] failed - Driver not connected
*UPS connects and reports all info back once I run upsdrvctl

@Gertjan there is a ton of these messages in dmesg:
<INNO TECH USB to Serial> at usbus1 (disconnected)

dennypage

@lesodm You are using a USB to serial adapter? Please post the output of "usbconfig dump_device_desc"

Please post the actual settings you have in the pfSense GUI (Services / UPS / Settings). Either a screen shot or the XML data from the config.

Please post the log messages that match any of these strings, "usb", "ups", or "nut" for the 2 minutes following reboot.

lesodm

@dennypage it's a USB A to B printer cable.
I might have resolved the issue by adding maxretry = 3 to ups.conf. After last nights power outage the UPS reconnected itself. However I see in syslog it reconnects every now and then, something still not 100% then maybe?
"usbconfig dump_device_desc": usbconfig dump.txt
Status/SystemLogs/System/General messages that match any of these strings, "usb", "ups", or "nut" following reboot: Syslog-USB.txt Syslog-UPS.txt Syslog-nut.txt
Actual settings:

doxymoron

@dennypage said in NUT package:

@doxymoron You may also wish to look at the HOSTSYNC variable, which would be set in the upsmon.conf section of the advanced settings.

Thanks for your help. I was able to get this working by just giving my 2 ESXi servers 2 minutes to shutdown before powering off the UPS using the override.ups.delay.shutdown option. I do have another question. If I were to get another UPS, am I able to also connect it via USB to my pfsense box and also monitor it? Thanks.

dennypage

@lesodm [Sorry for the delay, I'm traveling]

Okay, several things now come to mind. Firstly, I'm going to assume that the usb to serial converter is imbedded in the ups itself and not something that you have separately installed. Btw, you didn't say what the manufacturer/model of the ups was.

First, please ensure you can get into the box via ssh because I want you to run the following tests without anything but the ups connected via usb. In other words, no usb keyboard, no usb mouse. I'm also assuming that at this point you are not using an external usb hub. Please confirm.

Disable nut. Go into Services / UPS, select Disabled from the UPS Type drop down list, and press save.
Reboot pfSense.
Log in via ssh.
Run "usbconfig dump_device_desc" and save the output.
Run "ls -l /dev/cua*"
Move the usb connection to a different port on the motherboard.
Repeat steps 4-6 until you have tested all of the usb ports. Keep track of which port goes with which output.

Please post the results.