NUT package (2.8.0 and below)

q54e3w

Thanks for the pointers @dennypage, my problem appears to be resolved although I still need to tweak some final timings. Sharing here for any one else who runs across this issue.
The solution was that the snmp drivers 'auto' detection mode resulted in a less than optimal mibs parameter causing improper device reporting. The Eaton manual states that the Network-MS card is compatible with following MIBs:

MIB II (RFC 1213)
Internet Engineering Task Force (IETF) Standard UPS MIB (RFC 1628)
EATON Pulsar MIB (ex MGE) V1.7
EATON Powerware MIB (PowerMib)

Selecting the 'pw' mib allowed me to access the correct parameter and using a SNMP v1 write enabled passphrase allowed adjusting the shutdown parameters (although I've reverted to setting these in the UPS web UI). Adding 'overidelb' was needed to prod the correct OB/LB needed to initiative the shutdown procedure.

mibs = pw
pollfreq = 15
community = private
ignorelb

The 10 minute FINALDELAY I had used was because pfSense in my master node and I want pfSense to stay up whilst my ESXi hosts shutdown the various VMs to ensure routing across VLANs for storage requirements is available.

thanks again for your support and maintenance of this package.

dennypage

@q54e3w I'm glad you got everything straightened out. And I'd like to give a big thank you for posting details on the resolution--I'm sure this will help someone in the future.

doxymoron

Hello, I'm trying to do a couple different things. But I am not sure where I can set these options in. The first thing I want to do is modify the ups.delay.shutdown from 20 to 120. The second thing, is it possible to completely disable shutting down the UPS itself? Thanks.

dennypage

@doxymoron The power off delay, ups.delay.shutdown, is a variable stored in the ups. If it is settable in your ups, you would set this via upsrw.

As to disabling shutdown completely, it is possible, however I would strongly recommend against doing that. If you are looking to maximize your runtime, look at adjusting the battery low settings instead. But be careful of going to far... an extra minute of runtime isn't really worth the corruption risk of an ungraceful shutdown.

doxymoron

@dennypage Ok thanks, I will check if those options are available. I have 2 ESXi servers that I have with the Nut client installed and talking to the pfsense package. But when I did a test letting the battery run down I don't think the ESXi servers had enough time to shut down gracefully. Once my UPS hit the low battery threshold I set to 30%, it shut the UPS completely off 20 seconds later. I've been trying to figure out the best way to set this up, and I think I have to increase the shutdown delay. The ESXi servers only get triggered once the low battery threshold is hit I believe, then will probably take at least a minute to shutdown.

doxymoron

Ok I was able to run this command successfully: upsrw -s ups.delay.shutdown=120 -u admin -p **** CyberPower-01@localhost
OK
So am I correct that this means once my battery.charge.low is hit, it will wait 120 seconds and then issue a shutdown command? This shuts down pfsense and then the UPS itself?

dennypage

@doxymoron Unfortunately the term "shutdown" is a little overloaded. Is used to refer to initiating a shutdown of the operating system, and also to refer to sending a command to the ups to kill the mains.

When a low battery condition is determined to exist, nut initiates a shutdown of the operating system. The os shutdown can take seconds or minutes depending upon the operating system and what is running on it. As one of the final steps of the shutdown, the os calls back into nut which issues the kill command to the ups. This is where the delay described by ups.delay.shutdown comes into effect--the ups waits this amount of time before killing the mains. Usually the 20 seconds is sufficient because the os isn't doing much other than syncing disk buffers for a few seconds followed by a halt.

Compared to pfSense, ESXi can take a very long time to shut down. To get a better handle on this, you might want to consider running a nut slave on the ESXi server. Yes, surprisingly there is one available. A quick write up using FreeNAS can be found here. Using a remote client may give you a little more time and flexibility in how the ESXi shutdown works. Note that I do not have direct experience with the ESXi client and cannot answer questions about it, but there may be others here who have experience with it...

The pfSense information needed for enabling remote clients can be found in the second post of this thread.

doxymoron

Thanks so much for helping me, I understand a bit more. I do actually have the ESXi client installed that should properly shut down the host and all VMs. I think the problem I ran into was that from the time the low battery condition hit, to when the Pfsense box shutdown, and then the UPS itself shutdown, was just too fast. The ESXi server did not have time to do any of the proper shutting down before it lost power. I will test tomorrow to see if increasing the ups.delay.shutdown will help. If it delays the shutting down of the UPS, I think it should.

dennypage

@doxymoron You may also wish to look at the HOSTSYNC variable, which would be set in the upsmon.conf section of the advanced settings.

lesodm

Hi, every time I restart my pfsense box, I need to manually run:
/usr/local/sbin/upsdrvctl -u root start
To get the UPS driver to start.
I noticed it has the below in /usr/local/etc/rc.d/nut.sh
/usr/local/sbin/upsdrvctl start & sleep 1
But editing nut.sh just gets reset after a reboot. Any suggestions please?

dennypage

@lesodm Need more information to help.

What type of ups?
What ups settings do you have you in Services / UPS / Settings?
Following reboot, what entries are there in the system log (Status / System Logs / System / General) matching "ups" or "nut"?
How are you determining that you need to run upsdrvctl?

Gertjan

@lesodm said in NUT package:

I noticed it has the below in /usr/local/etc/rc.d/nut.sh

So it's run (started) when you boot.
Check dmesg and other logs why it fails.

edit :
If iI start in manually :

[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/usr/local/pkg/nut: /usr/local/sbin/upsdrvctl start
Network UPS Tools - UPS driver controller 2.7.4
Network UPS Tools - Generic HID driver 0.41 (2.7.4)
USB communication driver 0.33
Duplicate driver instance detected! Terminating other driver!

Broadcast Message from root@pfsense.brit-hotel-fumel.net
        (no tty) at 11:21 CET...

Communications with UPS ups lost

Using subdriver: APC HID 0.96

Broadcast Message from root@pfsense.brit-hotel-fumel.net
        (no tty) at 11:21 CET...

Communications with UPS ups established

which means : it was already running - it switches driver instance, the USB connection got kicked, NUT connects again => all is well.

lesodm

@dennypage,
*Local USB
*Settings: Driver blazer and listen port settings in upsd.conf and user settings in upsd.users
*There is this error every 5 seconds in the system log:
41044 Poll UPS [UPS] failed - Driver not connected
*UPS connects and reports all info back once I run upsdrvctl

@Gertjan there is a ton of these messages in dmesg:
<INNO TECH USB to Serial> at usbus1 (disconnected)

dennypage

@lesodm You are using a USB to serial adapter? Please post the output of "usbconfig dump_device_desc"

Please post the actual settings you have in the pfSense GUI (Services / UPS / Settings). Either a screen shot or the XML data from the config.

Please post the log messages that match any of these strings, "usb", "ups", or "nut" for the 2 minutes following reboot.

lesodm

@dennypage it's a USB A to B printer cable.
I might have resolved the issue by adding maxretry = 3 to ups.conf. After last nights power outage the UPS reconnected itself. However I see in syslog it reconnects every now and then, something still not 100% then maybe?
"usbconfig dump_device_desc": usbconfig dump.txt
Status/SystemLogs/System/General messages that match any of these strings, "usb", "ups", or "nut" following reboot: Syslog-USB.txt Syslog-UPS.txt Syslog-nut.txt
Actual settings:

doxymoron

@dennypage said in NUT package:

@doxymoron You may also wish to look at the HOSTSYNC variable, which would be set in the upsmon.conf section of the advanced settings.

Thanks for your help. I was able to get this working by just giving my 2 ESXi servers 2 minutes to shutdown before powering off the UPS using the override.ups.delay.shutdown option. I do have another question. If I were to get another UPS, am I able to also connect it via USB to my pfsense box and also monitor it? Thanks.

dennypage

@lesodm [Sorry for the delay, I'm traveling]

Okay, several things now come to mind. Firstly, I'm going to assume that the usb to serial converter is imbedded in the ups itself and not something that you have separately installed. Btw, you didn't say what the manufacturer/model of the ups was.

First, please ensure you can get into the box via ssh because I want you to run the following tests without anything but the ups connected via usb. In other words, no usb keyboard, no usb mouse. I'm also assuming that at this point you are not using an external usb hub. Please confirm.

Disable nut. Go into Services / UPS, select Disabled from the UPS Type drop down list, and press save.
Reboot pfSense.
Log in via ssh.
Run "usbconfig dump_device_desc" and save the output.
Run "ls -l /dev/cua*"
Move the usb connection to a different port on the motherboard.
Repeat steps 4-6 until you have tested all of the usb ports. Keep track of which port goes with which output.

Please post the results.

dennypage

@doxymoron said in NUT package:

Thanks for your help.

You're quite welcome.

I do have another question. If I were to get another UPS, am I able to also connect it via USB to my pfsense box and also monitor it?

While nut itself has that capability, the pfSense gui does not. Were you thinking of a second ups that would provide power to the pfSense box via a redundant power supply? Or were you thinking of a second ups that provides power to another system (and are just looking for a way to monitor it)? If the latter, it's better to monitor it using the system whose power the ups controls. If that isn't available, you could also monitor it using nut in a lightweight guest running under the ESXi server.

doxymoron

@dennypage I was looking to separate the load and add a second UPS for my second ESXi server and network equipment. I will look into running another nut server inside a VM on that ESXi server, that seems like the best way. Thanks.

lesodm

@dennypage
Hi, thanks for getting back to me. Only now been able to reboot.
Its a Tescom Otima Plus 3000 NM UPS.
No USB hub, though keyboard and mouse is connected via a KVM switch but I unplugged this before rebooting and testing.
Results for the 6 ports running "usbconfig dump_device_desc": port6.txt port5.txt port4.txt port3.txt port2.txt port1.txt
Ports 5 and 6 where through a PCI USB card
Running "ls -l /dev/cua*" has no results.