pfSense -> pfSense NUT connection issues

Gertjan

Your first pfSense, connected to the UPS with an USB cable, you have this :

?

On the second pfSense :

and you have create on the first pfSense this :

edit :

upsmon on the main, first pfSEnse, as a NUT server, listens to 'localhost' by default.

I presume your second pfSense is hooked up to the LAN of the first pfSense.
Add this NAT rule on the first pfSense (LAN) :

and from now on, incoming TCP traffic with destination port 3493 (default NUT server port) will be redirected to the internal 127.0.0.1 or localhost on that pfSense, where the NUT server will receive the requests from the client, second NUT pfSense.
I know that works, as I'm using it like this.

AngryAnt

@Gertjan Thank you for the detailed overview. UPS status on the first pfSense looks slightly different, but the message is the same:

The port forward looked a little different, but adjusting it to match yours had no effect:

My inbound firewall rules (two rules for two allowed hosts - the second pfSense box and a Synology box which also cannot connect) still indicate that lots of traffic is allowed through, but then somehow fails after the firewall (pfSense box 2 rule highlighted):

The Synology box has its "special" user credentials added to upsd.users via the GUI alongside the credentials for the second pfSense box, but while the incoming firewall rule for it does indeed also look to be correctly matched, it too fails to connect:

I did also try having only one set of credentials set, with no impact. Your other setup and assumptions are a match.

dennypage

@AngryAnt [FWIW, you really don't need to block out addresses when you post local firewall rules. There aren't any secrets there, especially when the addresses appear in the log entries you are posting anyway.]

Please post the entirety of your NUT configuration for the sever side. Be sure to include the entirety of the advanced section without masking. You can use single word temporary passwords for the time being, and change the passwords after it's working.

Also, please post the complete firewall and NAT rules governing access to the NUT server, without masking.

AngryAnt

@dennypage Thank you for the follow-up :)

The NUT server config is as follows:

And this is the firewall on that same host (192.168.1.1 - with the UPS attached and recognized via USB):

dennypage

@AngryAnt said in [pfSense -> pfSense NUT connection

Items unrelated to your problem:

You have "user = root" set in the optional arguments to driver. This is probably unnecessary and should be removed unless you have really good reason for it like a missing quirk. If the issue is a missing quirk, you should add the missing quirk instead.
You have NOTIFYCMD set in Additional configuration for upsmon.conf. Unless you are really expert with NUT configuration, this is probably a bad idea.

Items related to your problem:

You do not want the two lines granting access to This Firewall (self) on port 3493. These firewall rules would be used for the "option 2" approach to remote NUT access instead of the "option 1" approach you are taking. [Edit: note also that the destination address for these firewall rules is wrong for the "option 1" approach -- it would have needed to be 127.0.0.1]
In the NAT entry for 3493, you have the Filter rule association enabled, which produces the last line of your LAN firewall rules. Given that you have a firewall rule that allows LAN subnets, which presumably includes 192.168.1.0/24, to go anywhere, this is unnecessary. You can/should set the Filter rule association to "none". NB: The source of LAN subnets in the firewall rules could simply be changed to Any unless there is some mechanism by which packets that do not originate from the LAN can appear on the LAN interface.
You appear to have a block on 192.168.1.3 to RFC1918 addresses in your firewall rules. Given that your firewall is inside RFC1918 address space, this is a problem not only for NUT but also for DNS/NTP, etc.
Do you have anything in Floating rules that might be in override the rules for LAN?
I recommend using upsc to test connectivity prior to attempting use NUT remote access. So from a shell on the remote systems use the command "upsc ups@192.168.1.1".
Have you looked at the firewall log for entries relating to port 3493?

Gertjan

@AngryAnt

Added to what @dennypage said :

This one (green arrow) :

If you leave it empty, then the file
/usr/local/etc/nut/upsd.conf will contain :

LISTEN 127.0.0.1
LISTEN ::1

so the upsd process listen only to localhost.
Not 192.168.1.1 (pfSense) or anything else.
That's why the "LAN to 1278.0.0.1 TCP port 3493 NAT rule" was mentioned / intended.

Right now, your UPS clients on the LAN can talk to what they want to on port 192.168.1.1 port 3493 TCP but no server is listening on that port.

See : [see the upsd.conf manual page](see the upsd.conf manual page.).

You could place this in the /usr/local/etc/nut/upsd.conf

LISTEN 127.0.0.1
LISTEN ::1
LISTEN 192.168.1.1

and from now on, the upsd will listen also on your LAN IPv4.

As sais above, I didn't do that, I added a NUT NAR rule, that redirect to 127.0.0.1:3493.
Let's fact-check : ask pfSense :

[25.03-BETA][root@pfSense.bhf.tld]/root: sockstat -4 | grep 'upsd'
root     upsd       80869 4   tcp4   127.0.0.1:3493        *:*
root     upsd       80869 8   tcp4   127.0.0.1:3493        192.168.1.33:58575
root     upsd       80869 9   tcp4   127.0.0.1:3493        192.168.1.6:56464
root     upsd       80869 10  tcp4   127.0.0.1:3493        192.168.1.7:55913

and there you see that there are 3 of my LAN devices connected to the pfSense UPS server.

edit :

About :

you only use NOTIFYCMD /usr/local/sbin/upssched if your totally NUTs.
Using "upssched" means you have to actually read the NUT manual, look up many examples, take your time to simulate a power down, and check regularly if everything still works as planned.
Using "upssched" also means you have to edit config files that are not part of the pfSense GUI, so when updating / upgrading / re installing you have to edit these check and / or edit these files again.

AngryAnt

@dennypage Thank you for all of the suggestions :)

I removed the unrelated observed issues with no negative impact. Nicely spotted.

Not following your correction to the NAT rule. Surely the destination matching should be the pfSense LAN address (192.168.1.1) and then the NAT IP (where the package is redirected to) should be 127.0.0.1 where NUT listens by default? Or am I completely misunderstanding psSense port forwarding here?

Good spot on the redundant auto-added allow rule. Setting to "none" and removing it had no negative impact.

The block rule on 192.168.1.3 is intentionally placed after the NUT allow rules. DNS/NTP I have 192.168.1.3 use external IPs for, so blocking them on LAN addresses has not had negative impact.

No floating rules.

Thank you for the tip on using upsc. I added my admin terminal to the allow list as well and could use it for much better iteration time.

Only firewall log entry related to 3493 is from a port scan on WAN by known-bad actor according to https://www.abuseipdb.com

AngryAnt

@Gertjan Thank you for the detailed follow-up!

Also: Progress! I am unsure why my previous attempt at having LISTEN 192.168.1.1 in there was unsuccessful, but now it lets me successfully connect from all clients.

By my understanding this still is effectively a bypass of the firewall, so I would really prefer to have NUT only listen on 127.0.0.1 and then find my mistake in the NAT configuration. But obviously very reassuring to see real output on upsc and both clients.

AngryAnt

For reference, I have NAT & firewall rules configured similarly on the second pfSense to allow select clients on there access to the NUT server on the first pfSense via their local access to the second pfSense. So my NAT issue seems specific to trying to port forward to the localhost address rather than being more general.

Gertjan

@AngryAnt said in pfSense -> pfSense NUT connection issues:

Surely the destination matching should be the pfSense LAN address (192.168.1.1) and then the NAT IP (where the package is redirected to) should be 127.0.0.1 where NUT listens by default?

You saw it :

@AngryAnt said in pfSense -> pfSense NUT connection issues:

Only firewall log entry related to 3493 is from a port scan on WAN by known-bad actor according to https://www.abuseipdb.com

Normally, you don't place any rules on WAN.
The default firewall for every interface, including WAN is : "silently drop". This is not a pfSense behavior, every firewall on planet earth does this.
Don't ( ^^ ) have this drop rule log, as you'll be smacked with firewall log notification.

So, normally, un check this :

@AngryAnt said in pfSense -> pfSense NUT connection issues:

Also: Progress! I am unsure why my previous attempt at having LISTEN 192.168.1.1 in there was unsuccessful

A service needs to listen on an interface, and a port, using UDP and/or TCP.
and
You need on that interface (LAN) a firewall rule that allows that traffic to enter.
For your pfSense LAN, there is/was an install pass-all rule, so as soon as upsd was listening on 192.168.1.1, it would have worked.
That is : you also need to set up user auth.

AngryAnt

@Gertjan Thank you for the overview :)

My port forward does indeed match yours.

No surprises in the overview of socket mechanisms. I also confirmed that listening on 192.168.1.1 does indeed bypass the firewall. After disabling the two NUT-specific allow rules and replacing the two allow-all rules at the bottom of the list with an originally intended allow-outgoing rule at the top of the list:

... all connections to the NUT port are still successful - leading me to conclude that having NUT listening directly on 192.168.1.1 does as I suspected unintentionally bypass the firewall.

I would prefer to:

Only have NUT listen on 127.0.0.1.
Have port forwarding handle mapping to 127.0.0.1 from 192.168.1.1, thus looping in the firewall rules on the LAN port (192.168.1.1).
Explicitly allow access to the NUT port on LAN on a per-host basis.

Thanks again for all of your help.

dennypage

@AngryAnt said in pfSense -> pfSense NUT connection issues:

Not following your correction to the NAT rule. Surely the destination matching should be the pfSense LAN address (192.168.1.1) and then the NAT IP (where the package is redirected to) should be 127.0.0.1 where NUT listens by default? Or am I completely misunderstanding psSense port forwarding here?

No, when you NAT the destination of the [firewall] rule should be for the forward address, not the firewall interface address. 127.0.0.1 is the correct destination address [for the firewall rule].

Please see the instructions for "option 1" here.

AngryAnt

@dennypage I'm sorry but I seem to be misunderstanding something core here. Looking at the option 1 instructions from your link, all of those match my forward point for point. I originally used those exact instructions to create the forward:

(Note that I reintroduced the filter rule association as I moved away from the allow-all setup as outlined above)

dennypage

@AngryAnt said in pfSense -> pfSense NUT connection issues:

By my understanding this still is effectively a bypass of the firewall,

No, it's not bypassing the firewall. You aren't reposting your rules with every change, so we can't explain each success/failure, but I assure you nothing escapes the firewall rules.

FWIW, I'm not sure exactly what you are really trying to do with the RFC1918 alias, but you may want to look at "Local networks" instead. And perhaps the Local tab on the firewall rules page. These are often overlooked simplifiers.

dennypage

@AngryAnt The correction was for the destination of your firewall rule, not the destination of your NAT rule. Sorry this was ambiguous.

AngryAnt

@dennypage Apologies - an obvious omission when juggling changes here. The updates are as follows - with host & port aliases added for better referencing and typo avoidance on my part:

dennypage

@AngryAnt said in pfSense -> pfSense NUT connection issues:

@dennypage Apologies - an obvious omission when juggling changes here. The updates are as follows - with host & port aliases added for better referencing and typo avoidance on my part:

Assuming that you still have the NAT rule in place...

Okay, I think I may see the confusion. It's important to realize that the NAT packets do not pass through the firewall twice, only once, and that NAT is applied before the firewall are processed.

So, looking at your rules, the rules that have source "Nilt", "Dejima", "Haro", "Toulon" are meaningless because they have the wrong Destination address. These are the rules referred to earlier that would need to have a destination address of 127.0.0.1. As it sits, these rules will have no effect because of the NATing.

However, it won't really matter because below you have a rule that allows everyone access to 127.0.0.1.

With this in mind, it may help to re-read items 1 & 2 of the "Items related to your problem" list that I wrote above. I added a couple of words above to make it more clear.

AngryAnt

@dennypage said in pfSense -> pfSense NUT connection issues:

Okay, I think I may see the confusion. It's important to realize that the NAT packets do not pass through the firewall twice, only once, and that NAT is applied before the firewall are processed.

Bingo! This is exactly the piece of the puzzle missing from my understanding. So with that in mind and given the port forward above, what should give me the result I am looking for is a ruleset like this?

dennypage

@AngryAnt said in pfSense -> pfSense NUT connection issues:

@dennypage said in pfSense -> pfSense NUT connection issues:

Okay, I think I may see the confusion. It's important to realize that the NAT packets do not pass through the firewall twice, only once, and that NAT is applied before the firewall are processed.

Bingo! This is exactly the piece of the puzzle missing from my understanding. So with that in mind and given the port forward above, what should give me the result I am looking for is a ruleset like this?

Close.

If I understand what you are trying to do...

What I think you want is this (in order, following the Anti-lockout rule):

Allow Nilt/Dejima/Haro/Toulon to 127.0.0.1:nut. Just as you have the rules currently.
Rules to drop some of the Dejima packets without logging. FWIW, I would recommend this be done on an individual port basis to avoid issues down the road.
Allow * to ! Local networks. NB: "Local networks" is a pfSense provided system alias, found at the bottom of the destination selection list.

In addition, you will want rules to allow access to the firewall for things like DNS and NTP. The Local tab is a good place for this.

AngryAnt

@dennypage Your interpretation of my intent is exact. Thank you for the suggested improvements :)

Before I touch anything else here: In combination with the port forward, the given hosts should through these rules have access to the NUT server when it is only set to LISTEN 127.0.0.1 either explicitly or reliant on that default?