Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface

phil80

@phil80
Gemini even adds more questioning about how pfsense under the hood is associating origin in states to check a packet

Is this expected and undocumented or a bug?

johnpoz

@phil80 could give 2 shits what some LLM hallucinated.

You accessing a router on your wan in rfc1918 space would be allowed out of the box - there is nothing to do.

Out of the box inbound rfc1918 traffic is blocked - this has nothing to do with return traffic you started from your lan..

The default lan rule is any any.

Not sure what is the point in creating all these rules? What exactly are you trying to stop from happening?

phil80

@johnpoz

My described issue is not related to the LAN or WAN rules anyway
The problem can be resumed to

adding the recommended floating rule to "Block private networks traffic from leaving the WAN interface"
while keep access to ISP modem on WAN

The recommended way is to add a floating rule, direction out, interface WAN, before the block rule, and that passes traffic to modem IP on WAN

I noticed that the pass floating rule must apply to both WAN and LAN interfaces

In above pic, the floating rule allows access to modem from LAN and VLAN10

I'm just trying to undestand why and the networking logic behind it, for educational reasons. I'm not looking at: is it really useful, that issue is so rare and unprobable...

Do you have an explanation of this ?

phil80

If you look at it further, it is really strange
The floating rule has the proper source WAN address expected after NAT
The logs properly show the source address for requests is WAN address 192.168.1.2
However, the flosting rule must be applied to the originating LAN in addition to WAN

phil80

@phil80
And here's the only relevant part I could find in netgate docs. It clearly states that for outbound, NAT is done before and when the packet hits WAN, it is already natted. That's what exactly shows in logs. Still, having to apply the rule to teh originating LAN interface is unexplained. Do floating rules track the originating state ?

https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html

Processing Order¶
In the inbound direction, floating rules work essentially the same as interface or group rules except that they are processed first. Processing in the outbound direction is more complicated.

The firewall processes floating rules after NAT rules, so rules in the outbound direction on a WAN can never match a private IP address source if the firewall also applies outbound NAT to connections on that interface. By the time a packet hits the floating rule, the source address of the packet is the post-NAT WAN IP address. In most cases this limitation can be overcome by applying a tag to a packet inbound on the LAN and then matching that tag in an outbound floating rule (Marking and Matching).

The firewall processes floating rules before interface group rules and interface rules, so that must also be taken into consideration.

phil80

@johnpoz
By the way, I followed this official guide to add the flaoting rule:
https://docs.netgate.com/pfsense/en/latest/recipes/rfc1918-egress.html

And there is this note:

Notes¶
Adding this rule to the firewall will block access to bridge devices like cable modems or upstream routers outside of the WAN interface. For example, many cable modems use an IP address of 192.168.100.1 by default. This may or may not be desirable behavior for users. The RFC 1918 firewall rule needs to be bypassed with a specific pass rule above it, or disabled, if clients inside the LAN require access to this type of device.

On the interface options (Interfaces > WAN, for example) there is an option to Block private networks. This is a rule blocking inbound traffic, not outbound like the rule described here. As long as the firewall is not behind a WAN that uses private addressing, both types of rules are desirable and should be enabled.[/quote]

Now, to allow access to the modem on WAN, we need the above floating rule

Why the hell it needs to be applied to LAN in addition to WAN ?
Unless pfsense needs to track the connection with all its states through all its lifecycle accross all interfaces to properly allow it. In that case, I can understand the security reasoning. Still the doc part I linked doesn't explain it

johnpoz

@phil80 Where does it say to create an outbound rule on your lan side interfaces. The rules on the lan wouldn't be outbound..

You have some client on your lan - he wants to talk to something off his network - that would be INBOUND into the lan interface - not outbound.

And that guide also sure doesn't say use a reject rule..

I allow access to my modem 192.168.100.1 address, there is a bit you need on the rule to allow when its not your normal wan, ip range. I am using a vip on my wan in the 192.168.100 range - so need to set the do not generate reply-to on the allow rule.

But you shouldn't have to do that when your not using a vip on your wan, and its just a normal IP of the wan interface.

phil80

@johnpoz
Who said I am applying the rule on lan?
I'm aware interface rules are ingress only, sure.

Here's screenshot from netgate recommendation illustrating the floating rule on WAN

Now, with that rule, as the doc says, you need an allow rule before it. The thing is that the allow rule must apply to all interfaces the filtered packet travels through and not only Wan
I suppose it is a strict security that the stateful inspection needs to track the packet through all its lifespan

johnpoz

@phil80 said in Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface:

Who said I am applying the rule on lan?

so lan_vlan828 is a WAN interface? Odd name for a wan if you ask me. And they are not using rfc1918? And you don't want to talk to them?

I personally think putting reject there is bad example on when to use it - rejecting on a wan interface, users get the wrong idea from that. But since it is a outbound rule it would just be sending the reject back to pfsense itself or the lan side that tried to go there - but users might take that its ok to use reject.. They should put in a note on when and when not to use reject. Because you almost never would want a reject on a wan interface.

phil80

@johnpoz
Seems you just didn't get me :-( or I wasn't clear

1- I followed the netgate recommendation guide I linked above to add the floating outbound rule on WAN to "block private traffic from leaving the WAN"
2- Now, to allow traffic from VLAN10 and VLAN828 to the ISP modem, liek the guide hinted, I added a floating pass rule applying to WAN interface.
3- The Problem: Unless the pass rule is applied in direction Out, on the originating LAN (VLAN10 or VLAN828), in addition to WAN, the traffic to modem from theses VLANs is blocked by the block rule we added.

-> From my readings, I saw some opaque explanations that a stateful firewall needs to follow the packets through all their life cycle (complete state) before allowing them in a floating out rule. They say many firewally act like this.

I just didn't find any official documentation or logical explanation or statement

As you maybe agree, the pass rule, applied to WAN interface, is what we logically expect. The logs below also show that the final decison on pass rule is made on WAN (and not originating VLAN) and that the source is 192.168.1.2 (so, VLAN828 client natted to WAN address as expected)

If I remove VLANs from the floating out rule interfaces, I get this log from the netgate applied block rule:

As you can see, it seems to block the return traffic from Modem to LAN and not when it went out as I expected source 192.168.1.2 -> destination 192.168.1.1

I am just asking if it is really the tricky part about Out rules AND expected
Or if it is a bug

johnpoz

@phil80 well your 2nd block is a SA to some 10.x address.. Why are you hiding some 10. address? What is that 10.x - Since it is syn ack coming from your modem at 192.168.1.1 - how would that possible be outbound your wan? That is not your pfsense IP address that is 192.168.1.1 sending a SA to whatever 10.x is?

Seems like you have a real mess.. It would not be possible to see SA from some other device on pfsense as outbound on its wan interface.

Is this 192.168.1.2 IP of pfsense a vip or the actual native IP of pfsense wan. What is this 10.x, where is it? Are you bouncing off some proxy you running on pfsense, you have some port forward setup?

If I talk to some 192.168.1.1 IP from pfsense wan at 192.168.1.2, how would the SA be trying to go to some 10.x address out your wan?

my wan is 209.x.x.x (public ip) to reach my modem management IP on 192.168.100.1, I setup a vip on my wan interface.. Traffic going to my modem is allowed, see the rules I posted above from my floating. I do this because I block noise going out my wan to rfc1918 - being a good netizen - there is no rfc1918 I would ever want to reach, or even could reach other than that 192.168.100.1 IP of my modem.

If say I typo something and try and go to 10.1.2.3 then yes my wan outbound would block that since I don't have any 10.x.x.x network behind pfsense - so it would send it out its default gateway (my isp) via a public IP.

The only thing I had to do is set that do not create reply-to on my allow rule.. And I can talk to my modem management IP just fine from my pc on my lan of 192.168.9.100.. No other rules are needed.. If I sniff on my wan while talking to my modem IP it shows traffic between 192.168.100.1 and my wan vip of 192.168.100.2

But your modem trying to send a SA to 10.x shouldn't ever be outbound out your wan. I don't even know how that is possible for pfsense to see that traffic.. Unless your 192.168.1.1 gateway to try and get to 10.x is through pfsense and pfsense is then routing that traffic out its wan to get to 10.x - which makes no sense!!

phil80

@johnpoz
Ok
WAN interface is 192.168.1.2/24
pfsense is in the DMZ of the ISP router which is 192.168.1.1/24
pfsense WAN gateway is 192.168.1.1/24

That's the only possibility with my ISP router

VLAN 10 is 10.90.10.1/24
Client connecting to the modem in the logs is 10.90.10.30, gateway is pfsense 10.90.10.1
Floating tab has only the rules you see + match rule for VPN shaping
VLAN10 has just a debug allow all rule

Nothing fancy in my setup

johnpoz

@phil80 so your modem is sending its SA back to pfsense to the 10 address of the device wanting to talk to it from your vlan 10.

How modem even know this address? Unless it wasn't natted to your wan IP? But that wouldn't be outbound your wan interface?

You mention vpn?? How is that setup?

I have a VM pfsense behind my pfsense that is sim setup where my 2.8 pfsense wan is 192.168.3. and its gateway is my normal pfsense is its gateway 192.168.3.253..

Let me test with that doing the block outbound rfc1918..

edit: is this vlan10 on the same interrface as your wan?

phil80

@johnpoz
No, wan and vlan10 are on separate lan cards.
NAT works as the pass out rule log I posted shows

VPN has nothing to do with lans. It is for outside internet access and not used here
Pfsense is on dedicated hardware

I initially thought the logs were showing the states origin. But in any case, there's something wrong and I would post an issue on redmi if relevant

johnpoz

@phil80 so I tested this on my 2.8 VM behind my normal pfsense.

internet - pfsense 24.11 -- 192.168.3.253 -- 192.168.3.109 -- pfsense 2.8 -- 192.168.9.34 -- 192.168.9.30 client

And did not work - I had to set this for it to work.

Without that set - could not access my upstream pfsense on 192.168.3.253.. With it set no problem.

We ran into this awhile ago - thought it was related to using a vip. But it is related to native as well it seems.

If you set that on your allow rule above your deny rfc1918 it should work. Kept meaning to do a deep dive on what exactly that setting is doing that would prevent this sort of thing.. But haven't gotten around to it, there are some older threads about this. And if you will notice I asked before if you had that set or not..

https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#disable-reply-to

I could see how a vip or vpn could be seen as another wan and cause problems.. But in this test I did the only wan interface is pfsense normal wan with native 192.168.3.109 IP.. So I am not clear on why the reply-to causes a problem here.. But it does - if you disable it on your allow rule to your upstream router from pfsense you should work. It does in my test, and it does with my vip access to my modems management 192.168.100.1 IP.

phil80

@johnpoz
Thank you for testing
In fact, I just tried it. It works after applying disable reply-to to the pass rule and after setting it to only wan interface

The logs also show a proper traffic source-direction for the pass rule

So, it seems like disabling the reply-to has the exact same result as applying the rule to WAN AND VLan10 interface
Stil I cannot follow the routing logic behind it

phil80

@johnpoz
I know you don't give a s*** about AI and you're right about it. It can lie and if you're not an expert in the domain, you'd better don't rely on it. However, once you take it into account, it can help hinting the answer.

Here's AI thoughts about your fix, that is also in fact recommend in the netgate guide you linked when behind a bridged interface.

The Key Concept: reply-to Behavior
By default, pfSense applies a reply-to directive on rules for WAN interfaces, which forces return traffic to go back out the same interface it came in.

This behavior is designed for asymmetric routing scenarios, where multiple WANs might be used (like dual ISPs). But in your setup, it can interfere with legitimate return paths when pfSense is routing between internal networks (like LAN → WAN in a double-NAT setup).

️ What Happens Without Disabling reply-to:
A LAN client connects to 192.168.1.1.

pfSense forwards the packet out WAN interface.

The floating pass rule allows it.

However, due to reply-to being enabled by default on WAN rules, pfSense expects the reply to come from WAN and go back out WAN to the original sender, which is incorrect here — the reply needs to go back to LAN.

So return traffic gets misrouted or dropped, unless another rule compensates.

When You Disable reply-to on WAN Rule:
Disabling reply-to allows pfSense to use its routing table to determine the return path instead of forcing it out the WAN. That allows replies to go back correctly to LAN clients.

Hence, you no longer need to apply the floating rule to LAN — WAN is enough — because routing is no longer forced in a specific (incorrect) direction.

So: Which is the More Valid Approach?
Best practice in this case:
Use the floating rule on WAN only, and disable reply-to.

Why?
It’s more precise: you're matching traffic only where it's actually leaving (WAN).

You avoid unnecessary application on LAN.

You preserve pfSense's proper routing behavior, without the quirks of forced reply paths.

This aligns better with pfSense firewall design philosophy, where you disable reply-to when handling internal or non-public traffic on WAN interfaces (common in double NAT or transit VLAN setups).

This time, unlike the first explanation I found unclear and motivated me to post here, it looks logic and corresponds to the netgate guide you linked to about disabling repy-to behind a bridged interface

Thank you again for the support and proper fix you provided

PS: AI is really some bullshit for expert analysis as it just lies when it fails

johnpoz

@phil80 said in Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface:

PS: AI is really some bullshit for expert analysis as it just lies when it fails

yeah its horrible - since it provides wrong info for shit I actually know, how can I trust it asking it stuff I don't.. I filtered out the AI shit google provides when you google something.. Because it was always freaking wrong when I looked into their answers.

I wouldn't trust anything it spews out.. Might give you some clues to where to actually start looking.. Ok - and it might be ok at writing code or something.. But if you ask it an actual tech question.. Every single time I have asked it something it was BS..

might be ok if you ask it to create a summary of a wall of text?

I did a bit more digging from links in an older thread when reply-to came up.. I found one comment that said pfsense and the other option I will not name ;) are doing it wrong - heheh.. That it really shouldn't be on by default. But I have never ran into any issues other then doing this sort of thing.

phil80

@johnpoz
Thank you, i fixed my rules. I didn't suspect that not disabling reply-to at the end caused such wrong routing issues and that my fix was just a workaround maintaining the broken routing

phil80

@johnpoz

I pushed further and added these rules to WAN

Why I can still connect to the modem ?
I thought that implicit allow rules can be blocked by explicit block rules on an interface
By implicit, I supposed the stateful reply traffic is part of it
In that case, I expected the reply traffic from modem would be blocked, but it is not

Now after reading, I see that replies are always allowed unless blocked by a floating rule or by disabling the state. So the rule on wan would only apply to new connections. Since any incoming new connection is blocked by default, what's the use of the automatic rule "Block private networks and loopback addresses" (I just manually added it after allowing ping requests from modem as my modem seems to require these pings even if set as static ip)