Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface
-
@johnpoz
Who said I am applying the rule on lan?
I'm aware interface rules are ingress only, sure.Here's screenshot from netgate recommendation illustrating the floating rule on WAN
Now, with that rule, as the doc says, you need an allow rule before it. The thing is that the allow rule must apply to all interfaces the filtered packet travels through and not only Wan
I suppose it is a strict security that the stateful inspection needs to track the packet through all its lifespan -
@phil80 said in Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface:
Who said I am applying the rule on lan?
so lan_vlan828 is a WAN interface? Odd name for a wan if you ask me. And they are not using rfc1918? And you don't want to talk to them?
I personally think putting reject there is bad example on when to use it - rejecting on a wan interface, users get the wrong idea from that. But since it is a outbound rule it would just be sending the reject back to pfsense itself or the lan side that tried to go there - but users might take that its ok to use reject.. They should put in a note on when and when not to use reject. Because you almost never would want a reject on a wan interface.
-
@johnpoz
Seems you just didn't get me :-( or I wasn't clear1- I followed the netgate recommendation guide I linked above to add the floating outbound rule on WAN to "block private traffic from leaving the WAN"
2- Now, to allow traffic from VLAN10 and VLAN828 to the ISP modem, liek the guide hinted, I added a floating pass rule applying to WAN interface.
3- The Problem: Unless the pass rule is applied in direction Out, on the originating LAN (VLAN10 or VLAN828), in addition to WAN, the traffic to modem from theses VLANs is blocked by the block rule we added.-> From my readings, I saw some opaque explanations that a stateful firewall needs to follow the packets through all their life cycle (complete state) before allowing them in a floating out rule. They say many firewally act like this.
I just didn't find any official documentation or logical explanation or statement
As you maybe agree, the pass rule, applied to WAN interface, is what we logically expect. The logs below also show that the final decison on pass rule is made on WAN (and not originating VLAN) and that the source is 192.168.1.2 (so, VLAN828 client natted to WAN address as expected)
If I remove VLANs from the floating out rule interfaces, I get this log from the netgate applied block rule:
As you can see, it seems to block the return traffic from Modem to LAN and not when it went out as I expected source 192.168.1.2 -> destination 192.168.1.1
I am just asking if it is really the tricky part about Out rules AND expected
Or if it is a bug -
@phil80 well your 2nd block is a SA to some 10.x address.. Why are you hiding some 10. address? What is that 10.x - Since it is syn ack coming from your modem at 192.168.1.1 - how would that possible be outbound your wan? That is not your pfsense IP address that is 192.168.1.1 sending a SA to whatever 10.x is?
Seems like you have a real mess.. It would not be possible to see SA from some other device on pfsense as outbound on its wan interface.
Is this 192.168.1.2 IP of pfsense a vip or the actual native IP of pfsense wan. What is this 10.x, where is it? Are you bouncing off some proxy you running on pfsense, you have some port forward setup?
If I talk to some 192.168.1.1 IP from pfsense wan at 192.168.1.2, how would the SA be trying to go to some 10.x address out your wan?
my wan is 209.x.x.x (public ip) to reach my modem management IP on 192.168.100.1, I setup a vip on my wan interface.. Traffic going to my modem is allowed, see the rules I posted above from my floating. I do this because I block noise going out my wan to rfc1918 - being a good netizen - there is no rfc1918 I would ever want to reach, or even could reach other than that 192.168.100.1 IP of my modem.
If say I typo something and try and go to 10.1.2.3 then yes my wan outbound would block that since I don't have any 10.x.x.x network behind pfsense - so it would send it out its default gateway (my isp) via a public IP.
The only thing I had to do is set that do not create reply-to on my allow rule.. And I can talk to my modem management IP just fine from my pc on my lan of 192.168.9.100.. No other rules are needed.. If I sniff on my wan while talking to my modem IP it shows traffic between 192.168.100.1 and my wan vip of 192.168.100.2
But your modem trying to send a SA to 10.x shouldn't ever be outbound out your wan. I don't even know how that is possible for pfsense to see that traffic.. Unless your 192.168.1.1 gateway to try and get to 10.x is through pfsense and pfsense is then routing that traffic out its wan to get to 10.x - which makes no sense!!
-
@johnpoz
Ok
WAN interface is 192.168.1.2/24
pfsense is in the DMZ of the ISP router which is 192.168.1.1/24
pfsense WAN gateway is 192.168.1.1/24That's the only possibility with my ISP router
VLAN 10 is 10.90.10.1/24
Client connecting to the modem in the logs is 10.90.10.30, gateway is pfsense 10.90.10.1
Floating tab has only the rules you see + match rule for VPN shaping
VLAN10 has just a debug allow all ruleNothing fancy in my setup
-
@phil80 so your modem is sending its SA back to pfsense to the 10 address of the device wanting to talk to it from your vlan 10.
How modem even know this address? Unless it wasn't natted to your wan IP? But that wouldn't be outbound your wan interface?
You mention vpn?? How is that setup?
I have a VM pfsense behind my pfsense that is sim setup where my 2.8 pfsense wan is 192.168.3. and its gateway is my normal pfsense is its gateway 192.168.3.253..
Let me test with that doing the block outbound rfc1918..
edit: is this vlan10 on the same interrface as your wan?
-
@johnpoz
No, wan and vlan10 are on separate lan cards.
NAT works as the pass out rule log I posted showsVPN has nothing to do with lans. It is for outside internet access and not used here
Pfsense is on dedicated hardwareI initially thought the logs were showing the states origin. But in any case, there's something wrong and I would post an issue on redmi if relevant
-
@phil80 so I tested this on my 2.8 VM behind my normal pfsense.
internet - pfsense 24.11 -- 192.168.3.253 -- 192.168.3.109 -- pfsense 2.8 -- 192.168.9.34 -- 192.168.9.30 client
And did not work - I had to set this for it to work.
Without that set - could not access my upstream pfsense on 192.168.3.253.. With it set no problem.
We ran into this awhile ago - thought it was related to using a vip. But it is related to native as well it seems.
If you set that on your allow rule above your deny rfc1918 it should work. Kept meaning to do a deep dive on what exactly that setting is doing that would prevent this sort of thing.. But haven't gotten around to it, there are some older threads about this. And if you will notice I asked before if you had that set or not..
https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#disable-reply-to
I could see how a vip or vpn could be seen as another wan and cause problems.. But in this test I did the only wan interface is pfsense normal wan with native 192.168.3.109 IP.. So I am not clear on why the reply-to causes a problem here.. But it does - if you disable it on your allow rule to your upstream router from pfsense you should work. It does in my test, and it does with my vip access to my modems management 192.168.100.1 IP.
-
@johnpoz
Thank you for testing
In fact, I just tried it. It works after applying disable reply-to to the pass rule and after setting it to only wan interfaceThe logs also show a proper traffic source-direction for the pass rule
So, it seems like disabling the reply-to has the exact same result as applying the rule to WAN AND VLan10 interface
Stil I cannot follow the routing logic behind it -
@johnpoz
I know you don't give a s*** about AI and you're right about it. It can lie and if you're not an expert in the domain, you'd better don't rely on it. However, once you take it into account, it can help hinting the answer.Here's AI thoughts about your fix, that is also in fact recommend in the netgate guide you linked when behind a bridged interface.
The Key Concept: reply-to Behavior
By default, pfSense applies a reply-to directive on rules for WAN interfaces, which forces return traffic to go back out the same interface it came in.This behavior is designed for asymmetric routing scenarios, where multiple WANs might be used (like dual ISPs). But in your setup, it can interfere with legitimate return paths when pfSense is routing between internal networks (like LAN → WAN in a double-NAT setup).
️ What Happens Without Disabling reply-to:
A LAN client connects to 192.168.1.1.pfSense forwards the packet out WAN interface.
The floating pass rule allows it.
However, due to reply-to being enabled by default on WAN rules, pfSense expects the reply to come from WAN and go back out WAN to the original sender, which is incorrect here — the reply needs to go back to LAN.
So return traffic gets misrouted or dropped, unless another rule compensates.
When You Disable reply-to on WAN Rule:
Disabling reply-to allows pfSense to use its routing table to determine the return path instead of forcing it out the WAN. That allows replies to go back correctly to LAN clients.Hence, you no longer need to apply the floating rule to LAN — WAN is enough — because routing is no longer forced in a specific (incorrect) direction.
So: Which is the More Valid Approach?
Best practice in this case:
Use the floating rule on WAN only, and disable reply-to.Why?
It’s more precise: you're matching traffic only where it's actually leaving (WAN).You avoid unnecessary application on LAN.
You preserve pfSense's proper routing behavior, without the quirks of forced reply paths.
This aligns better with pfSense firewall design philosophy, where you disable reply-to when handling internal or non-public traffic on WAN interfaces (common in double NAT or transit VLAN setups).
This time, unlike the first explanation I found unclear and motivated me to post here, it looks logic and corresponds to the netgate guide you linked to about disabling repy-to behind a bridged interface
Thank you again for the support and proper fix you provided
PS: AI is really some bullshit for expert analysis as it just lies when it fails
-
@phil80 said in Bug or undocumented? Floating rule on out ditection not properly applying on final interface unless it is also applied to originating interface:
PS: AI is really some bullshit for expert analysis as it just lies when it fails
yeah its horrible - since it provides wrong info for shit I actually know, how can I trust it asking it stuff I don't.. I filtered out the AI shit google provides when you google something.. Because it was always freaking wrong when I looked into their answers.
I wouldn't trust anything it spews out.. Might give you some clues to where to actually start looking.. Ok - and it might be ok at writing code or something.. But if you ask it an actual tech question.. Every single time I have asked it something it was BS..
might be ok if you ask it to create a summary of a wall of text?
I did a bit more digging from links in an older thread when reply-to came up.. I found one comment that said pfsense and the other option I will not name ;) are doing it wrong - heheh.. That it really shouldn't be on by default. But I have never ran into any issues other then doing this sort of thing.
-
@johnpoz
Thank you, i fixed my rules. I didn't suspect that not disabling reply-to at the end caused such wrong routing issues and that my fix was just a workaround maintaining the broken routing -
I pushed further and added these rules to WAN
Why I can still connect to the modem ?
I thought that implicit allow rules can be blocked by explicit block rules on an interface
By implicit, I supposed the stateful reply traffic is part of it
In that case, I expected the reply traffic from modem would be blocked, but it is notNow after reading, I see that replies are always allowed unless blocked by a floating rule or by disabling the state. So the rule on wan would only apply to new connections. Since any incoming new connection is blocked by default, what's the use of the automatic rule "Block private networks and loopback addresses" (I just manually added it after allowing ping requests from modem as my modem seems to require these pings even if set as static ip)
-
@phil80 that allow rule? you have it going to pfsense wan IP.. Not your modem IP.. All that allows you to do is go to your pfsense wan IP the 192.168.1.2..
Oh your modem pings pfsense IP?? Really why would it be doing that? But yeah that should work. I see hits on it.. Unless you have some rule in floating that blocks it? Floating is evaluated before interface rules. But if that was the case you wouldn't be seeing hits to it - ie that 523kb
when I get a chance I will fire up my 2.8 VM and try that.
-
@johnpoz
My question is not about puing rule.
It is the last block rule I added manually to replace the auto rule "Block private networks and loopback addresses"
If I enable the auto rule, ping requests will be blocked as the auto rule is at the topMy question is: in general, what's the use of the auto rules on WAN (or my manual rule), since:
- they don't block stateful valid reply traffic
- by default, all incoming requests are blocked
-
@phil80 great question.. Yeah I personally don't think either of those default rules make a lot of sense any more.. For starters there are very few bogon actually left.. And your isp shouldn't be routing them if they are bogon. So for them to get to your connection they would have to be coming from your ISP network. Or spoofed - what would be the point of spoofing bogon or rfc1918?
Same goes for rfc1918.. It is bogon in that it wouldn't route across the public internet.. So its either your local wan network, or something in your isp network.
And yeah your right without any allow rules they would all be blocked anyway.. So those rules only block stuff you would be wanting to allow in the first place.. Would it matter if a few stray packets from your isp network hit your open ports?
I leave them to just see out of curiosity how much traffic hits them ;)
So you can see my rule counters have been up for a while - my plex rule has passed almost 4TB.. And I have seen a total of 480B on bogon ;) and a whopping 70KB on my rfc1918 rule ..
And since I limit source IPs to my open ports, and not any.. Bogon and Rfc1918 would have be in my allow list.. Which they are not - so even if bogon or rfc1918 source hit my wan towards one of my open ports they wouldn't be allowed anyway.
If I saw the counters on those 2 rules going up - I would be curious to what it is, and start a packet capture, or set them to log.. I currently have logging of them off.
I log what I want to log via log settings on my rules.. I have some rules at end of my wan to log traffic I am interested in. SYN and common UDP ports.. But the default log of default deny doesn't interest me.. I mean it was blocked and is going to be noise.. I would only enable that logging if was troubleshooting something for example
-
@johnpoz
Thank you for validating my thoughts and setup
By the way, the best thing I did was moving intervlan routing to pfsense and keep fast 10 Gb servers on same VLAN on the switch. So simpler to maintain and stopped using the asymmetric insecure routing for internet -
@johnpoz
I have an issue with the floating ruleBlock private networks traffic from leaving the WAN interface
It also blocks VPN traffic:Block private networks traffic from leaving the WAN... (1620978529) 88.198.122.154:53 10.3.3.81:40710It is the reply traffic from websites accessed through VPN by client 10.3.3.81 in this case
Is it recommended to amend the pass rule to allow VPN traffic or just completely delete the block rule ?
Also, I don't well understand that log error as I thought that VPN interface was behind WAN and natted, with floating rule applying after NATIf I edit the floating tab like below, it works
So, as I understand, floating rules do apply befor NAT on out direction and after NAT on IN direction
-
Wel, really strange
I disabled the Allo VPN floating rule and restarted pfsense
Now, VPN works even with the block rule and without pass rule, as expected
Really strange that it needed a reboot and the logs I posted above
