Why do no floating rules match?
-
@johnpoz Hi John, thank you so much for jumping in!
It's – for testing's sake – the only floating rule.
I changed it to an "out" rule and additionally tested it with source port "50454".
Nothing get's logged.
Do I need to "Reset States" on every change to make it work?Thanks a lot for your help!
-
@Rockojfonzo said in Why do no floating rules match?:
with source port "50454".
"source port" is nearly always a 'random value.
If you have to use a port number, its always a 'destination port'.@Rockojfonzo said in Why do no floating rules match?:
"out"
That's from pfSense (the thing that does the routing, aka pfSense) to an interface, so going out of the box to some remote destination - could be LAN based, or WAN based..
@Rockojfonzo said in Why do no floating rules match?:
Do I need to "Reset States" on every change to make it work?
Depends ^^
Be ware : the following is what I think of how it works :
If you have a rule that matches - gets applied, then a state is created for a data stream that matches.
Every rules has an "ID" number, so when it gets changed, the related states also get reset. That is, that would be the case in a perfect world - and what options are used : see also System>Advanced >Firewall & NAT
If you had a pass rule, and you changed the rule, and the event was missed, the traffic still flows even when it couldn't match the rule anymore.
The other way around isn't true : if a rule wasn't matching, no states are created. As soon as the rule gets changed for a situation that could match, it will match - and starts to create a state for this traffic stream.When you change rule,
-
@Gertjan Hi Gertjan, thank you, too.
Yeah, I read that about the source port, I tried just out of despair.When I switched to "out" on John's recommendation I also selected WAN as the interface (where the packet capture originates from).
Any other idea to try?
Thank you so much!
-
Example : I have this web server that uses port 2828. It's a Monit instance I use don a dedicated server, somewhere in a data center.
I've created the match rule show above - don't ask what the difference is between pass and match, neither what quick does (are there slow rules ?
I visit my web server, TCP, using this destination port 2828.
The rule started to be used (it was applying) :
Imho, because its a match rule, it just count the traffic.
-
@Gertjan As for the "Quick" checkbox I read somewhere that this is related to already established connections. Hence I was asking. But I checked it anyways.
Thank you for your example. I don't see any vital difference to my rule and that's why I'm pulling my hair out. I don't get it why it doesn't log anything while I can capture the packets.
-
@Rockojfonzo Crucially with floating rules, the last match wins unless quick is selected.
They also happen after NAT (n/a for IPv6).
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#processing-order
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#match-action
“Without Quick checked, the rule will only take effect if no other rules match the traffic. It reverses the behavior of “first match wins” to be “last match wins”. “What are you trying to accomplish? Floating rules can be tricky for the above and other reasons and often there are other solutions.
-
@SteveITS Hi there, also thank you contributing. Really appreciated!
In the end I only want to add a Limiter/Queue to this data stream. And from what I've read for now one week I need a matching rule first. And since all is happening via port 50454 I thought this was the easiest criteron to match on.
As I said (and showed in screenshots) there is (as of now) only one (floating!) rule.
Or does the "if no other rules match the traffic" hold true for interface-based rules? (BTW, I think no other rule under the WAN tab matches...)
-
@Rockojfonzo One can add a limiter to any rule in its advanced settings. It’s on the first connection so incoming on that port…e.g. for a web server download it would be inbound on 443 so the reply packets are limited. I can send an example in a little while.
I use floating for traffic shaping/QoS rules. There I tend to use the wizard, select one of each type to let it create all the queues, then disable or edit/copy those rules as necessary.
-
@SteveITS said in Why do no floating rules match?:
@Rockojfonzo One can add a limiter to any rule in its advanced settings.
Yes, I know. But I'm not able to create ANY matching rule for traffic on WAN to/from port 50454.
-
@Rockojfonzo I'm a bit late to the game but did give it a go on a test VM. The VM is on 25.03-BETA right now and IPv6 get's NAT-ed because reasons :) . What pfSense version are you testing it on?
First, according to the Netgate doc "Quick" has no effect on 'Match' rules since it only marks traffic, but that just FYI:
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#match-action
Where and what tools to you use to check if the traffic gets matched? Do you run tcpdump or do you use the GUI for it? And on which interface(s)?
I tested it by creating a rule exactly like yours and making a DNS query to Quad9 on a client behind the pfSense:
dig -6 ibm.com +short @dns.quad9.net 23.67.128.173... which opens a UDP connection to port 53 of a Quad9 server.
In the firewall logs I can see that connection and also if it is run from the ssh console on the pfSense. 'Match' seems not to create a state.
In the below output rule 96 being the floating rule, vtnet0 is WAN and vtnet2 is the LAN interface.
[root@pfp.home.arpa]/root: tcpdump -e -ttt -ni pflog0 net 2620:fe::/64 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes 00:00:00.000000 rule 96/0(match) [ridentifier 1741178672]: match in on vtnet2: fdaa:b2b4:d8b2:1099:be24:11ff:fefb:cd2e.43239 > 2620:fe::9.53: 43909+ [1au] A? ibm.com. (48) 00:00:00.000018 rule 91/0(match) [ridentifier 1000005716]: pass out on vtnet0: fdaa:b2b4:d8b2:1000::40.13714 > 2620:fe::9.53: 43909+ [1au] A? ibm.com. (48) -
@Rockojfonzo The rule counters will show if traffic is matching. The second one here has not matched anything:
Here is a floating/match rule we have for an inbound connection, to a 1:1 NAT alias on WAN:
and then lower down the queue for QoS:
Here is a regular firewall rule, in this case to apply a limiter to outgoing connections (and hence, their responses/download):
(In=Upload because this rule is on LAN)I changed it to an "out" rule and additionally tested it with source port "50454".
A floating Out rule happens after NAT so likely the source port would change if you are using NAT here. Also the source IP would be your WAN IP. There's a section on tagging packets here:
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#marking-and-matchingrule on LAN to tag:
and the rule on WAN to set the queue for tagged packets:
-
@patient0 I'm on
2.7.2-RELEASE (amd64)
built on Mon Mar 4 20:53:00 CET 2024
FreeBSD 14.0-CURRENTI removed the "Quick" checkmark but nothing changed.
I used GUI with settings resulting in
Running packet capture: /usr/sbin/tcpdump -ni igc2 -c '1000' -U -w - '((port 50454)) and ((not vlan))'See results in first post
Then I changed the port in my only floating rule like you to 53 (DNS).
Using your fine commands on my machine result in the same:tcpdump -e -ttt -ni pflog0 port 53 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes [2.7.2-RELEASE][admin@fw]/root: dig -6 ibm.com +short @dns.quad9.net 23.67.128.173 [2.7.2-RELEASE][admin@fw]/root: 00:00:00.000000 rule 114/0(match) [uid 0] [ridentifier 1741168563]: match out on igc2: 2a00:6020:1000:30::2ac4.38059 > 2620:fe::fe.53: 30644+ [1au] A? ibm.com. (48)Now, changing rule back to port 50454:
tcpdump -e -ttt -ni pflog0 port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytesOnly silence. :-(
-
@Rockojfonzo just do a tcpdump to see if you see that traffic at all.. Again if you're trying match pfsense generated traffic the direction would be out not in..
What is generating this traffic to port 50454?
-
@Rockojfonzo It is indeed strange. To test it full I got a very simple UDP server on port 50454 (in go) that answers the text you enter back and let that run a jump host of mine.
The client behind the pfSense does ancto port 50454 and forces the source port to 50454 too.## jump host with the client sending 'hello' jmp $ /tmp/udpserver :50454 > hello ## the client call jane@devuan5:~$ nc -uv -p 50454 <jmp host IPv6> 50454 Ncat: Version 7.93 ( https://nmap.org/ncat ) Ncat: Connected to <jmp host IPv6>:50454. hello Hello UDP Client ^C ## let run tcpdump on the WAN interface of the pfSense directly [2.7.2-RELEASE][root@pfsense.home.arpa]/root: tcpdump -ni vtnet0 ip6 and udp and port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on vtnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 07:19:17.685890 IP6 fdaa:b2b4:d8b2:1000::45.50454 > <jmp host IPv6>.50454: UDP, length 6 07:19:17.689845 IP6 <jmp host IPv6>.50454 > fdaa:b2b4:d8b2:1000::45.50454: UDP, length 17 ## and alternatively run tcpdump on pflog0 [2.7.2-RELEASE][root@pfsense.home.arpa]/root: tcpdump -ne -ttt -i pflog0 ip6 and udp and port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes 00:00:00.000000 rule 100/0(match) [ridentifier 1741208020]: match in on vtnet1: fdaa:b2b4:d8b2:1020:be24:11ff:fefb:cd2e.50454 > <jmp host IPv6>.50454: UDP, length 5 00:00:00.000008 rule 96/0(match) [ridentifier 1000003713]: pass out on vtnet0: fdaa:b2b4:d8b2:1000::45.50454 > <jmp host IPv6>.50454: UDP, length 5If direction set to
inin the rule, the firewall interfacepflog0does see the matching package in the LAN interface vtnet1 first, when it "enters" pfSense and then out of the WAN interface by the default allow-out-rule.
The returning package is not matched on WAN, maybe because I do NAT on IPv6 and pfSense has the outgoing packages state save? Or the state is kept in any way and it will only match on the LAN interface if the direction is set toin, not sure about that.But long story short it really should show some output.
The current firewall rules can be seen on the pfSense in/tmp/rules.debug. Can you check that thematchrule does havelogenabled withfgrep match /tmp/rules.debug?[2.7.2-RELEASE][root@pfsense.home.arpa]/root: fgrep match /tmp/rules.debug match in log quick inet6 proto udp from any to any port 50454 ridentifier 1741208020 label "USER_RULE: Testing and Logging Match UDP" label "id:1741208020" -
@johnpoz Sorry, been a busy day.
tcpdump results are more or less the same as in 1st post:[2.7.2-RELEASE][admin@fw]/root: tcpdump -ni igc2 port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on igc2, link-type EN10MB (Ethernet), snapshot length 262144 bytes 20:21:17.215574 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.215589 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.233468 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 96 20:21:17.233699 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 96 20:21:17.233761 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.233878 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:27.255462 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 32 ^C 7 packets captured 621 packets received by filter 0 packets dropped by kernelIt's a wireguard tunnel I want to limit. No fancy in-tunnel filtering, simply the whole stream would do for me.
-
@patient0 Hi Thomas, so much effort you put in here. Thank you!
Yes, log is enabled:
[2.7.2-RELEASE][admin@fw]/root: more /tmp/rules.debug |grep match
match log inet6 proto udp from any port 50454 to any ridentifier 1741292534 label "USER_RULE" label "id:1741292534"BTW, I also tried matching SMB out on LAN port: Same result, no match possible:
[2.7.2-RELEASE][admin@fw.]/root: more /tmp/rules.debug | grep match match log on { igc1 } inet proto tcp from any port 139 to any ridentifier 1741294762 flags S/SA label "USER_RULE" label "id:1741294762" match log on { igc1 } inet proto tcp from any port 445 to any ridentifier 1741295003 flags S/SA label "USER_RULE" label "id:1741295003"Traffic:
[2.7.2-RELEASE][admin@fw]/root: tcpdump -ni igc1 port 445 or port 139 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on igc1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 21:06:49.795443 IP 192.168.11.6.43750 > 192.168.100.100.445: Flags [F.], seq 3855556357, ack 853146708, win 229, options [nop,nop,TS val 1399014346 ecr 112590208], length 0 21:06:49.795459 IP 192.168.11.6.43752 > 192.168.100.100.445: Flags [S], seq 334166509, win 29200, options [mss 1460,sackOK,TS val 1399014346 ecr 0,nop,wscale 7], length 0 21:06:49.814822 IP 192.168.100.100.445 > 192.168.11.6.43752: Flags [S.], seq 3542171948, ack 334166510, win 28960, options [mss 1460,sackOK,TS val 112593711 ecr 1399014346,nop,wscale 7], length 0 21:06:49.814986 IP 192.168.11.6.43752 > 192.168.100.100.445: Flags [.], ack 1, win 229, options [nop,nop,TS val 1399014365 ecr 112593711], length 0 21:06:49.833525 IP 192.168.100.100.445 > 192.168.11.6.43750: Flags [F.], seq 1, ack 1, win 227, options [nop,nop,TS val 112593711 ecr 1399014346], length 0 21:06:49.833684 IP 192.168.11.6.43750 > 192.168.100.100.445: Flags [.], ack 2, win 229, options [nop,nop,TS val 1399014384 ecr 112593711], length 0Zero / Zero
:-(
-
@Rockojfonzo said in Why do no floating rules match?:
It's a wireguard tunnel I want to limit.
So the traffic coming from pfsense.. And would be allowed the hidden rule that allows pfsense to go anywhere.. Which would explain why your not seeing any triggers. That is the 2 endpoints of your tunnel I take it
https://docs.netgate.com/pfsense/en/latest/firewall/pf-ruleset.html
# let out anything from the firewall host itself and decrypted IPsec traffic pass out inet all keep state allow-opts ridentifier 1000015165 label "let out anything IPv4 from firewall host itself" pass out inet6 all keep state allow-opts ridentifier 1000015166 label "let out anything IPv6 from firewall host itself"As to your smb rules - where are those rules? In floating? The only stuff I could that could possible match is that 100.100.445 to 11.6.43752 syn,ack [S.]
-
@johnpoz Ah, ok, I thought, because I can see it in tcpdump I could also match it with a rule. Bummer.
Yes, that rules are again the only ones in floating, direction "any" for testing purposes.
I'd also think they should be matching 100.100.445 to 11.6.43752 syn,ack [S.]
But again no luck. -
@Rockojfonzo where are those 2 networks at exactly connected to pfsense?
-
@johnpoz 192.168.100.0 is the remote side of the Wireguard tunnel (tun_wg0 so to say), 192.168.11.0 is on LAN (igc1)
I can match to destination 192.168.100.100:445 on LAN (which is "in" on LAN, out to Wireguard), but I need to limit (and therefore match) the packets that go out on LAN.
