Why do no floating rules match?
-
@SteveITS Hi there, also thank you contributing. Really appreciated!
In the end I only want to add a Limiter/Queue to this data stream. And from what I've read for now one week I need a matching rule first. And since all is happening via port 50454 I thought this was the easiest criteron to match on.
As I said (and showed in screenshots) there is (as of now) only one (floating!) rule.
Or does the "if no other rules match the traffic" hold true for interface-based rules? (BTW, I think no other rule under the WAN tab matches...)
-
@Rockojfonzo One can add a limiter to any rule in its advanced settings. It’s on the first connection so incoming on that port…e.g. for a web server download it would be inbound on 443 so the reply packets are limited. I can send an example in a little while.
I use floating for traffic shaping/QoS rules. There I tend to use the wizard, select one of each type to let it create all the queues, then disable or edit/copy those rules as necessary.
-
@SteveITS said in Why do no floating rules match?:
@Rockojfonzo One can add a limiter to any rule in its advanced settings.
Yes, I know. But I'm not able to create ANY matching rule for traffic on WAN to/from port 50454.
-
@Rockojfonzo I'm a bit late to the game but did give it a go on a test VM. The VM is on 25.03-BETA right now and IPv6 get's NAT-ed because reasons :) . What pfSense version are you testing it on?
First, according to the Netgate doc "Quick" has no effect on 'Match' rules since it only marks traffic, but that just FYI:
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#match-action
Where and what tools to you use to check if the traffic gets matched? Do you run tcpdump or do you use the GUI for it? And on which interface(s)?
I tested it by creating a rule exactly like yours and making a DNS query to Quad9 on a client behind the pfSense:
dig -6 ibm.com +short @dns.quad9.net 23.67.128.173... which opens a UDP connection to port 53 of a Quad9 server.
In the firewall logs I can see that connection and also if it is run from the ssh console on the pfSense. 'Match' seems not to create a state.
In the below output rule 96 being the floating rule, vtnet0 is WAN and vtnet2 is the LAN interface.
[root@pfp.home.arpa]/root: tcpdump -e -ttt -ni pflog0 net 2620:fe::/64 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes 00:00:00.000000 rule 96/0(match) [ridentifier 1741178672]: match in on vtnet2: fdaa:b2b4:d8b2:1099:be24:11ff:fefb:cd2e.43239 > 2620:fe::9.53: 43909+ [1au] A? ibm.com. (48) 00:00:00.000018 rule 91/0(match) [ridentifier 1000005716]: pass out on vtnet0: fdaa:b2b4:d8b2:1000::40.13714 > 2620:fe::9.53: 43909+ [1au] A? ibm.com. (48) -
@Rockojfonzo The rule counters will show if traffic is matching. The second one here has not matched anything:
Here is a floating/match rule we have for an inbound connection, to a 1:1 NAT alias on WAN:
and then lower down the queue for QoS:
Here is a regular firewall rule, in this case to apply a limiter to outgoing connections (and hence, their responses/download):
(In=Upload because this rule is on LAN)I changed it to an "out" rule and additionally tested it with source port "50454".
A floating Out rule happens after NAT so likely the source port would change if you are using NAT here. Also the source IP would be your WAN IP. There's a section on tagging packets here:
https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html#marking-and-matchingrule on LAN to tag:
and the rule on WAN to set the queue for tagged packets:
-
@patient0 I'm on
2.7.2-RELEASE (amd64)
built on Mon Mar 4 20:53:00 CET 2024
FreeBSD 14.0-CURRENTI removed the "Quick" checkmark but nothing changed.
I used GUI with settings resulting in
Running packet capture: /usr/sbin/tcpdump -ni igc2 -c '1000' -U -w - '((port 50454)) and ((not vlan))'See results in first post
Then I changed the port in my only floating rule like you to 53 (DNS).
Using your fine commands on my machine result in the same:tcpdump -e -ttt -ni pflog0 port 53 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes [2.7.2-RELEASE][admin@fw]/root: dig -6 ibm.com +short @dns.quad9.net 23.67.128.173 [2.7.2-RELEASE][admin@fw]/root: 00:00:00.000000 rule 114/0(match) [uid 0] [ridentifier 1741168563]: match out on igc2: 2a00:6020:1000:30::2ac4.38059 > 2620:fe::fe.53: 30644+ [1au] A? ibm.com. (48)Now, changing rule back to port 50454:
tcpdump -e -ttt -ni pflog0 port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytesOnly silence. :-(
-
@Rockojfonzo just do a tcpdump to see if you see that traffic at all.. Again if you're trying match pfsense generated traffic the direction would be out not in..
What is generating this traffic to port 50454?
-
@Rockojfonzo It is indeed strange. To test it full I got a very simple UDP server on port 50454 (in go) that answers the text you enter back and let that run a jump host of mine.
The client behind the pfSense does ancto port 50454 and forces the source port to 50454 too.## jump host with the client sending 'hello' jmp $ /tmp/udpserver :50454 > hello ## the client call jane@devuan5:~$ nc -uv -p 50454 <jmp host IPv6> 50454 Ncat: Version 7.93 ( https://nmap.org/ncat ) Ncat: Connected to <jmp host IPv6>:50454. hello Hello UDP Client ^C ## let run tcpdump on the WAN interface of the pfSense directly [2.7.2-RELEASE][root@pfsense.home.arpa]/root: tcpdump -ni vtnet0 ip6 and udp and port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on vtnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 07:19:17.685890 IP6 fdaa:b2b4:d8b2:1000::45.50454 > <jmp host IPv6>.50454: UDP, length 6 07:19:17.689845 IP6 <jmp host IPv6>.50454 > fdaa:b2b4:d8b2:1000::45.50454: UDP, length 17 ## and alternatively run tcpdump on pflog0 [2.7.2-RELEASE][root@pfsense.home.arpa]/root: tcpdump -ne -ttt -i pflog0 ip6 and udp and port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), snapshot length 262144 bytes 00:00:00.000000 rule 100/0(match) [ridentifier 1741208020]: match in on vtnet1: fdaa:b2b4:d8b2:1020:be24:11ff:fefb:cd2e.50454 > <jmp host IPv6>.50454: UDP, length 5 00:00:00.000008 rule 96/0(match) [ridentifier 1000003713]: pass out on vtnet0: fdaa:b2b4:d8b2:1000::45.50454 > <jmp host IPv6>.50454: UDP, length 5If direction set to
inin the rule, the firewall interfacepflog0does see the matching package in the LAN interface vtnet1 first, when it "enters" pfSense and then out of the WAN interface by the default allow-out-rule.
The returning package is not matched on WAN, maybe because I do NAT on IPv6 and pfSense has the outgoing packages state save? Or the state is kept in any way and it will only match on the LAN interface if the direction is set toin, not sure about that.But long story short it really should show some output.
The current firewall rules can be seen on the pfSense in/tmp/rules.debug. Can you check that thematchrule does havelogenabled withfgrep match /tmp/rules.debug?[2.7.2-RELEASE][root@pfsense.home.arpa]/root: fgrep match /tmp/rules.debug match in log quick inet6 proto udp from any to any port 50454 ridentifier 1741208020 label "USER_RULE: Testing and Logging Match UDP" label "id:1741208020" -
@johnpoz Sorry, been a busy day.
tcpdump results are more or less the same as in 1st post:[2.7.2-RELEASE][admin@fw]/root: tcpdump -ni igc2 port 50454 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on igc2, link-type EN10MB (Ethernet), snapshot length 262144 bytes 20:21:17.215574 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.215589 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.233468 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 96 20:21:17.233699 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 96 20:21:17.233761 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:17.233878 IP6 2abc:1234:1234:30::2ac4.50454 > 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454: UDP, length 96 20:21:27.255462 IP6 2abd:11:5678:5678::e228:6dff:feb9:4ae8.50454 > 2abc:1234:1234:30::2ac4.50454: UDP, length 32 ^C 7 packets captured 621 packets received by filter 0 packets dropped by kernelIt's a wireguard tunnel I want to limit. No fancy in-tunnel filtering, simply the whole stream would do for me.
-
@patient0 Hi Thomas, so much effort you put in here. Thank you!
Yes, log is enabled:
[2.7.2-RELEASE][admin@fw]/root: more /tmp/rules.debug |grep match
match log inet6 proto udp from any port 50454 to any ridentifier 1741292534 label "USER_RULE" label "id:1741292534"BTW, I also tried matching SMB out on LAN port: Same result, no match possible:
[2.7.2-RELEASE][admin@fw.]/root: more /tmp/rules.debug | grep match match log on { igc1 } inet proto tcp from any port 139 to any ridentifier 1741294762 flags S/SA label "USER_RULE" label "id:1741294762" match log on { igc1 } inet proto tcp from any port 445 to any ridentifier 1741295003 flags S/SA label "USER_RULE" label "id:1741295003"Traffic:
[2.7.2-RELEASE][admin@fw]/root: tcpdump -ni igc1 port 445 or port 139 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on igc1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 21:06:49.795443 IP 192.168.11.6.43750 > 192.168.100.100.445: Flags [F.], seq 3855556357, ack 853146708, win 229, options [nop,nop,TS val 1399014346 ecr 112590208], length 0 21:06:49.795459 IP 192.168.11.6.43752 > 192.168.100.100.445: Flags [S], seq 334166509, win 29200, options [mss 1460,sackOK,TS val 1399014346 ecr 0,nop,wscale 7], length 0 21:06:49.814822 IP 192.168.100.100.445 > 192.168.11.6.43752: Flags [S.], seq 3542171948, ack 334166510, win 28960, options [mss 1460,sackOK,TS val 112593711 ecr 1399014346,nop,wscale 7], length 0 21:06:49.814986 IP 192.168.11.6.43752 > 192.168.100.100.445: Flags [.], ack 1, win 229, options [nop,nop,TS val 1399014365 ecr 112593711], length 0 21:06:49.833525 IP 192.168.100.100.445 > 192.168.11.6.43750: Flags [F.], seq 1, ack 1, win 227, options [nop,nop,TS val 112593711 ecr 1399014346], length 0 21:06:49.833684 IP 192.168.11.6.43750 > 192.168.100.100.445: Flags [.], ack 2, win 229, options [nop,nop,TS val 1399014384 ecr 112593711], length 0Zero / Zero
:-(
-
@Rockojfonzo said in Why do no floating rules match?:
It's a wireguard tunnel I want to limit.
So the traffic coming from pfsense.. And would be allowed the hidden rule that allows pfsense to go anywhere.. Which would explain why your not seeing any triggers. That is the 2 endpoints of your tunnel I take it
https://docs.netgate.com/pfsense/en/latest/firewall/pf-ruleset.html
# let out anything from the firewall host itself and decrypted IPsec traffic pass out inet all keep state allow-opts ridentifier 1000015165 label "let out anything IPv4 from firewall host itself" pass out inet6 all keep state allow-opts ridentifier 1000015166 label "let out anything IPv6 from firewall host itself"As to your smb rules - where are those rules? In floating? The only stuff I could that could possible match is that 100.100.445 to 11.6.43752 syn,ack [S.]
-
@johnpoz Ah, ok, I thought, because I can see it in tcpdump I could also match it with a rule. Bummer.
Yes, that rules are again the only ones in floating, direction "any" for testing purposes.
I'd also think they should be matching 100.100.445 to 11.6.43752 syn,ack [S.]
But again no luck. -
@Rockojfonzo where are those 2 networks at exactly connected to pfsense?
-
@johnpoz 192.168.100.0 is the remote side of the Wireguard tunnel (tun_wg0 so to say), 192.168.11.0 is on LAN (igc1)
I can match to destination 192.168.100.100:445 on LAN (which is "in" on LAN, out to Wireguard), but I need to limit (and therefore match) the packets that go out on LAN.
-
@Rockojfonzo I for sure do not know enough about how/where the firewall ties into the wireguard interface, etc.
Might be better to bring this up in the wireguard section.. On how to set limiters on this.. This seems like more related to that than actual firewall where packets enter and leave a physical interface or vlan.
Same could be related to say an ipsec or openvpn tunnel.. I have never tried tried or wanted to set a limit on such a connection. I have put some firewall rules on the openvpn interface - but never tried to match traffic so I could limit, etc. Maybe its just not possible, or need to do it a different way then match in a firewall rule?
-
@Rockojfonzo No worries, wouldn't do if I'm not interested. And I can't let go :) ... a SysAdmin thing, no sure.
After replacing the UPD simple-server with a Wireguard tunnel I observe the same as you, nothing gets logged.
Even if I enable logging for the default 'Wireguard' interface - in the firewall rules - allow-all-ip6 generic rule. The rules works and I can ping the pfSense from outside with it but it does not get logged. Not sure if that's a bug a limitation.
I guess @johnpoz suggestion makes sense to ask it in the wireguard section.
-
@patient0 Yeah, I read a lot about traffic shaping and VPN in the last days. That's why I didn't dream about doing something in the tunnel. But at least on the WAN interface or the (to my understanding) "de-VPNed" packets on the LAN interface should be "loggable".
However, thank you guys for all your energy and insights. I'll head over to the wireguard section and nag people there. ;-)
AND I will set up a similar thing on a pristine sophos xg that sits here and waits for installation. Let's see if it's worth the money. >:-) -
R Rockojfonzo referenced this topic on
-
Last notice on this: I built a setup with a Sophos XGS which doesn't support wireguard but IPsec. There I could track traffic (but only on non-floating rules!) without any problems, matching on addresses and/or ports.
So it seems to be only wireguard-related.
