Impossible to reset states with pfctl

bole5

Hello,
I have a rule to block internet connection for my children based on a defined schedule.

Since blocking rules on schedule do not flush already established states I have a cron job to flush connections a minute after the blocking rules apply.

Now to the problem. One of the clients is constantly talking over WhatsApp when the rule becomes effective and my cron job fails to flush connections for this client. Flushing all states does work.

Given that I am using commands suggested by @jimp in two older posts, could it be that we have a bug in the firewall?
More debugging details are presented below.

Thank you in advance.

Offender IP: 192.168.30.22
Trying to kill states with pfctl -k x.x.x.x ; pfctl -k 0.0.0.0/0 -k x.x.x.x

[2.5.2-RELEASE][root@fortet.lan]/root: /sbin/pfctl -k 192.168.30.22 ; /sbin/pfctl -k 0.0.0.0/0 -k 192.168.30.22 ; pfctl -ss | grep 192.168.30.22
killed 2 states from 1 sources and 0 destinations
killed 1 states from 1 sources and 1 destinations
re0 tcp 91.17.1.38:58512 (192.168.30.22:53904) -> 17.57.146.169:5223       ESTABLISHED:ESTABLISHED
re0 udp 91.17.1.38:54982 (192.168.30.22:50015) -> 52.114.249.46:3478       MULTIPLE:MULTIPLE
re1.30 udp 52.114.249.46:3478 -> 192.168.30.22:50015       SINGLE:NO_TRAFFIC

As you can see above some states are still present...
Same happens if I replace IP with whole KIDS network: 192.168.30.0/24 in the above commands.

Now let's try the "nuclear" option

[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -F all
rules cleared
nat cleared
35 tables deleted.
altq cleared
303 states cleared
source tracking entries cleared
pf: statistics cleared
pf: interface flags reset
[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -ss | grep 192.168.30.22
[2.5.2-RELEASE][root@fortet.lan]/root:

Success! No states left.

Suggestions that failed came from following posts:

https://forum.netgate.com/topic/41736/resetting-states-from-console
https://forum.netgate.com/topic/51702/reset-states-from-cron

stephenw10

@bole5 said in Impossible to reset states with pfctl:

re1.30 udp 52.114.249.46:3478 -> 192.168.30.22:50015 SINGLE:NO_TRAFFIC

That is an inbound state which is interesting. Do you have, say, upnp enabled allowing connections to re-establish?

What states exist before you run those commands? I.e. which states is it matching and killing?

If you use scheduled pass rules instead of block the states would be removed anyway.

Steve

bole5

@stephenw10 Thank you for suggestions.
There are no upnp rules. I can dump the states next time.

I did have scheduled "opposite" allow rules and this did not work. It works for most other states but the WhatsApp phone calls are proven hard to "kill".

Regarding the states it killed, I dumped the states before running the kill states commands:

[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -ss | grep 192.168.30.22
re0 tcp 91.17.1.38:58512 (192.168.30.22:53904) -> 17.57.146.169:5223       ESTABLISHED:ESTABLISHED
re0 udp 91.17.1.38:54982 (192.168.30.22:50015) -> 52.114.249.46:3478       MULTIPLE:MULTIPLE
re1.30 udp 127.0.0.1:5300 (192.168.30.1:53) <- 192.168.30.22:59349       SINGLE:MULTIPLE
re1.30 udp 52.114.249.46:3478 -> 192.168.30.22:50015       MULTIPLE:MULTIPLE
[2.5.2-RELEASE][root@fortet.lan]/root: /sbin/pfctl -k 192.168.30.22 ; /sbin/pfctl -k 0.0.0.0/0 -k 192.168.30.22 ; pfctl -ss | grep 192.168.30.22
killed 2 states from 1 sources and 0 destinations
killed 1 states from 1 sources and 1 destinations
re0 tcp 91.17.1.38:58512 (192.168.30.22:53904) -> 17.57.146.169:5223       ESTABLISHED:ESTABLISHED
re0 udp 91.17.1.38:54982 (192.168.30.22:50015) -> 52.114.249.46:3478       MULTIPLE:MULTIPLE
re1.30 udp 52.114.249.46:3478 -> 192.168.30.22:50015       SINGLE:NO_TRAFFIC

stephenw10

Hmm, try showing the states with more verbose flags so you can see what rule is allowing them to pass. You can probably kill them by label or ID from that.

bole5

@stephenw10
Sorry it took me some time to get back to this problem.
It is weird that some states simple can't be killed even when I use id. See below commands.

[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -s state -vv | grep 192.168.30.15 -A 3
re1.30 tcp 17.57.146.167:5223 <- 192.168.30.15:51017       ESTABLISHED:ESTABLISHED
   [1527814839 + 131048] wscale 7  [1069456121 + 47232] wscale 5
   age 00:10:56, expires in 23:57:19, 19:19 pkts, 7798:5382 bytes, rule 387
   id: 0100000061b4938d creatorid: 49e23653
re0 tcp 94.16.155.37:64992 (192.168.30.15:51017) -> 17.57.146.167:5223       ESTABLISHED:ESTABLISHED
   [1069456121 + 47232] wscale 5  [1527814839 + 131048] wscale 7
   age 00:10:56, expires in 23:57:19, 19:19 pkts, 7798:5382 bytes, rule 63
   id: 0100000061b4938e creatorid: 49e23653
[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -k id -k 0100000061b4938e
killed 0 states
[2.5.2-RELEASE][root@fortet.lan]/root: pfctl -s state -vv | grep 192.168.30.15 -A 3
re1.30 tcp 17.57.146.167:5223 <- 192.168.30.15:51017       ESTABLISHED:ESTABLISHED
   [1527814839 + 131048] wscale 7  [1069456121 + 47232] wscale 5
   age 00:11:14, expires in 23:57:01, 19:19 pkts, 7798:5382 bytes, rule 387
   id: 0100000061b4938d creatorid: 49e23653
re0 tcp 94.16.155.37:64992 (192.168.30.15:51017) -> 17.57.146.167:5223       ESTABLISHED:ESTABLISHED
   [1069456121 + 47232] wscale 5  [1527814839 + 131048] wscale 7
   age 00:11:14, expires in 23:57:01, 19:19 pkts, 7798:5382 bytes, rule 63
   id: 0100000061b4938e creatorid: 49e23653

The state with id: 0100000061b4938e (WhatsApp) is not killed... What am I doing wrong?

stephenw10

Hmm, the returned killed 0 states implies it's not matching any states. Sure looks likeit should though.