Problems with all floating rules setup?
-
I'm having some weird results with an all floating rules setup. I'm wondering if this is an expected use case, or if I'm living on the edge here?
Allow me to explain my setup a little. I'm running an pfsense internal router for a testing environment at my company. I've got like 50 subnets and traffic is basically allowed to be routed anywhere, the real firewall work is many steps above me. We don't want to run in router-only mode because then we lose traffic logging, which our security team wants to consume. I created a floating rule with everything allowed and quick action checked. .
So what's wrong? Well things are mostly fine, but some are odd. Like despite a pass everything floating rule, I'm still seeing a bunch of blocked traffic hitting the default deny rule. It all looks like legitimate traffic, most showing TCP:A as the protocol in the logs. I saw some traffic logged as going out my uplink connection, despite being an internal subnet. Like it's just weird and I can't figure out what if anything I'm doing wrong. I'm wondering if anyone else has had strange behavior from floating rules before?
EDIT: Oh, I'm using aliases to group a bunch of non-continuous IP spaces into simple rules. Is that maybe putting me into some weird edge case?
EDITEDIT: Oh, I just thought of one last thing. I'm using a LAGG to link to our switch. If there's a mismatch there, would that maybe cause weird behavior like this?
-
Can you show us any specific examples?
Seeing TCP:A traffic blocked can just be because the state closed and some host is still sending traffic. Not necessarily an issue.
Do you actually see any connections fail you expect to be passed?
See: https://docs.netgate.com/pfsense/en/latest/troubleshooting/log-filter-blocked.html
Steve
-
@stephenw10 Unfortunately because work stuff I can't really share too many specifics.
To your point about dropped TCP:A traffic not necessarily being an issue, I'd agree except that I've done everything to make it go away and it's still happening. I have the firewall set to conservative and I have allow IP options and all TCP flags checked on my allow all rule. Should that not catch things like this, or am I not understanding something?
Seems like everything is working as expected, so no major complaints.
-
You'd need sloppy states set too if you actually want to allow out-of-state TCP packets. But you probably don't. That traffic should almost always be blocked.
-
@stephenw10 Does that mean that the TCP flags setting doesn't do anything unless I set sloppy states?
-
It doesn't do nothing but otherwise pf tracks tcp connections and expects to see them in the correct order. If you've got some route asymmetry or similar you have to stop it doing that to pass them.
But you almost certainly don't want to pass those!
-
@stephenw10 Well that is unless you're doing what I'm doing and trying to make a router that logs traffic. And there is some asymmetric routing going on in my environment by design. We're testing out various network security tools and inventive routing is sometimes part of that.
I've found what's either a bug or an undocumented requirement of using the TCP flags feature of a rule. If you create a rule (floating or otherwise) with the protocol set to any or TCP/UDP and then check TCP flags any, the rule will actually show up as flags S/SA in pfctl. Only if you set the rule's protocol to TCP will the TCP flags feature have any affect.
-
It actually shows that in
pfctl -vsr?Interesting. And, yes, if you intentionally have asymmetric routing you will need that. Or add stateless rules both ways.
-
@stephenw10 said in Problems with all floating rules setup?:
It actually shows that in pfctl -vsr?
Yes, definitely. I got into work this morning and added a TCP rule with flags any above my any protocol rule and it immediately resolved all the blocked traffic I was seeing.
I can understand this was maybe done by design? Like I see the logic in "if they want to do weird TCP flag stuff, they should only be doing it in a TCP only rule." I just wish it was documented in the UI, or at least the docs site. Would have saved me a lot of head scratching. Reading through the pf documentation, I can't seem to find anything saying you can't do a flags any with proto any. So could be a bug or a similarly undocumented "feature."
In the end it turns out that it had nothing to do with floating rules, that was just where I noticed it. For me this really about explaining the behavior that wasn't making sense to me.
What's the procedure for reporting this? I've never had an issue I've actually needed to file before.
-
Open an issue for it in redmine: https://redmine.pfsense.org/
I don't see anything open for it currently. Which is surprising.
-
@stephenw10 I'll get that submitted tonight. Thanks for talking through this with me.
