Advanced Options - Multiple State / Connection Controls Not Working

[ky41083]

It seems like this has been working on and off throughout beta 2.1 snapshots, the last month or two at least, not working at all…. I figured it was an issue with my rules (as it typically is) but I know it was absolutely working at one point with the exact same rule set, and now it is not showing any signs of working what so ever. My upstream router is consistently showing 0/1024 sessions available. In the past it would cap off between 256-512/1024 sessions available, depending on the number of active torrents in uTorrent.

Issue: Set "Maximum state entries this rule can create" on any LAN interface rule (I set this on my BitTorrent rule and my default outbound rule, see attached image) and the value is not respected.

Expected behavior: Any traffic matching this rule would be limited to the number of states (basically sessions, at least one state per session) you set this value too. Even if traffic is finding a way around the BitTorrent rule, the default outbound rule should still catch it.

Actual behavior: States / sessions are not limited at all, even on the default rule, and allow lan side hosts to quickly exceed the session limit on any upstream devices (like currently deployed AT&T VDSL routers that have a 1024 session limit, or a similarly extremely low global limit).

The reason (yes, I know someone will say "just do it the other way") that I do not just set the global session limit of pfSense, should be fairly obvious in my rule descriptions. With the firewall hosting many internal LAN only services like DNS, NTP, DHCP, WoL, etc., that have roughly 512 sessions up in the air at any given time for strictly internal services, I specifically do not want these internal services effected / interrupted / broken simply because the upstream internet router I am forced to go through and have no control over completely blows. With these "Advanced Options" for specific rules we should not have to impose limits on everything, when we only need to limit a few very specific types of traffic.

And yes (I know this will also be said), a lot (90%?) of BitTorrent traffic is now UDP, and yes, UDP traffic is considered a "StateLess" protocol, but if you go look at your pfSense States table, there are a fair amount of UDP entries. Why? Because firewalls, routers, edge devices, etc., still need a way to track the "back and forth" of data streams no matter the protocol, and as such still keep state table entries for all protocols, whether or not the actual protocol itself relies on the state of these backend state table entries.

So, as long as the traffic in question matches the rule, the state limit value on that rule should be applied to it, and limit UDP "states" as well. If you want more evidence to this fact, take the state table from the upstream device in this case, every single UDP "session" counts towards its global limit of 1024 sessions, and you will find this true on any device that exposes its true state table.

[wallabybob]

What is the state timer in the upstream router? Perhaps pfSense expires "idle" states before the router.

What does pfSense think is the state count and how does that compare with the session count in the upstream router? Console/ssh shell command```
pftop

Please post the contents of pfSense file /tmp/rules.debug. (after masking IP addresses if you consider it necessary.) Perhaps the GUI isn't translating the maximum state entries attribute to the appropriate pf rule option.

[kejianshi]

I have had very little success with max state entries or connections per unit of time.  Usually, it just sort of shuts down my connections.  Thats on 2.03.

[ky41083]

State timer in the upstream router matches exactly the TCP and UDP session timeouts on pfSense. Had to make some sysctl entries to get it to behave that way, but yes, thought of that too when it wasn't working.

[ky41083]

Also, while I'm on that train of thought, I have also tried every other state limiting option and the established connections limiting option under "Advanced Options" and either all of them seem to have an effect (with the older snapshots) or none of them seem to have an effect (with the latest snapshots, at least a months worth or so).

[kejianshi]

The newer ones have an effect for me.  They break my connections rather than limiting them.

[ky41083]

The absolute only thing I have had consistent success with, if you can call it that, is globally limiting the number of sessions. But as I said in my first post, this disrupts LAN services, especially internal DNS.

[ky41083]

Here is a copy of my /tmp/rules.debug, IP's masked, quickly proofed, IP masks should be consistent.

It does indeed appear that those limits are not being applied to any rules.

Edit: With those values in all 4 fields (Maximum state entries this rule can create, Maximum number of unique source hosts, Maximum number of established connections per host, Maximum state entries per host) this is the result from /tmp/rules.debug:

pass  in  quick  on $LAN inet from 192.168.1.0/24 to ! $PrivateIPv4  label "USER_RULE: Session Limit Internet Traffic"

Same is true with the rule on the WAN interface, no additional parameters appear.

rules.debug.txt

[kejianshi]

Its strange that it applies to nothing. 
Whenever I've attempted to apply a limit to the port associated to the port that forwards to my mjpeg tilt/pan cameras, it has effectively killed the port.  I've applied connection/second and max states and both have had an effect that feels like a DOS attack. The ports just seem to work rarely when I apply limits.  It must be affecting some parameter somewhere.

[kejianshi]

I've actually checked this.  I have access to computers all over the globe (I do some admin here and there) and I have checked the effects of those settings outside my networks.  It makes a difference.  Not a good difference or expected difference, but a difference never the less.

[kejianshi]

Don't get discouraged.  Explain your findings.  I'd love to know why this isn't working as expected.

[ky41083]

Not discouraged, just waiting to see if a dev will notice and chime in. Will most likely create a bug report on the issue tracker if nothing that makes solid repeatable sense comes up first.

Clearly none of the values are being applied at all, not even in the wrong location, according to the /tmp/rules.debug log.

[kejianshi]

I also had same feeling as you, but I've been too busy to dig into the "why" of it the way you have.
I think that you are correct that its not working as advertised.
Do you have a proposed fix?  Can you see where the failure is?

[kejianshi]

Have you done a pfctl -sa dump into a file then apply a rule and pfctl -sa into a seperate file and compare the files?
See if there are differences?

[ky41083]

The failure is the script that reads / parses the saved options and turns them into PHP commands.

Yes, I could find it. No, I couldn't fix it without about a days worth of Googling for the correct way to parse / implement it on the command line and massaging it all together.

That is why I would just submit a bug in the issue tracker, the devs know far better than me off hand, what script specifically processes or is supposed to process these options, and the proper way to code in the commands to implement the options properly again.

For me, it's 2:35am and time to relax until tomorrow ;-)

[kejianshi]

I'll check and see what differences I see after another beer (-:  Then rest.

[ky41083]

According to rules.debug, after applying a Maximum new connections / per second(s) value of 704 per 5 seconds it, nor any of the other settings under the "Advanced Options" section appear, the rule remains as follows:

pass  in  quick  on $LAN inet from 192.168.1.0/24 to ! $PrivateIPv4  label "USER_RULE: Session Limit Internet Traffic"

Update: Running - pfctl -sa | grep "704" - at the command line returns absolutely nothing.

[kejianshi]

That is weird.  Must be broken.

[phil.davis]

filter.inc

	$noadvoptions = false;
	if(isset($rule['statetype']) && $rule['statetype'] <> "") {
		switch($rule['statetype']) {
			case "none":
				$noadvoptions = true;
				$aline['flags'] .= " no state ";
				break;

If the state type (another advanced rule option) is set to "none" then the "Advanced Options" settings on the rule do not apply. That is the only obvious way I see that the code can bypass putting max-src-states and similar into the pf rule.

[kejianshi]

I've tried this last night with beer and today without and got the same results.

So, this time I'm only applying 853 connections / 1 second then doing a filter reload.
Checked to make  sure its still "keep state"

pfctl -sa | grep "853"
pass in quick on nfe0 reply-to (nfe0 78.22.113.1) inet proto tcp from any to 192.168.13.1 port = 49132 flags S/SA keep state (source-track rule, max-src-conn-rate 853/1, overload <virusprot>flush global, src.track 1) label "USER_RULE: NAT 49132 Stunnel to Foscam"

It is making changes to PF on my version.  If same thing in 2.1 doesn't end up in PF rules it must be broken.
Mine is 2.03</virusprot>