Can't get more than 10k connections on an IP - Resolved – see 4th post

[tomv]

I'm not sure which gets more traffic, so this is posted on the mailing list as well.. sorry if this against any forum rules  :)

We are running pfSense v: 1.2.2  and running ejabberd and we are unable to have more than 10K connections to the same IP.
(When this happens, I can still connect to other IPs on the same firewall so it seems to be a per IP limit.)

While searching for the settings, we found the following:

vmstat -z

ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES
.
pfsrctrpl:                124,    10013,     9803,      210,   592703,   332183
.
.

pfsrctrpl seems to be our issue.. What is this and how can we change it?

Advanced Options on the rule we are having trouble with is blank so it should be be used.
We've tried setting it to 15000 and that didn't make any difference.

Firewall Maximum States: is set to 100000 and we also tried to change it to 200000

Any ideas?
thanks,
tom

[tomv]

pfsrctrpl seems to translate to src-nodes

pfctl -sm

states        hard limit  200000
src-nodes    hard limit    10000
frags        hard limit    5000
tables        hard limit    1000
table-entries hard limit  100000

I can change the src-nodes limit by editing pf.cfg with the following
set limit { src-nodes 23456 }
and then running
pfctl -f pf.cfg

this shows the following changes
src-nodes    hard limit    23456
and
pfsrctrpl:                124,    23467,    2635,    7378,  650614,  336039

but I lose access to the firewall.
When I reload the firewall it resets the src-nodes

In addition to the above, I added the following line to the top <system>section of the /cf/conf/config.xml
<max-src-nodes>23456</max-src-nodes>

but it still didn't work.</system>

[tommyboy180]

the file you need to edit is /tmp/rules.debug but this file is regenerated upon every fw change.
If you don't change the FW then just enter your limit here.

[tomv]

This seems to have worked.. I'll report the results after we see our traffic rise to more than the 10K we were blocked at.

Chris' reply to the mailing list:

Edit /etc/inc/filter.inc, find these two lines:
      $rules .= "\n";
      $rules .= "set skip on pfsync0\n";

above those, add:
      $rules .= "set limit src-nodes 23456\n";

or whatever number you want it to be. Save changes, edit and save a
rule and apply changes to kick off a filter reload.