Pfsense in a high traffic environment

alanon

I work for a company that is an Internet search engine. We currently have 4 pfsense (running on dell PE1750's) and a Watchguard firewall. We load balance our crawlers across the firewalls, not only for performance reasons but also do have them appear to come from different IP's, we have multiple IP blocks.

What I am wondering is are there any tweaks for this type of traffic, each firewall averages in the 20-30Mb range constantly, with spikes to 40Mb (download traffic). The config I use is just the default configuration, the only additional rule is to allow pings from our monitoring solution.

Any suggestions would be greatly appreciated.

Perry

Are the servers stressed in any way?
Just a wild guess.
System -> Advanced functions
Firewall Optimization Options
@http://www.pfsense.org/index.php?option=com_content&task=view&id=40&Itemid=43:

State table optimization options - pf offers four options for state table optimization.

* Normal - the default algorithm
* High latency - Useful for high latency links, such as satellite connections. Expires idle connections later than normal.
* Aggressive - Expires idle connections more quickly. More efficient use of hardware resources, but can drop legitimate connections.
* Conservative - Tries to avoid dropping legitimate connections at the expense of increased memory usage and CPU utilization.

http://www.pfsense.org/index.php?option=com_content&task=view&id=52&Itemid=49

alanon

As an individual firewall approaches 40Mb we seem to get more timeouts. We have a full gig to the Internet so bandwidth is not an issue. The CPU and RAM usage don't seem to be the limiting factor, so my guess is either the NIC's (integrated Broadcom Gigabit) or a setting in pfsense.

Guest

You may also want to check your firewall max states and make sure you aren't close to exhausting them. If so, raise them. System -> Advanced.

alanon

I just looked at the state table and one of them was at over 17,000, the two that I would expect to be the highest had no value listed. Newbie question: Can having the state table value too low severely impact performance? With that firewall at 17,000 and considering there are two firewalls that easily to 5-7 times the traffic of that one, I am assuming that I may need to increase the table size from default (10K) to maybe 100K? (we have 4Gb of RAM in these firewalls.)

Guest

Yes, a low max states will absolutely impact performance. You have plenty of memory so I would encourage you to increase your max states. Each FW state will utilize approximately 1-3k of memory so you have plenty of space to expand.

cmb

When you run out of states you can't open any more connections, so yes, it will cause connection failures. With 4 GB RAM, I'd up it to at least a million.

alanon

Thanks, I appreciate all of the feedback. I upped the value on all of them. One of them went from 18K up to 32K pretty quickly. I am checking with the developers to see if this eased the timeout problem.

Thanks again.