NAT, port forwarding and the C10k problem

[ovidiu]

Hi,

I'm working on a solution that requires a lot of persistent connections from many devices (web browsers and smartphones) to a web server. This is also known as the C10k problem (http://www.kegel.com/c10k.html), though I'm shooting for a lot more than 10,000 open connections, perhaps around 500,000 connections spread across multiple virtual machines.

Most of the time these connections are idle, and the messages being passed back and forth are relatively small (less than 10KB). Right now I have a pfSense firewall sitting in front of the web servers. I use NAT and port forwarding to forward traffic from the Internet to the web servers.

My question is whether I would be able to handle such a configuration and what are the parameters that I need to tune to achieve so many connections. How can I tune the memory on the pfSense side so it only uses a small amount for each connection?

Thanks,
Ovidiu

[cmb]

There's no tuning involved, just making sure your state table is adequately large. 500,000 connections isn't that many, many installs run more than that.

[ovidiu]

Do I need to lower kern.ipc.maxsockbuf from the default of 4MB, or other params to make sure the memory usage stays under control? On Linux for example I adjust net.ipv4.tcp_mem and friends to limit the memory allocated in internal OS buffers for sockets.

[jimp]

That would be for connections terminating at the firewall/router - these are simply passing through.

You might need to worry about such things on your server, but not the firewall.