Recommendations for a lot of simultaneous connections

highpec

Hi,

I work a lot with proxies, scanning them and other things. I have high simultaneous connection load at most times and my WRT1200acv2 with DD-WRT, although a decent router, gets really bogged down at around 15,000 connections. One thing that helped immensely was setting the TCP timeouts lower, in particular the TIME_WAIT timeouts.

My question is, what causes this, and what sort of machine could I run Pfsense on that would allow me to have 30k+ simultaneous connections without feeling the impact too much? Pfsense can set advanced TCP timeouts in the same way that DD-WRT can? (SYN, TIME_WAIT, etc?)

WRT1200ac v2 specs
Dual core 1333mhz
512mb ram

Here's the weird part, the CPU usage never goes above 30-35%, and the RAM is always 90% free! That being the case, I can't figure out what causes the router to perform so poorly under a lot of connections. Is it the DD-WRT firmware that's the issue?

Let's say I build an inexpensive little machine, a quad core perhaps (Pentium G4560 3.5ghz? Or perhaps something lower like 2.0ghz with lower TDP?), with idk, 8gb ram? What sort of connection numbers could I hit? Is it the speed of the processor, even at low CPU usage, that causes it not to be able to handle a lot of connections? I've researched this for probably a hundred hours and cannot find a clear answer.

Note that my actual Mbps speed rarely reaches above 50mbps, although I have seen it jump to 100 mbps sometimes, but it's rare. It's usually the amount of connections that causes the router to be sluggish. For reference, I have 150mbps Internet and unlimited data per month.

Thank you for the help!

stephenw10

30k connections should be no problem at all for anything even vaguely recent.
With 8GB of RAM the theoretical state limit is going to be close to 8M. But you will be using RAM for other things. That is also firewall states and each connection opens states on LAN and WAN so half that for connections. Also if you actually ever opened 8M states you'd find some parts of the gui difficult to use! The actual firewall should function though.

Seeing 100K states is not unusual. If you do you see that be sure to check 'Require State Filter' in System > General before you try to view the state table.

Steve

SteveITS

I don't know much about DD-WRT but we have run into instances with lower end routers not handling lots of connections. I think some just have a fixed size state table. The first was a LONG time ago when we starting having our clients' PCs connect in to our management service. We switched to m0n0wall (and then later to pfSense) on an old/spare PC and it cleared right up. A couple years ago we ran into it again at a client with a mid range (for D-Link) D-Link router who had about 5 PCs and 10 phones...the router would just stop passing traffic and you couldn't connect to its web interface. We've since just given up on D-Link type hardware for more than about 5-10 PCs/devices.

Currently our traffic goes through an SG-3100 for our building an then an old cast off PC we use that runs Suricata. My point is the hardware is likely not limiting your connections and you should NOT need shiny new hardware for pfSense...most likely some sort of limitation in DD-WRT.

The only limitation for pfSense moving forward is that v2.5 will require AES-NI CPU support...so about 2012 or later CPUs if I recall correctly.