Recommend specs for the caching for 100,000 and thousand user

[jhayvee]

hi me again

thank you heper, Nachtfalke, Supermule for replying me for my question, i have a another question below i need your suggestion

what is your recommend for the specs of the server for the 100,000 users and settings for cache management?

and what is your recommend for the specs of server for the thousand of users and settings for cache management?

here is my specs on my server

Intel(R) Xeon(R) CPU X3430 @ 2.40GHz
4g ram
1 TB hard disk

[stephenw10]

The maximum throughput of your machine is very important here. What is the speed of your WAN connection?

Steve

[Nachtfalke]

This should help:
http://wiki.squid-cache.org/SquidFaq

Seeing the big HDD you should have a look at this:
http://wiki.squid-cache.org/SquidFaq/SquidMemory

[jhayvee]

@stephenw10:

The maximum throughput of your machine is very important here. What is the speed of your WAN connection?

Steve

HI Steve

our total throughput on WAN is 150MB

[Nachtfalke]

I found something additional on pfsense page:

Large state tables - State table entries require about 1 KB of RAM each.  The default state table, when full at 10,000 entries, takes up a little less than 10 MB RAM. For large environments requiring state tables with hundreds of thousands of connections, ensure adequate RAM is available. 

so:
10.000 states = 10MB
100.000 states = 100MB

If you have 100.000 users and every user will have 10 states then you will get
1.000.000 states = 1000MB

So for handling the states you need at least 1GB RAM

For handling 1GB HDD you need 14MB RAM
You will perhaps use the 1TB=1000GB HDD which will cost you:
1000 * 14MB = 14.000MB = 14GB RAM

Then you need RAM for pfsense itself and other processes and you need RAM for network traffic.
So I would suggest you that if you want to use all your HDD you need at least 16GB of RAM - better would be probably 24-32GB or you decrease the HDD size to lets say 500GB.

Then you will use some memory to cache "hot objects" with squid.

150MBit/s should be possible with the CPU you use if we could belive the hardware sizing guid on pfsense page.

[stephenw10]

There was a similar thread recently, here: http://forum.pfsense.org/index.php/topic,54475.0.html

In it a real figure for states per user is given as 120. In that case you'd need MUCH more ram.  ;)

Steve