is allocate more CPU resources to Captive Portal possible?
-
Is it possible to tweak the amount of resources allocated for the captive portal? I would like to allocate more if possible.
We are working with a venue which can support up to 50k users but 24-30k(non covid era stats) on average is seen during events. In the event break, we see between 3-5k users logging on to the network in as little time as two to three minutes. Stats is from a Cisco WLC.
Due to limited resources(covid/event industry), we have had to make due with a bit older hardware(their old vsphere cluster hardware), We have 64 cores and more ram than we would ever need, distributed across 3 separate servers running pfsense (no virtualization)
Each server has a associated VLAN and clients are distributed equally across vlans by the Cisco WLC.
Captive portal auth is simply a checkmark and pressing login (nobody ever downloads the terms of service ofc)My worry is not dealing with client traffic (99% smart phone traffic) after clients has been through the captive portal, but we do see a pretty good spike on the CPU side when they initially have to associate and get authorized.
We have gone as far as possible with minimizing graphics and such, to avoid the nginX having to server large files etc, but i was wondering if there was a way where i can assign more cores/resources to the captive portal nginX process (or if this is done dynamically).. if its done dynamically, maybe i can lift up the default assigned amount of resources so there is a bit more power from the start?
I hope my questions is detailed enough, but do feel free to ask for any information and i will try to provide something.
All the best.
-
@ketil Hi,
-
The nginx config for managing the captive portal is located at
/var/etc/nginx-{$cpzone}-CaptivePortal-SSL.conf(andnginx-{$cpzone}-CaptivePortal.conffor the non-SSL webserver) -
This config get automatically updated (& nginx get restarted) every time any update is made on the captive portal configuration. The code managing this update can be found here :
https://github.com/pfsense/pfsense/blob/RELENG_2_5_1/src/etc/inc/system.inc#L1235. -
There is an nginx variable,
worker_processes, which defines how many nginx workers should run in parallel. Each worker is running on a specific core. By default, this variable is set to 6 workers for the captive portal, but this can be adjusted by changing Max Processes in the advanced system configuration. (The number of nginx workers for the captive portal configuration is defined as such : number of nginx workers for the GUI + 4) -
Each worker can have up to 1024 simultaneous connections (Because
worker_connections 1024is hardcoded here). -
Depending on how you change these settings, you may have to update some turntables (i'm thinking specifically about
kern.maxfilesandkern.maxfilesperproc, which are the FreeBSD ulimit).
Also, i would like to warn you about tweaking these settings: "Higher is not necessary better". By increasing the number of simultaneous connections, you may run into other bottleneck and slow down each connection...It is preferable to serve 1k simultaneous connections in 1 second & then to serve another 1k in the next second, than to serve 2k simultaneous connections in 4 seconds.
-
-
First of all, i want to say thank you @free4 for the very nice and detailed explanation.
We ended up going through a month of events with a peak of concurrent 15k users (where usage segments was longer than 10 minuts)
We luckyly never went above 2gigs in upload and 1 in downloads, so it was good, and i feel confident we will be able to easily deal with around 30-40k concurrent users in a worstcase senario. Updates will properly be needed for 40+, but a budget has been allocated.
We experienced most load around the joining processs with the webserver having to deal with the captive portal visits. But we never moved towards a load which would have been considered critical.
have a great day and thanks !
-
@ketil Woow.
Out of curiosity, what are the hardware specs of your pfSense server?
-
I will post the specs shortly :)