Hardware for 500+ users connection
-
Hi.
A customer asked me to develop a public wifi area, with a captive portal with users authentication, capable of ~500+ users connected simultaneously, with a database of 20.000 users on an external server. the captive portal will authenticate users on this external db and allow the connection to browse.
on the wifi side, i'll use the ubiquity uniFI hw to create the wireless area, but i'm not sure about wich hw i'll need for a pfsense server capable to handle all this connection.
actually the problem is not the budget, but the sizing of hardware.
i was thinking to something like
Intel
Xeon Processor E5-2630 (15M Cache, 2.30 GHz, 7.20 GT/s Intel QPI)
32Gb Ram
4 HD 10.000 rpm in Raid 10 configuration
on the wan side i'm thinking at at least 3 adsl 20mb or something better like VDSL of OPTICAL FIBER.can you help me with your experiences?
thank you. -
That's 20K users right? Coz you typed 20.000. If the authentication verification goes to an external server then you don't need to worry much about pfSense CPU power. Just posted on another thread about CPU power with real life experience.
First ensure you have enough Ubiquity unFI to be able to handle or balance 500+ concurrent users. I see that as a major bottleneck.
Secondly, 3 adsl 20Mbps ( I hope you meant Mbps and no MB) lines is like 60Mbps. With 500+ users I see that as another bottleneck. Go for optical fiber if you can.
Third, is this pfSense going to be just a simple firewall or are you planning to add IDS and other resource intensive packages? CPU cycles get gobbled up with too many user requests being processed by IDS and other apps. If its just simple firewall, then your hardware config is perfect and will last for a good amount of time. You might even be able to service them with 16GB RAM if there is no other package installed. -
A customer asked me to develop a public wifi area, with a captive portal with users authentication, capable of ~500+ users connected simultaneously, with a database of 20.000 users on an external server. the captive portal will authenticate users on this external db and allow the connection to browse.
You'd better post CP-related questions in the Captive Portal sub-forum.
If this is going to be an open Wifi hotspot (e.g. for a big event or conference) that "has to work", you'll have to do considerable testing in both the Wifi infrastructure and pfsense CP itself. UBNT Unify may be dirt-cheap, but they don't support 5GHz (except the latest model which seems to have some issues).
About 15 months ago, I did quite a bit of (simulated) testing of pfsense 2.0 CP and identified some bottlenecks and aspects where some "hardening" of the CP would be desirable, if it was to withstand abuse. I haven't yet tested the CP on the new 2.1-BETA based on FreeBSD 8.3 to see how it performs.
Depending on your users' needs, 3 x 20Mbps may not suffice. Also consider adding a transparent proxy.
-
thank you for your help.
this is going to be an public hotspot for a shopping mall main saloon capable of about 2500 customers.
the goal is to provide free internet access to share content on social networks mostly (more they share from there more the mall is known and more customers come)
for the internal network i'm going to use the ubiquity uniFI access point, the modell will be the UniFi AP Pro (UAP-Pro) capable of 200 concurrent connections, i'll use 10 ap to cover the entire place overlapping multiple ap's range to share the concurrent connection. each ap will be connected with 2 gigabit ethernet connection in lacp on a cisco managed switch. the lan server i'm thinking about using optical fiber, i don't know if pfsense support the trunking on the nic.
about the adsl probably we have no coverage for fiber, i can add a fourth 20mbps adsl.
honestly my concern are not about the speed of the connection, but about the handling of thousands on connections simultaneously (whatsapp, skype, facebook, push notification etc for each device).
i didn't say before, but beeing a public hotspot i'll need to log all the traffic generated for each account, so i'm going to use a lot of hd and ram i think. i don't need any filter and so i'll use squid just to log all the traffic and store it for some months.if you have experience about high density wireless area please share :)
regards.
-
for the internal network i'm going to use the ubiquity uniFI access point, the modell will be the UniFi AP Pro (UAP-Pro) capable of 200 concurrent connections
Well, that's the "latest model which seems to have some issues" I was referring to in my previous post (check the posts at the UBNT forums).
-
You've given two figures for simultaneous connections, 500 and 2500. Which are you asking for?
pfSense is good with large numbers of connections so you probably wouldn't have a problem either way.
The inner pedant in me needs to ask you to be careful with unit prefixes. mbps is not the same as Mbps, it's 1000000000 times slower! Yes everyone here knows what you mean and has done it themselves but one day you might accidentaly specify something very badly wrong. End of pedantry. ;)
Steve
-
The mall is capable of 2500 people, but we are sizing the hw to support up to 500 concurrent connection(1 person each 5 will use internet, the others i hope will enjoy in a different way :)).
You are right, the dsl connections are 20Mbps i'm sorry about my mistake, but the tablet is not the best tool to write long messages with the right cases and formatting:)Dhatz, I'm reading right now about uniFI issues with apple products, are u referring to that?
-
Not only that with the UniFi UAP-Pro but your gonna be VERY unhappy with them if your wanting them to support more
than about 40 people connected to EACH one….(i tested 6 of these in the Lab extensively and decided to NOT deploy
them due to ALOT of issues, have one warehouse that has WELL over 3k+ wireless devices attached to wireless AP's)then comes the next question... where are you gonna run the controller software for the UniFi's at?
software seems to take about 15 minutes to be able to use the GUI on pfsense (after pfsense IS booted)... and theres NO package
for it... so you get to roll your own. theres some guides on ubiquiti forum and im sure theres probably a few here on this
forum.were running Supermicro servers for our pfsense boxes with 24G of ram and 2 Quad Core L5420 cpu's with plenty of room
to spare processor wise. were running close to 80% utilization a GigaBit Ethernet Wan port in most of our DataCenters and
Offices/Warehouses. and we see ALOT of users and employees...were running squid as well but no IDS..
Good Luck.
-
@SunCatalyst:
Not only that with the UniFi UAP-Pro but your gonna be VERY unhappy with them if your wanting them to support more
than about 40 people connected to EACH one….(i tested 6 of these in the Lab extensively and decided to NOT deploy
them due to ALOT of issues, have one warehouse that has WELL over 3k+ wireless devices attached to wireless AP's)then comes the next question... where are you gonna run the controller software for the UniFi's at?
software seems to take about 15 minutes to be able to use the GUI on pfsense (after pfsense IS booted)... and theres NO package
for it... so you get to roll your own. theres some guides on ubiquiti forum and im sure theres probably a few here on this
forum.were running Supermicro servers for our pfsense boxes with 24G of ram and 2 Quad Core L5420 cpu's with plenty of room
to spare processor wise. were running close to 80% utilization a GigaBit Ethernet Wan port in most of our DataCenters and
Offices/Warehouses. and we see ALOT of users and employees...were running squid as well but no IDS..
Good Luck.
Hello Sun,
so what did you use for wireless coverage, since you didn't deployed the uniFI ap's?
based on your experience i think that for the pfsense server we are ok, maybe it's also "too powerfull" and that was why i was thinking, in case i won't be confident about uniFI sw on pfsense, to modify the server to host 2 vm, 1 for the unifi sw and the other for the pfsense platform, using ESXi Hypervisor, in this way should be also easier handle the nic trunking.thank you for sharing with me your experience :)
-
@SunCatalyst:
Not only that with the UniFi UAP-Pro but your gonna be VERY unhappy with them if your wanting them to support more
than about 40 people connected to EACH one….(i tested 6 of these in the Lab extensively and decided to NOT deploy
them due to ALOT of issues, have one warehouse that has WELL over 3k+ wireless devices attached to wireless AP's)That's a common issue with most access points. I'm using UniFi APs (LR, not Pro) at my office and haven't had any issues, though I've only got about 100 devices connecting to them (the bulk of these on 2 APs, around 40 & 40, with the remaining 20 spread across 8 more APs in 3 buildings).
The solution is simply using more of them, closer together, at lower power settings. These APs are so cheap it doesn't really matter. 500 people spread throughout a mall (focused at high traffic areas like the food court) should be a pretty easy project for Unifi APs, particularly once they hit 3.0 and have seamless transitions between APs.
-
so jason, it should be fine having 10 ap pro for 500 users… should be 50 for ap on the "top level" device of ubiquity.
talking about the captive portal, i think i will need to authenticate on a webpage hosted on the uniFI server, on iis server, having a freeradius server authenticating on a ms sql crm database and the registration form writing the record on the crm db.
the only point still to be decided is how to validate the information submitted from customer, if with an sms, or making a call to confirm the phone number you gave or just with an email (but u must disconnect from the wireless to receive it not beeing authenticated)...
i will share with you the next steps of this project, hoping to give you some good information and receive also your experience on it.
thank you! -
That's a common issue with most access points. I'm using UniFi APs (LR, not Pro) at my office and haven't had any issues, though I've only got about 100 devices connecting to them (the bulk of these on 2 APs, around 40 & 40, with the remaining 20 spread across 8 more APs in 3 buildings).
The solution is simply using more of them, closer together, at lower power settings. These APs are so cheap it doesn't really matter. 500 people spread throughout a mall (focused at high traffic areas like the food court) should be a pretty easy project for Unifi APs, particularly once they hit 3.0 and have seamless transitions between APs.
It depends on what the WLAN clients will be doing and what the morphology of the place is (among other factors). If the majority of those 40 clients per AP are just associated with the WLAN but generate minimal traffic (e.g. smartphones checking email), and the morphology creates some sort of natural isolation between RF sources, then you might be able to use UniFi-Pro APs.
If, on the other hand, it was for a WLAN at an event where some users need to upload videos, or a city hotel where guests would want to get actual work done (e.g. connect via VPN), or a warehouse with 100s of devices, you'd probably have to do some smart RF engineering or buy smarter APs, ideally a combination of both.
-
Now were using Cisco 1252AP's with Regular IOS (12.4 NOT 15.x) not the one that requires the Controller software.
there Brutally Fast , Cover a BIGGER area (twice the coverage of the UAP-Pro) and just RELIABLE…
we have them Mounted in the ceilings.ours run a few versions back from Current version of IOS as the release
were using is just RELIABLE and cant afford ANY downtime.some of our AP's have 200 (approx) devices associated with them but allowed to roam from AP to AP. Cant do
that with ubiquiti software now... they CLAIM next version... ya right... ive heard that for MANY MANY
releases of ubiquiti software updates.understand we do have Wireless links in Some of the buildings via ubiquiti hardware on their AirOS software.
5.3.3 is the latest that works the best performance wise, even though 5.5.3 is the latest (even though
5.5.4 is in RC release)... these are on the chopping block to be replaced by Fiber instead in the coming
month as soon as Fiber shows up to be installed due to 100Mbit ethernet limitation even though they CLAIM
170Mbit realtime performance on the wireless side. (were here before me or they would have Never been
deployed)if your looking for Reliability with a Features that work NOW , these AP's rock.
if your looking for CHEAP that works for some people and you want headaches , Ubiquiti UniFi is ok.the UAP-Pro's are $200ish each, $300ish for the Cisco's.
NOTE: Cisco GUI sucks Horribly. its slow and Clunky....I build all the configs initially in the CLI editor and push them to the AP via
tftp and then deploy.NOTE 2: Unifi Controller software is in Java as well... seems to work ok, but not being able to have downtime, 15 minute wait time for
the software to work on pfsense ISNT acceptable for US.Good Luck.
-
Have been using the Cisco 1142 dual band APs for over a year. Love the APs. Rock solid. I recommend using Cisco 1140 series rather than Ubiquiti.
They have come down in price these days and easily deployed.
-
the 1142's are Rock solid as well.. we ran those before moving to the 1242's (after testing
the UniFi's in the lab and then the 1242)1142's can be found CHEAP these days though there only A/b/g AP's… but there RELIABLE
as all get out.. -
Unless I'm missing something, most of the database work will be on another box, and there is no VPN here right?
I wouldn't go with an LGA2011 CPU for that build. An i3 can push like 80mbps of VPN throughput, and can do wire speed gigabit routing. Ditto on the 32GB of ram as it's also overkill unless you're running PF inside of a VM on the same machine. You may also run into trouble with the RAID configuration as I know the ICH10 has trouble with AHCI unless you use the beta snapshot of Pfsense (or so I'm told). I'd put some of the extra cash towards more quality external access points and more PCIE slots on the motherboard side.
EDIT:
The 32GB of ram might not be the worst thing if you go crazy with Snort and Squid, but the LGA2011 still is overkill IMO.
