How Far Have You Scaled Your PFS Box?
-
I'm just personally curious. I've been using PFS off and on for a few years now, and for the past 8 months have been running it at home in an old Celeron 1 GHz tower with 4 NICs, 2 of them WAN's, one of them my LAN, and the last interface provides bandwith to a couple AP's for about a dozen or so people on my street. (see www.socalfreenet.org)
I've enjoyed watching the project mature over the years, and am about to put my first PFS box in a commercial application. Nothing serious, just Dual WAN failover and basic firewall services for a small office.
That being said, I'd like to open the floor to discussion on scalability. How far have you pushed your PFS? How many NICs, what CPU, how much RAM, what kind of bandwidth, how many users are you supporting? Got any photos?
And of course, my apologies if this has been done before.
-
I'll start small and let people go from there…
We have 30 people behind a P3 1.7ghz running squid attached to two bonded T1s (3mb/s). Two 3com 905 NICs.
-
Just one out of many examples I have:
We have a 266mhz firewall running all of the pfSense servers 5+. Our servers take a royal pounding and are always pushing ATLEAST 1.5 megabit. It spikes up and down all day to 10 megabit (limited by network interface 10 megabit link).
Never had any issues and has been running strong for many years.
-
I have a 100meg link to the internet Pfsense is setup in a small lab network off of this. Have 3 servers behind it. Two linux, and one Windows box. Below you can see my bandwdith use since starting pfsense 1.2 beta1. The only time I have had trouble is with Tor. It seems it can be a little hard on connection use. It will kill a linksys good! Only problem that I have run into is I need to clear the state table sometimes. I blaim Tor for this however, it should close them a little faster by default. I raised the number of states to 20,000 for my little box.
Beta's http://stashbox.org/77126/pfsense.bmp
1.2 release http://shup.com/Shup/37288/108319191050-gw.adotnet.net-Status-RRD-Graphs-Windows-Internet-Explorer.png
Every time I have had a lockup it has been HARD. Had to power cycle pfsense box but it has always back come up. 1.2rc4 has been the most stable to date for me, it has yet to crash. Going to load it down hard and let tor just go nutts in my little network.
Forgot to add that I use a Via 1ghz mini-itx board, 256megs ram, IDE solid state drive.
-
WOw, that's some bandwidth :o BUt err….what happened in October? lol
-M@
-
I have more than 30 pfsense runing at my clients office or datacenter. From 1 to 100 mbits. Every site has a two nodes cluster, built on HP proliant system or IBM Xseries, form 100 to 5 000 users behind the boxes.
First machine has been started in october 2005 (v0.69 I don't mistake myself).
All systems are runing basic features (IPSEC, pptp, captive portal and multi wan). For example, I have a cluster protecting medical applications (9 J2EE clusters) processing data for 750 000+ patients.I'am starting a new project wich will use the squid+squidguard packages in addition to the other basic features, in this project I will deploy pfsense boxes all over the world (europe,us,asia,africa), that's why some of you have already heard me speaking about a management host on the IRC channel. On another project, I am going to change openBSD based pf firewalls to pfsense (European country military context).
I love pfsense and its developers team. The project is awesome.
-
1. Firewall for 100+ staff, and 50+ public wireless. Load balances 2x 6MB DSL connections, handles incoming PPTP and IPSEC (just a handful).
-Running on a Dell Poweredge 1550, Pentium III, 1Ghz, 512MB RAM.
-CPU is never more than 2% except for spikes, and RAM is never more than 20%
-Solid as a freaking rock.2. Firewall for IIS web server. Serves streaming media on a 100MB internet connection, sometimes topping 40MB/sec. Many thousands of hits per day.
-Running on a Dell Poweredge 1650, Pentium III, 1.13Ghz, 512MB RAM
-RAM is never more than 30%
-Also solid as a freaking rock. -
i work for purdue university,
we've been using it for about 3 or 4 months in testing. I have some pictures ill upload later of our setup. We setup a failover system with 4 gateway e1600 systems (2 on one subnet and 2 on another) with about 2556mb of ram, 30gb hd, 1ghz or so processing. We have not put the 200 plus machines behind the firewall yet but we'll see how far it can go. We took the subnet given to use by the university and split it into our local lan of /22 (about 1000 machines or so support).
We use another e1600 for a windows 2003 server ias server (radius) and another e1600 for our GHOST.
We use NUT to manage our two UPS for the machines (APC Smart UPS 1500). We tested it today and it worked, it shutdown once the battery got to very low status (took about 2-3 hours to do)
We have not gotten captive portal to work with mac authentication yet but that will be a future testing phase.
We use the DNS, OpenNTPD, PPTP, DNS, DHCP, Carp, and a few other services i cant remember. Theres only a about a 10% load max on the machine!
we are still playing with squid to try to get atransparent proxy working as well to help keep the T3 bandwidth usage low.
so yea its been a pretty intense and eventful testing. PFSense works great and we are very pleased and appreciate the community for the program.
-
We use it at a wireless community setup as firewall/gateway to the internet and also in some cases as wireless client/ap router.
The internet box is a Compaq Deskpro SFF (PII 400Mhz | 512MB RAM | 80GB HDD | 3 x Intel Pro 100 NIC's), manages 6Mbps/512Kbps PPPoE ADSL connection, soon to add a 2nd ADSL connection… It runs pfSense with squid and lightsquid and provides access to internet to 30-40 users.
It's been rocksolid for months, specially after the upgrade from 256MB to 512MB of RAM.
As a wireless client/ap router we use the same Compaq boxes with atheros cards and only 64MB RAM and booting from CF cards. The boxes provide access to the wireless community network and internet through their wireless card (WAN) and basic NAT/firewall/DHCP to the user's home network. Sometimes a 2nd atheros card is added to provide wifi in the area. Also works fine, although recently we've been using beta versions of m0n0wall to this function due to lack PC100 SDRAM to this boxes, and m0n0 works better with 64MB in this setup.
Overall, m0n0wall/pfSense are great projects, we've been using them for 3 years in this network.
-
We use it at a wireless community setup as firewall/gateway to the internet and also in some cases as wireless client/ap router.
The internet box is a Compaq Deskpro SFF (PII 400Mhz | 512MB RAM | 80GB HDD | 3 x Intel Pro 100 NIC's), manages 6Mbps/512Kbps PPPoE ADSL connection, soon to add a 2nd ADSL connection… It runs pfSense with squid and lightsquid and provides access to internet to 30-40 users.
Ya, I used to run Squid, until I got the 2nd internet feed (I've got one 6 Mb/s DSL line, and one 15 Mb/s cable line load balancing and failover) but found out the hard way that squid doesn't work in dual wan mode. Actually, its seems most add-on packages break when you add a 2nd gateway. But forced to choose between 21 Mb/s combined bandwidth and squid, I'll choose 21 Mb/s lol ….tho I do miss tailing the squid logs and watching the random URL's go by. Maybe I'll get a 2nd box for squid....who knows, I could always use a higher power bill :)
-M@
-
Ya, I used to run Squid, until I got the 2nd internet feed (I've got one 6 Mb/s DSL line, and one 15 Mb/s cable line load balancing and failover) but found out the hard way that squid doesn't work in dual wan mode.
My goal with 2nd WAN is to make all high priority traffic (http/dns/pop3/voip) go through the WAN1 and all other traffic go to WAN2. I'm not trying to aggregate bandwidth or do failover, just simple routing policy, and really need squid/lightsquid statistics. Shouldn't it work this way?
PS: sorry if this is a bit offtopic…
-
From my experience….sorta. Its been about 6 months since I stopped using squid, but if my memory serves me, clients would be directed over the transparent proxy if I directed them to go over the default gateway. If you add a 2nd gateway, squid has no idea about it, it doesn't really know it exists.
-M@
-
I suppose you get set up 3 PFsense boxes:
2 boxes connect to each DSL line respectively, both run Squid (both only have one connection, right?)
1 box is the gateway for your LAN, and it load balances to the other two boxes.
-
I have Foxnews.com and Foxbusiness.com behind two redundant pfsense firewalls running on Dell 2950's.
-
Foxnews.com and Foxbusiness.com
Just out of curiosity: how much traffic do they generate on average?
-
Obviously most of the http requests are Akamaized, but not all of it. There's ads and everything else we deal with, plus odd projects, etc. So, it's not so much 'bandwidth/traffic' as it is 'packets'.
I don't have a whole lot behind them yet, I'm in the process of moving more services from other datacenters.
At any rate, currently they're only doing about 60mbps. In 3 months time, I expect to be doing about 1gpbs consistently.
-
In 3 months time, I expect to be doing about 1gpbs consistently.
What sort of hardware you planning on using for THAT ?
-
@sai:
In 3 months time, I expect to be doing about 1gpbs consistently.
What sort of hardware you planning on using for THAT ?
I've already got 2 Dell 6850's allocated for it. 2950's will do the job easily, but you always want to be prepared for future growth. 6850's will let me ignore any firewall related hardware upgrades in the future.
-
I've already got 2 Dell 6850's allocated for it. 2950's will do the job easily, but you always want to be prepared for future growth. 6850's will let me ignore any firewall related hardware upgrades in the future.
Are you serious? You are my new hero if you are!
-
Yep, I'll be at the datacenter tomorrow, I'll take some pictures of our cage with my phone. We've got 6 6850's in production right now, mostly for database servers. Then another 40 or so 1950's/1850's in our cage, all behind the firewalls - which again, are 1950's for the time being.
1950's will surprise you, before I went live with the pfsense firewalls, I got around 600mbps through them in testing, stable. Bursts up to around 800mbps.
I really don't want to do anymore changes to the firewalls until pfsense 1.2 is released. FreeBSD 7 is going to help things a lot more than you might think.
