How Far Have You Scaled Your PFS Box?
-
Update:
We have an Intel SR1520ML server that we are testing. This thing has 2 separate servers in it. Each side we have a 2.4Ghz quad core cpu and 4GB DDR2-800 memory with the x83ML boards. With the onboard nics that it comes with, the thing will saturate 1GB. Right now we are pushing it over 750MBit pretty consistently. The cpu isn't even hitting 45%.
I have attached a pic for the non believers ;)
-
:o Yup…..now THAT'S what I'm talking about :o Now lets see it do that in the real world ;)
-
Actually now it is in the real world. We have proxyarped a few servers behind it pushing real traffic.
-
Awsome thread!
I have PFS running on a Mini-ITX intel board by Jenson, 1.6ghz core-duo, 2 gig ram stick, 1 gig ide solid state. (4) gigabit ethernet adapters, 3 Wireless USB sticks. Running PFS 1.2.3. I am currently combining 3 WAN connections into our home for personal use.
No pictures right now of the case, it looks like a fat common router case.
-
I use pfSense at about six locations providing internet to a total of about 500 users and growing. Each location has at least a 16mbps/2mbps line to the internet, some are mutli WAN. The pfSense boxes use very little resources and I literally "set it…. and forget it " :) I login remotely every couple weeks just to look at some of the usage graphs and poke around a little bit. I cannot wait for 2.0 to be released, but I am almost positive I will be using it as its still in RC. When 1.2 was in RC it was solid as a rock, so I'm kind of hoping the same for 2.0.
This project is truly amazing and I can't complement the developers enough. Although money is also tight here, I try and help the project by providing them with a VM in a top notch datacenter.
Don't forget to help support the pfSense developers and the project if you're using pfSense in any way, especially in a commercial environment!!!!
-
I work as a free lancer for a buisness school. In our network there are around 300-400 users connected over wifi to the network, around 200 faculty systems ( some are linux terminal servers, others are fat clients/laptops). We have 3 x 2 mbps links which are load balanced by pfsense for all the users. Pfsense has been running super stable since last 8 months now with about 3-4 reboots ( just for heck of it, pfsense never gave up) . We have already pushed 3.85 TB of data to the internet. Thank you pfsense.
-
This is the main MRTG graph on a 10gbit link attached to a PFSense box. Pfsense is really rocking! I can't discuss too much detail about the systems, but they have 16GB ram and Xeon Processors. 35% CPU utilization on 8 cores with this level of traffic. Network cards are PCI-E 8x and pfsense is acting as a core router. No NAT, No firewall. Same performance as a $250,000 Cisco. We were deciding between a fully loaded Catalyst 6500 and this system. Price and performance won out here. We were using custom FreeBSD for this, but we have applied our tweaks to PFSense, so that there is a more friendly interface to those that need to support this system. I find that the biggest issue isn't amount of traffic, but packets. We are averaging ~215,000 packets per second on the WAN. This is a ridiculous number to wrap around, and for those of you trying to do the math, the packet size in the real world is not a perfect 1500. For those of you that know this, please forgive me, I just don't want to explain that the average bandwidth should not be 2.4gbps based on the pps.
-
I tend to go a little overboard on all my builds so my Firewall was no exception. I just finished my third box, I started with a VIA 900mhz embeded motherboard with 3-NICs and 1-Wireless NIC, 512 RAM. Second was a retired Inspiron 6000 laptop: 2-NICs(would have done more but only 1 PCMCIA slot) and 1-Wifi, 1.3Ghz Celeron and 2GB RAM, the fianl one I am using currently is a Intel D945GNC ATX board with a Q9400 Quad Core, 3.5GB of RAM, 2-Intel PRO1000 NICs for LAN, Onboard 10/100 for the WAN and 1- 10/100 PCI NIC for WAN2, and a D-Link Wireless card. The utilization is far below what it can handle even with the 7 VPN connections that run all the time. I will have this one for quite awhile. Gotta love Pfsense…
-
Unlocking topic. Let's hear some more stories. 2.0 stories would be nice, too!
-
Fun to see a thread I started so long ago still kicking :)
This one isn't all that exciting as far as througput goes, but it's mildly interesting. I've got an Intel 1.6 GHz single core Atom board running with a Soekris quad NIC in it, and load balancing 4 DSL lines. This box is actually running in the pressbox of Qualcomm Stadium for the San Diego Chargers media folks. It's sad that in a stadium named after a communications giant, all we can get is standard ADSL lines, but on a budget, I put this box together and have about half a dozen Cisco Aironets hanging off it, and the thing actually load balances quite well. I'm thinking about upgrading to 2.0 for the coming season, as I hear the load balancing is improved. The only other feature I'm using is the Captive Portal. But in my experience thus far, the Intel Atom boards are EXTREMELY stable running PFSense!
Is the load balancing in 2.0 improved enough to warrant the upgrade?
-
Is the load balancing in 2.0 improved enough to warrant the upgrade?
IMHO, it's way easier to set up and it's a lot more flexible. You an achieve a very complex setup with a mix of loadbalancing, failover and policy based routing.
As to whether it's actually better at load balancing? I have no numbers to show any real improvement. I could say it 'seems' better. It has produced less odd website problems. It certainly seems to play nicer with speedtest.net, though that could just be speedtest updating their client. ;)Steve
-
I see the same thing with speedtest.net on 1.2.3 I can get over 15 Mb/s out of a bunch of 6 Mb/s DSL lines. I think the speedtest client just opens several concurrent connections and aggregates them for the speed display, though I haven't run a packet capture or anything on it.
-
'm actually in the middle of this argument with one of my bosses. He wants Cisco, mainly because of paid support - which I completely understand. I told him I'm more comfortable with pfsense, I know what it can and can't do. I don't know anything about Cisco IOS.
-
'm actually in the middle of this argument with one of my bosses. He wants Cisco, mainly because of paid support - which I completely understand. I told him I'm more comfortable with pfsense, I know what it can and can't do. I don't know anything about Cisco IOS.
There is commercial support for pfsense, too:
http://www.pfsense.org/index.php?option=com_content&task=view&id=62&Itemid=73 -
'm actually in the middle of this argument with one of my bosses. He wants Cisco, mainly because of paid support - which I completely understand. I told him I'm more comfortable with pfsense, I know what it can and can't do. I don't know anything about Cisco IOS.
Depends on the application, but for most small businesses, PFSense is much more capable than Cisco, and doesn't nickle and dime you to death. I like to use PFSense for firewalls/routers, and Cisco for switches. PFSense does "Router on a Stick" with a Cisco switch just as well as Cisco does.
-
Of all the imaginable settings I believe has been all tried out. I am still new and still feel like didn't even figured out how 10% of PFS works, anyhow here is my set up:
8 PFS(1.2.3) on different subnets in one single LAN, providing wifi to a group of university students of some 20k from several campuses spread over different geographical location, CP is enable and auth'ing on windows server RADIUS so everybody login with their AD accounts. Squid is on transparent mode. Of course, the DHCP range won't be enough for all of them, I am getting maximum some 400 concurrent CP users logged in.
I am interested to know that of all the scale you guys have here, how do you keep track of your servers and total bandwidth usage? Who downloaded most ahemm cartoon? Total connected users? Server load… etc
NMAP and Nagios is one way to find out if your servers are alive and how well they are doing. But here is how I did it:
From a dedicated linux box, have all the ssh keys set up, then make a bash script that looks something like:
get_stat=$(
ssh $host "grep -c "192.168." /var/db/captiveportal.db;
grep -c "192.168." /var/dhcpd/var/db/dhcpd.leases;
grep -c "active" /var/dhcpd/var/db/dhcpd.leases")Then make it into a function so you can do something like:
getpfsstat "pflondon"
getpfsstat "pfnewyork"
getpfsstat "pfkinabalu"And arrange the output nicely on the screen with simple printf:
Server: London
Status: up users: 98, dhcpd: 269, active: 180
Server: New York
Status: up users: 78, dhcpd: 384, active: 172
Server: Kinabalu
Status: OMG SERVER DOWN HIT PANIC BUTTON NOWRun# watch -n20 ./servermonitor.sh
And then you can happily counting how many total users you got over your network :DOptionally you can also output to a html file, host it on lighttpd. Then you can access to that webpage and brag about how many people is using your servers now.
Now, seriously, has this been a common practice or I have been doing a simple thing complicated way...
Fun to see a thread I started so long ago still kicking :)
Fun to see that the TS is still kicking too :)
'm actually in the middle of this argument with one of my bosses. He wants Cisco, mainly because of paid support - which I completely understand. I told him I'm more comfortable with pfsense, I know what it can and can't do. I don't know anything about Cisco IOS.
You can fire your boss, Cisco won't, that's why. :p
