Strange Test results…..
-
Hi All,
Im very y new to FreeBSD and pf, but I need to upgrade my corporate firewall and pfSense looked like it had a lot of good features.
I did a bit of a comparison test of pfSense with Smoothwall and Checkpoint.
They are all running in a VMWare GSX environment on a Dell PE2850 3.2Ghz Xeon w/512Mb RAM. ALso running on VMWare are 2xCentOS4.1 servers to provide Apache.
The results are really odd to me, admittedly I know very little about FreeBSD, but it seems that Smoothwall (RH9 I think) outperformed everything else. The ruleset was very simple, an external interface NAT'd to a webserver in the DMZ. I used Webbench to ramp up the connections to the firewalls over 3 mintues. Heres the results:
Smoothwall: 279.77 req/sec - Errors 0
Checkpoint NGX: 234.466 req/sec - Errors 0
pfSense (2 Load balanced Apache servers): 31.9083 req/sec - Errors 10+
pfSense(Single NAT'd APache server) - 18.0167 req/sec - Errors 10+Can someone tell me why my results were so bad for pf? i think its a great firewall, and has many features I would like to use, but considering we run a very busy website, I dont think it would handle the traffic, especially once I start putting 25-30 rules in there.
Comment, questions, suggestions, criticism welcome
D.
-
How are you determining that there was errors? Is this part of the client software?
It may come down to a bug in LB. It's a brand new feature…
The errors have me wondering.
Also, do you have the vmware tools loaded in each? Are you using the vmnet drivers?
-
Yes, the client software reports the errors, I believe they were all request timeouts, when I say 10+ I mean there were on average about 10-15 errors in 9000 requests
I dont have the vmtools loaded on any of the servers, I will try that next week.
Any idea why the request/sec was so low for pf? I thought that it may be because the client software I am using is sending all requests from one machine? Does pf have some sort of connection throttling? Is it trying to defend itself against a SYN flood? Is there anywhere I might start to look for errors?
D.
-
Yes, the client software reports the errors, I believe they were all request timeouts, when I say 10+ I mean there were on average about 10-15 errors in 9000 requests
I dont have the vmtools loaded on any of the servers, I will try that next week.
Any idea why the request/sec was so low for pf? I thought that it may be because the client software I am using is sending all requests from one machine? Does pf have some sort of connection throttling? Is it trying to defend itself against a SYN flood? Is there anywhere I might start to look for errors?
D.
In a nutshell: VMWare + FreeBSD networking performance sucks. I would try these tests with real hardware. I know this is not what you want to hear but its true.
-
In a nutshell: VMWare + FreeBSD networking performance sucks. I would try these tests with real hardware. I know this is not what you want to hear but its true.
Thats completely understandable, and I have a Dell PE850 waiting to install pfSense on, which takes me back to our emails re pe850s and sata drives :)
Until the next release comes out I'll run pfSense from the LiveCD/USBKey and do some more testing using real hardware on Monday.
I'll post the results back here. -
Yes, the client software reports the errors, I believe they were all request timeouts, when I say 10+ I mean there were on average about 10-15 errors in 9000 requests
I dont have the vmtools loaded on any of the servers, I will try that next week.
Any idea why the request/sec was so low for pf? I thought that it may be because the client software I am using is sending all requests from one machine? Does pf have some sort of connection throttling? Is it trying to defend itself against a SYN flood? Is there anywhere I might start to look for errors?
D.
In a nutshell: VMWare + FreeBSD networking performance sucks. I would try these tests with real hardware. I know this is not what you want to hear but its true.
There's also a possibility that it's state table collisions (pf flushes expired states every 10 seconds by default). In the real world you'll see connections from a larger number of IP addresses so this tends to be less of an issue. This may, or may not be the problem here, just offering up another suggestion ;)
FWIW, I've got hosts that do 1000 state table insertions and removals / second with 90K active states w/ no problems. This is on PF's native platform though, I can't speak for FreeBSD although a number of people have mentioned similar numbers to me personally.
–Bill