WebGUI dying under heavy load (Internal Server Error 500 etc.) ?
-
Thanks for doing the comprehensive tests. Let me run it by a couple of FreeBSD developers and get their input.
Thanks. :) You're right. It looks like it could also affect FreeBSD generally…?! ??? ...so it must certainly already have been discussed and solved. ;)
-
Solved, I doubt. What I bet money that people are going to say is that the box is under powered for the task.
-
Solved, I doubt. What I bet money that people are going to say is that the box is under powered for the task.
As you already understood ;) : I'm not looking at the throughput performance, but to get a non-hanging system in overload conditions.
They can try a 50 Mbits/s (*) 64-bytes packets load on any up-to-date multi-GHz machine and see if it stays alive ;D
Maybe then they will understand that it's not a question of horsepower, but of security design. A DDoS attack can be full line throughput of 64 bytes packets… A firewall should be able to handle that without freezing, whatever the line and size.
(*) If my math is right:
- 3000 kbytes/s of 1500 bytes packets is about 2000 packets/s handled on a 266 MHz machine.
- Means 20'000 PPS on a 2.6 GHz machine, let's say 40'000 to take in account caching and other stuff. * 64 bytes, that's only 20 Mbits/s 64-bytes-packets throughput limit before freezing the system.
[EDIT: Corrected my math, but conclusions remain same]
-
Okay, please don't take this the wrong way but you are speaking to the chorus already. I am just telling you that I know the FreeBSD community well and know how they will react to this. The box is clearly underpowered.
-
No offense, but a 266 mHz box with cheap cheasy realtek adapters being expected to handle a 64 byte packet storm at 30 mB/s is rediculous. The system is meant for 5 mBit and less WAN use. When in doubt, overbuild, don't underbuild your firewalls. You don't have the luxury of super optimized TCP offloading and handling by the NIC's unless you're using nice Intel Server grade PCI-X gigabit adapters (which I do when I can). The real issue of PPS is the fact that you are memory / copy throttled. You can only make a dozen or so memory copies so fast per packet before you run out of internal machine bandwidth. It's not so much GHz as much as memory latency, bus latency, and overhead on the packet processing. If you have specialized packet processors that do nothing but offload TCP headers and minimize copy processes you have something more in line with a Cisco / Juniper networks box.
Just speaking from personal experience here, if you want balls to the wall performance with 64 byte packets, overbuild the crap out of the box with an opteron processor or two and really fast low latency memory and a PCI-X TCP offloading nic. I've pushed over 100 mBit of 64 byte packets this way.
But yeah, the FreeBSD guys are going to laugh and say, get a bigger box.
The only way to fix this behavior at all is to study how many copy's happen between packet input and output, then reduce that number. Or otherwise implement some packet storm throttling / protection to damp down an attack.
But yeah, why would you expect a 266 mHz box to perform at 30 mBit? Cisco puts 166-266 mHz processors in their T1 grade routers with full TCP offload and only rate them for 1-2 mBit connections. Just FYI.
You won't ever reach overload conditions on a 2 mBit pipe. If you're trying to push a 10 mBit pipe you need a bigger box, period.
-
Hi.. i´m dealing with the same "problem" and therefore i ask..:
How much hardware do you need to run +80MBit/s througput?
Im getting max. 10MBit/s with my pIII 933/512MB mem and a 3Com 3C982 dual NIC
If i try the m0n0wall with the same HW the result is 22MBit/s ??
-
Hi.. i´m dealing with the same "problem" and therefore i ask..:
How much hardware do you need to run +80MBit/s througput?
Im getting max. 10MBit/s with my pIII 933/512MB mem and a 3Com 3C982 dual NIC
If i try the m0n0wall with the same HW the result is 22MBit/s ??
Something is wrong here. I'm able to get about 87 mbit/s throughput from one of my C3 1GHz LAN->WAN in factory default configs. If I remove the pfSense I only get about 2 mbit/s more and that is with crappy viarhine onboard nics. Your system should push much more.
-
Actually, the throughput should not be measured in Mbits/s but in packets/s (pps), like in routers.
This will be putting better light on the true performance, as the packet size usually matters less than the packet handling overhead.
Typically, with lots of 1500 bytes packets one way, and few 64 bytes ack packets the other way, if you are well below wirespeed, you can take the Mbits/s figure and divide it roughly by 1500*8 to get the pps throughput. e.g. 22 Mbits/s = 1'830 pps.
The true throughput performances are usually made with minimum size packets of 64 bytes, and I would be curious to see some of such performance figures :)
Please don't take this as critisism, pfSense is great work and runs great :)
-
hmm wierd..
in bone stock config i get 45MBit/s with m0n0 and 22 with pfSense..
Any ideas?
My HW is P3 933CPU on MSI mainboard, 512MB PC133 SD, and a 3Com Dual NIC 3C982, well, it is the same with 2 Realtek 8139C…
-
FreebSD 6.1 is much slower than 4.11. This is the reason that m0n0wall was hesitant to switch initially. Why they want to switch now is beyond me because its going to slow down every installation.
I would suggest going back to m0n0wall.
-
yea but still the m0n0wall isnt doing the job either.. i should be able to get +80MBit througput on my existing hardware…
and the pfSense has some of the features i need
-
Well then it sounds like you need beefier hardware then if you want to stick with pfSense.
-
hmm.. ???
hoba wrote:
"Something is wrong here. I'm able to get about 87 mbit/s throughput from one of my C3 1GHz LAN->WAN in factory default configs. If I remove the pfSense I only get about 2 mbit/s more and that is with crappy viarhine onboard nics. Your system should push much more."
-
I can push 40 megabit with a 500 mhz box so I somewhat agree with Holger.
What the exact problem is with your hardware is hard to predict, however.
-
hmm yes.. im am trying with the default config, så it must be the HW.. im gonna try another machine tommorow then..
Thanks so far :)
-
Ok.. the results with the new HW
HW: P4 2,5Ghz, 256MB DDR2, Onboard GB NICs
Config: 100% default
pfSense:
Max speed = 60MBit/s
Payload = 100%
System unstabel!
WebGUI = unreachablem0n0wall:
Max speed = 85MBit/s (the switch couldt handle more, trying another GB Switch tommorrow)
Payload = 30%
System very stabel!
WebGUI = 100% functioningseems 2 me that pfSense has a problem?
-
You are not specifying which type of NICS. Please specify your NIC types.
-
So far I have been able to push ~400Mbps through a dual port Intel GigE in a P3 1Ghz and 22Mbits+ through a via rhine(vr0)/3com 2000-T (sk0) combination on a Via Eden 933.
Then again, I have not been trying to access the webgui.
A p3 1Ghz doing 400Mbits is about ~55% interrupt. So that one doesn't die from load.The Via with 22Mbits and a traffic shaper was pretty loaded but the queues page at 22Mbits was still accessible.
One point of thought though. This testing was performed somewhere in the pre RC1-ish days.
I have not ran benchmarks one rc2 and above yet.Something may indeed be amiss.
-
@sullrich, actually that isnt important in this case, i have tryied with 3Coms, Realteks & Onboards both 100MBits and 1GBits.. the difference is less than 2MBit/s when im using pfSense..
But if you need the exect type of NICs, please let me know.
The interrupt is 99/100% when pfSense is doing 60MBit/s on my actual system.. wich was; P4 2,5Ghz, 256MB DDR2
and perhaps your (databeestje) results where on some RC1-ish, but in my oppinion the newer version shouldnt be slower?
-
This is a known problem. I've posted to the freebsd lists but nobody has responded yet.