PfSense overhead

berrick

Hi All,

What sort of overhead (impact on broadband speed) should I expect to see

I have just installed pfsense 2.01 on an old Nokia IP330. This connects to a vDSL modem connected to a 40/20 FTTC link.

There is no fancy config in pfsense yet I see a consistent 10Mbps drop in download speed and a 6Mbps drop in up load speeds when pfsense is used.

jasonlitka

This is my go-to list for people reporting slowdowns:

Duplex mismatch between your pfSense box and the DSL/Cable modem.
Double NAT.
pfSense box is underpowered (eg. trying to run a 100/100 line on an Alix or trying to use Snort, Squid, L7 shaping, etc. without being aware of how much CPU power those need to perform well).

Depending on what model IP330 you've got, it might well be something like a 166MHz Pentium w/ MMX (like this guy from 2006). If so, be thankful you're getting what you're getting.

berrick

No Duplex mismatch - I meant to state that but forgot
No double NAT - the vDSL modem is configured as a bridge, NAT, PPPoE etc to be handled by another device
the CPU doesn't seem to break a sweat though when testing

dreamslacker

If you have traffic shaping setup, that could be the problem. The wizard generated values might be limiting the bandwidth available for your downloading.

stephenw10

I wouldn't expect that CPU to have a problem at 40Mbps and you have said it doesn't appear to have excessive CPU use. How are you measuring that?
Try running 'top SH' at the console.

Are you running the default install or have you added packages, enabled extra services, tweaked it in any way?

Steve

berrick

OK,

Using a different router to perform the modem bit, a different hard disk and pfSense 2.1 just for good measure I have a slight improvement but still dropping about 8Mbps on the download and 5Mbps on the upload speedtest.

I'm taking speedtests to the same sites just before and just after I insert the F/W and always see this "overhead"

top sh shows

CPU: 54.9% user, 0.0% nice, 7.4% system, 0.8% interrupt, 37.0% idle - no speedtest
CPU: 0.0% user, 1.4% nice, 6.8% system, 91.7% interrupt, 0.0% idle - Download speedtest
CPU: 41.8% user, 0.0% nice, 8.8% system, 49.4% interrupt, 0.0% idle - Upload speedtest

So would my assessment that the high interrupt values is down to the NIC's and that this is where the overhead is coming from?

heper

what is using 55% of cpu when it's supposed to be almost idle ? this can't be right … not even on a P2

berrick

@heper

Its not a what its several processes at the same time. That is if I'm doing this correctly.

I used top -S and very frequently the CPU peaks at around the 50% mark. When this happens several processes are running such as netstat, php,grep.

Does that answer your concerns?

stephenw10

Do you have any sort of power saving features enabled?
92% interrupt load doesn't look good though. What's causing that? Try running 'vmstat -i'.

Steve

berrick

I ran the command and below is what it returned. I dont know this command, not sure how to read the output.
Please be aware I was installing ntop at the smae time

interrupt total rate
irq0: clk 955353 99
irq4: uart0 3272 0
irq5: fxp2 148409 15
irq7: ppc0 1 0
irq8: rtc 1222813 127
irq10: fxp0 194081 20
irq14: ata0 19767 2
Total 2543696 266

stephenw10

Ok, well that looks fine the rates are all reasonable. Try re-running it when you are downloading at max speed.
You haven't enabled powerd I assume?

Steve

berrick

This is a fresh vanilla install of 2.1 Just in stalling ntop then squid to get user statistics.

I will try another download test prior to setting up squid.

So would my assessment that the high interrupt values is down to the NIC's and that this is where the overhead is coming from?

Was my statement above correct?

Here is the output from vmstat -i whilst performing a download and upload test

interrupt total rate
irq0: clk 1444232 99
irq4: uart0 3272 0
irq5: fxp2 281746 19
irq7: ppc0 1 0
irq8: rtc 1848559 127
irq10: fxp0 288395 19
irq14: ata0 75292 5
Total 3941497 272

interrupt total rate
irq0: clk 1449051 99
irq4: uart0 3272 0
irq5: fxp2 283602 19
irq7: ppc0 1 0
irq8: rtc 1854727 127
irq10: fxp0 289780 19
irq14: ata0 75378 5
Total 3955811 272

interrupt total rate
irq0: clk 1451151 99
irq4: uart0 3272 0
irq5: fxp2 284229 19
irq7: ppc0 1 0
irq8: rtc 1857415 127
irq10: fxp0 290190 19
irq14: ata0 75403 5
Total 3961661 272

interrupt total rate
irq0: clk 1452137 99
irq4: uart0 3272 0
irq5: fxp2 287504 19
irq7: ppc0 1 0
irq8: rtc 1858677 127
irq10: fxp0 292891 20
irq14: ata0 75406 5
Total 3969888 273

interrupt total rate
irq0: clk 1452770 99
irq4: uart0 3272 0
irq5: fxp2 290780 20
irq7: ppc0 1 0
irq8: rtc 1859487 127
irq10: fxp0 296712 20
irq14: ata0 75461 5
Total 3978483 273

interrupt total rate
interrupt: Command not found.
irq5: fxp2 281746 19
irq7: ppc0 1 0
irq8: rtc 1848559 127
irq10: fxp0 288395 19
irq14: ata0 75292 5
Total 3941497 272

heper

try removing ntop …. it could be the elephant in the room

stephenw10

Presumably you are downloading between fxp0 and fxp2? The interrupt rates look low. I expected far higher given the interrupt load on the cpu.

Can you show the full output of 'top -SH'?

Steve

berrick

Hi,

Sorry for the delay.

This is the output pre Speed test

During Download test, the screen refresh stops during this

During Upload test

I have added nTOP, Squid, Lightsquid, Sarg and Dansguardian as I needed to demostrate these features.

Look forward to your comments

stephenw10

Those packages are going to really hit that box hard. Many people would consider them unsuitable for the Alix box with it's 500MHz CPU and 256MB ram. Those interupt figures don't look inconsistent in any way. Perhaps that's just the limit of the hardware. :-\ It could be that I'm showing my age but I remeber the K6-2 as being quite fast. Happy hours on Doom2! The alix has a NAT/firewall limit of 85Mbps, significantly faster.
It's difficult to find and benchmarks that compare the two processors. Somthing useful can be found here:
http://new.haveland.com/povbench/graph.php
There we can see the Geode 700LX (500MHz) score 8.19x (the speed of a Pentium 100) where as the K6-2 at 400MHz scores 3.46x. Thus if both boxes are dependent purely on processing power and the Alix tops out at 85Mbps I would expect yours to manage 36Mbps. Close to what you're seeing. That doesn't really explain the drop when uploading though. :-\

Steve

berrick

Ok, Good link.

I understand what you are saying I'm guessing that its the NAT part which is the overhead. The reason for stating this is I configured the IP330 as a transparent bridge and repeated the speed tests, no overhead. Get the same figures whether I go through the IP330 or not.

If I'm wrong please correct me.

stephenw10

Yep you will see significantly faster throughput in transparent mode. There are a lot less processing steps when you disable NAT, even less when you are bridging.

However that still doesn't explain why you are seeing reduced upload speeds. You would normally see no significant reduction in throughput until you hit the limits of the hardware.

Steve