Traffic Limit Speeds Up Transfer Rates!

geyser

I am doing up a pfsense box to sent to a datacenter, where I am charged on 95% usage. So I wanted to put a limit on all WAN bandwidth to ensure I don't have too many spikes and result in my 95% usage going way up.

Before doing so I wanted to see how the box performed under load. The box for those who are interested in a Firebox x750e withe the CPU upgraded to Pentium M @ 1.7GHz and 2 gigs RAM.

My testing environment is all gigabit, copying a 8.5gig file from a QNAP to the local computer. On my workstation I can easily saturate the 1 GIG network - but that is direct to the QNAP, not going through any firewall.

When going through the pfsense box the transfer speed sucks to say the least. The speed is erratic and not fast at all. Normally hovering around 25 Mbps

Then if I enable a traffic shaper: limiter and place a bandwidth limit of 200 Mbit/s on the firewall rule, then transfer the same file the graph looks like this.

It essentially pins the connection out at the 200 Mpbs limit that I have imposed. The two blips where it looks like the speed goes to 0 is not seen in the FTP client, and the graph shows an inbound speed with a negative in front - so I am assuming that is an issue with pfsense graph.

So how/why would the connection be faster when a limit is placed on it?

Also interesting is if I increase the limit, to say 250 Mpbs then the connection speed drops back down to around the 25 Mpbs.

I can't imagine why putting a limit on the connection speed would increase the speed.

Any ideas?

Derelict

I don't think you understand 95% billing. You are better off having no traffic most of the time, then move it all in "spikes" that get discarded by the billing mechanism.

Say your provider uses 5-minute intervals for its usage calculation. If you had a certain amount of data to send, you could send nothing for 95 * 5 minutes (7.91 hours) then send all your data at max link speed for the next 25 minutes. Your net bill should be zero because the 5% of samples where you were actually transferring would be tossed out.

If you have to send traffic in realtime, as most do, spikes make no difference. If you limit the traffic rate, but send the same amount of data, your provider will just have more samples at a slightly higher rate instead of tossing out the spikes. It all works out.

All that said, there has to be something else going on besides just enabling the limiter in the graphs you show.

geyser

@Derelict:

All that said, there has to be something else going on besides just enabling the limiter in the graphs you show.

I agree, there has to be something going on here. But all I am doing is enabling or disabling that limiter and conducting my test again - absolutely no other changes. There is no other traffic on the network as this is just a lab test.

I just have no idea what to even look at to try and determine what the issue is when there is no limiter in play. The traffic when there is no limiter is really erratic.

Derelict

Can you post exactly what you're doing in the limiter and the firewall rules?

geyser

I created two limiters, a InLimitWAN and OutLimitLan, they are both identical with a bandwidth limit of 200 Mbit/s.

Then on my LAN's default allow rule (allow everything out) I selected InLimitWAN/OutLimitLan under the In/Out section.

That's it.

Then to conduct my test I either enable or disable the limiter. No other changes between the testing.

Derelict

All looks right. Instead of enabling and disabling the limiters, what happens if you set the default LAN allow rule back to none/none instead?

Anything else set in "Advanced" on LAN or WAN?

Is this a router with plenty of CPU for the speeds your looking to accomplish?

And it's not just the graphs doing weird things? The actual transfer speeds reflect what they show right?

Also see if Diagnostics->Limiter Info shows anything interesting in both cases.

geyser

I have nothing set under advanced, thought I would start with the basic.

Transferring at 200Mbps the CPU is at 36% so that should not be a problem. Others with the same box report it doing much faster speeds than that. I actually want the limit on as I don't want spikes to 1000Mpbs, limiting my spikes to 200Mpbs is good for my use.

I actually have no complaint about the performance of the box when the limiter is on. I just am baffled why it performs so badly when there is no limiter in place.

I plan to leave the limiter enabled and send it off to the datacenter. Odds are good that the problem is something in my lab setup and the box is fine.