Traffic Shaping limits being ignored
-
I upgraded to the latest version this AM: 2.0-RC1 (i386) built on Wed Mar 16 17:04:38 EDT 2011. Been going through everything making sure it works as expected, and I've found that the traffic shaper doesn't seem to be following the size limits established. My primary WAN is capable of traffic >10Mbps, but I've configured the shaper WAN queue to only be 2.5Mbps. When I run speed tests behind the shaper, I can see the traffic is hitting the shaper queues, but I'm getting speeds in the realm of 15Mbps… basically it's topping out my connection and not caring about the 2.5Mbps queue size I established.
Is this a bug, or is this behaviour in the shaper that I'm not aware of?
-
I also have this problem.
-
Depends on how you have configured it.
-
Can you give us some more details ermal? The shaper is so changed from version 1.2 that the instructions in the pfSense book are very hard to apply.
-
You have to give me details so i can give some to you back.
Show me your config and specify which limit is not honored and i will tell you a reason on that. -
@ermal:
You have to give me details so i can give some to you back.
Show me your config and specify which limit is not honored and i will tell you a reason on that.Sorry to hijack the thread but on this topic.
I've set the interface bandwidth (LAN) in the shaper on pfSense 2.0 RC1 and the traffic shaper doesn't seem to drop packets or limit accordingly. I was previously using the Nov. 27th build and it worked.
i.e.
I have a Cable connection rated for 15Mbps but it can burst and sustain 17Mbps during peak hours whilst rocketing to a sustained 28Mbps during off peak hours.
The downside is that once the rates exceed 16.5Mbps during peak hours, congestion kicks in uplink and my average latencies can increase by as much as 500ms.Now, I've tried setting the LAN interface (root) to 16Mbps but the downloads can still exceed and sustain far in excess of the figure. I've reset the states to try as well so that's not the issue.
This affects both HFSC and PRIQ (tried both). It does not affect uploads though.Is this meant to happen? I assume some changes were made since the shaper wizard (I configure manually) no longer defaults to having downstream shaping activated.
-
I am sorry but without giving proper information on queues config i cannot answer anything.
-
Ermal I've attached my shaper config, renamed as .txt. The basic config is for 3 WANs and 2 LANs. The LANs are 100Mbps and don't need any shaping applied, while the WANs include 1 larger pipe, and 2 identical smaller DSL lines.
Here's a basic FW rule to put traffic into my queues:
<rule><id><type>pass</type>
<interface>lan</interface>
<tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
<os><protocol>tcp</protocol>
<source>
<network>lan</network><destination><any><port>HTTP_HTTPS</port></any></destination>
<gateway>LB1_ICA</gateway>
<defaultqueue>qDefault</defaultqueue>
<ackqueue>qACK</ackqueue>
<l7container>HTML_queing_STAFF</l7container></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule> -
Ok. That's what i've done.
Launched shaper wizard (traffic_shaper_wizard.xml), it's made a basic set of queues/rules.
First of all. I've set both bandwidths in wizard (uplink - 20Mbps and downlink - 30Mbps). But as result of wizard's work i see only 20Mbps applied on WAN interface.
Second. There is set of floating rules, but they all only for upload (for instance: "m_P2P BitTorrent outbound" and etc). No rules for download and no queues for download (LAN if).
As far as i understand there is no "pipes" in new ALTQ shaper and it shapes only outgoing traffic on corresponding interface. So it's just can't work properly without set of queues/rules for at least 1 WAN (upload) and 1 LAN (download). But as i already said, they are absent for LAN if.
Here is set of queues created by wizard:WAN
|–qACK
|--qOthersDefault
|--qP2P
|--qOthersHigh
|--qOthersLow
LANAs you can see there is nothing for LAN. It's not configured at all. I'll not copy/paste the rules as there are to many of them and also that will be quiet useless. You can just believe me there are no rules for upload.
And here the question. Are wizards work wrong or it's done intentionally? Shaper is useless if it shapes only in 1 direction, that's obvious. So i see only one way, wizards work improperly.
Correct me please if i'm wrong.P.S. Sorry if i made some mistakes. I'm Russian.
-
Give this a shot:
Try Single WAN multi LAN instead of the very top wizard. I've found that this one works and fills in the proper LAN section while the top one doesnt. This has been slightly goofed up in this way since a few months ago in my experience.
Also load balancing on multiple WANs is just not very workable at all…
But if you have one WAN and one LAN choose the second wizard from the top and it should work. Always has for me.
-
1st. Same thing with rules as all other wizards got. There are only rules with suffix "outbound".
2nd. There is a bug ticket: http://redmine.pfsense.org/issues/749
And here is the part of post:Updated by Ermal Luçi 3 months ago
Status changed from New to Feedback
Traffic_shaper_wizard, traffic_shaper_wizard_dedicated and Traffic_shaper_wizard_multi_all will not show this problems anymore.As you can see multi lan wizard is not in this list.
Anyways the situation with shaper in 2.0 is frustrating. Wizards are useless because result of they work need to be edited a lot. Much easier make all this by hands or create your own macro. And old dummy net shaper doesn't work at all. All it allows you to create is pipes. But further attempts to create queues gives you only an error about queue name while name is ok. I understand it's still in development, but see no bug notes about this situation. So is all this a bug or a feature? ;D
Here is log message on attempt to create queue in limiter:
php: /firewall_shaper_vinterface.php: SHAPER: could not create queue high on interface uplink because: Array ( [0] => Queue names must be alphanumeric and _ or - only. )
But i see no problem in name "high". Apparently script does.
-
Also load balancing on multiple WANs is just not very workable at all…
Do you mean server load balancing, or outbound gateways/routing groups? We've been using gateway group balancing in multi-WAN in 2.0 for months and it works well.
-
Hello, I think I know what is going on with this thread. See ticket 749
http://redmine.pfsense.org/issues/749
Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.
I guess they are still discussing the change, as stated in the ticket.
Traffic shaping only works on traffic leaving an interface, so the wan queues are only for upload traffic. There really is no way to truly shape the download traffic in a multi lan environment, since each lans queues are handled separately. It can work most of the time depending on usage patterns... hopefully they will put it back in.
Josh -
Hello, I think I know what is going on with this thread. See ticket 749
http://redmine.pfsense.org/issues/749
Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.
I guess they are still discussing the change, as stated in the ticket.
Traffic shaping only works on traffic leaving an interface, so the wan queues are only for upload traffic. There really is no way to truly shape the download traffic in a multi lan environment, since each lans queues are handled separately. It can work most of the time depending on usage patterns... hopefully they will put it back in.
JoshActually i said the same in my post, stompro. ;) And what they've done is not solution of the problem. ALTQ shaper just can't work in this case, old school dummy net shaper (personally like it much more) required for this purpose. At this moment it must be already fixed. Didn't tested it yet, but i will in a next few hours. Wizards must be fixed. If shaper shapes only in one direction, who need it at all? It's a nonsense. But we all must understand pfSense 2.0 is not ready yet, it's still in development stage so all we can do is wait or help developers to finish it faster.
-
Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.
That answers my question. Thanks. I was wondering why the LAN side bandwidth limit (on LAN Root) didn't work. I guess I need to find a way to perform LAN shaping via Limiter queues instead. However, this screws up my ack queues (ack packets are getting dumped into default queue instead).
-
Thanks stompro… that makes sense. Thanks Loke too... I didn't quite understand the way you wrote it, but I understand now. I'll follow up on that ticket. We'd be happy to sponsor development on this, it's pretty crucial and like you said, a shaper that only works one way is pretty useless.
-
Actually i said the same in my post, stompro. ;) And what they've done is not solution of the problem. ALTQ shaper just can't work in this case, old school dummy net shaper (personally like it much more) required for this purpose. At this moment it must be already fixed. Didn't tested it yet, but i will in a next few hours. Wizards must be fixed. If shaper shapes only in one direction, who need it at all? It's a nonsense. But we all must understand pfSense 2.0 is not ready yet, it's still in development stage so all we can do is wait or help developers to finish it faster.
Sorry Loke, I see it now, sorry for just repeating you. I have thought of using dummynet for this, but I haven't figured out how to use multiple dummynet pipes together yet. I want to keep using the built in limiters for the captive portal users, along with for certain other uses, and also run all traffic through one that limits to %90 of bandwidth, so I don't trigger the buffer bloat http://www.bufferbloat.net/ problem.
-
Well you can clone the WAN config on LAN from the Queue view tab and just change bandwidth settings.
That will enforce 'download' to be followed with the same rules. -
Dummynet shaper is quiet simple to understand. And when you'll understand how it works it's very simple to create any configuration you need. ALTQ shaper got a great minus. It can shape traffic only on exit from interface. Dummynet can shape in both directions and can limit speed not only for diff traffic, but also for diff sources/destinations. And this is a huge advantage over ALTQ in some cases.
-
@ermal:
Well you can clone the WAN config on LAN from the Queue view tab and just change bandwidth settings.
That will enforce 'download' to be followed with the same rules.Thanks Ermal, I've done that now. The shaper limiting is now working as it was some months ago. We're still interested in sponsoring this to come to a good solution for all parties.