Traffic Shaping limits being ignored



  • I upgraded to the latest version this AM: 2.0-RC1 (i386) built on Wed Mar 16 17:04:38 EDT 2011. Been going through everything making sure it works as expected, and I've found that the traffic shaper doesn't seem to be following the size limits established. My primary WAN is capable of traffic >10Mbps, but I've configured the shaper WAN queue to only be 2.5Mbps. When I run speed tests behind the shaper, I can see the traffic is hitting the shaper queues, but I'm getting speeds in the realm of 15Mbps… basically it's topping out my connection and not caring about the 2.5Mbps queue size I established.

    Is this a bug, or is this behaviour in the shaper that I'm not aware of?



  • I also have this problem.



  • Depends on how you have configured it.



  • Can you give us some more details ermal? The shaper is so changed from version 1.2 that the instructions in the pfSense book are very hard to apply.



  • You have to give me details so i can give some to you back.
    Show me your config and specify which limit is not honored and i will tell you a reason on that.



  • @ermal:

    You have to give me details so i can give some to you back.
    Show me your config and specify which limit is not honored and i will tell you a reason on that.

    Sorry to hijack the thread but on this topic.

    I've set the interface bandwidth (LAN) in the shaper on pfSense 2.0 RC1 and the traffic shaper doesn't seem to drop packets or limit accordingly.  I was previously using the Nov. 27th build and it worked.

    i.e.

    I have a Cable connection rated for 15Mbps but it can burst and sustain 17Mbps during peak hours whilst rocketing to a sustained 28Mbps during off peak hours.
    The downside is that once the rates exceed 16.5Mbps during peak hours, congestion kicks in uplink and my average latencies can increase by as much as 500ms.

    Now, I've tried setting the LAN interface (root) to 16Mbps but the downloads can still exceed and sustain far in excess of the figure.  I've reset the states to try as well so that's not the issue.
    This affects both HFSC and PRIQ (tried both).  It does not affect uploads though.

    Is this meant to happen?  I assume some changes were made since the shaper wizard (I configure manually) no longer defaults to having downstream shaping activated.



  • I am sorry but without giving proper information on queues config i cannot answer anything.



  • Ermal I've attached my shaper config, renamed as .txt. The basic config is for 3 WANs and 2 LANs. The LANs are 100Mbps and don't need any shaping applied, while the WANs include 1 larger pipe, and 2 identical smaller DSL lines.

    Here's a basic FW rule to put traffic into my queues:

    <rule><id><type>pass</type>
    <interface>lan</interface>
    <tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
    <os><protocol>tcp</protocol>
    <source>
    <network>lan</network>

    <destination><any><port>HTTP_HTTPS</port></any></destination>

    <gateway>LB1_ICA</gateway>
    <defaultqueue>qDefault</defaultqueue>
    <ackqueue>qACK</ackqueue>
    <l7container>HTML_queing_STAFF</l7container></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule>

    shaper-config-sinai.tacf.org-2011031804385.txt



  • Ok. That's what i've done.
    Launched shaper wizard (traffic_shaper_wizard.xml), it's made a basic set of queues/rules.
    First of all. I've set both bandwidths in wizard (uplink - 20Mbps and downlink - 30Mbps). But as result of wizard's work i see only 20Mbps applied on WAN interface.
    Second. There is set of floating rules, but they all only for upload (for instance: "m_P2P BitTorrent outbound" and etc). No rules for download and no queues for download (LAN if).
    As far as i understand there is no "pipes" in new ALTQ shaper and it shapes only outgoing traffic on corresponding interface. So it's just can't work properly without set of queues/rules for at least 1 WAN (upload) and 1 LAN (download). But as i already said, they are absent for LAN if.
    Here is set of queues created by wizard:

    WAN
      |–qACK
      |--qOthersDefault
      |--qP2P
      |--qOthersHigh
      |--qOthersLow
    LAN

    As you can see there is nothing for LAN. It's not configured at all. I'll not copy/paste the rules as there are to many of them and also that will be quiet useless. You can just believe me there are no rules for upload.
    And here the question. Are wizards work wrong or it's done intentionally? Shaper is useless if it shapes only in 1 direction, that's obvious. So i see only one way, wizards work improperly.
    Correct me please if i'm wrong.

    P.S. Sorry if i made some mistakes. I'm Russian.



  • Give this a shot:

    Try Single WAN multi LAN instead of the very top wizard. I've found that this one works and fills in the proper LAN section while the top one doesnt. This has been slightly goofed up in this way since a few months ago in my experience.

    Also load balancing on multiple WANs is just not very workable at all…

    But if you have one WAN and one LAN choose the second wizard from the top and it should work. Always has for me.



  • 1st. Same thing with rules as all other wizards got. There are only rules with suffix "outbound".
    2nd. There is a bug ticket: http://redmine.pfsense.org/issues/749
    And here is the part of post:

    Updated by Ermal Luçi 3 months ago
    Status changed from New to Feedback
    Traffic_shaper_wizard, traffic_shaper_wizard_dedicated and Traffic_shaper_wizard_multi_all will not show this problems anymore.

    As you can see multi lan wizard is not in this list.

    Anyways the situation with shaper in 2.0 is frustrating. Wizards are useless because result of they work need to be edited a lot. Much easier make all this by hands or create your own macro. And old dummy net shaper doesn't work at all. All it allows you to create is pipes. But further attempts to create queues gives you only an error about queue name while name is ok. I understand it's still in development, but see no bug notes about this situation. So is all this a bug or a feature? ;D

    Here is log message on attempt to create queue in limiter:

    php: /firewall_shaper_vinterface.php: SHAPER: could not create queue high on interface uplink because: Array ( [0] => Queue names must be alphanumeric and _ or - only. )
    

    But i see no problem in name "high". Apparently script does.



  • @cybergamer:

    Also load balancing on multiple WANs is just not very workable at all…

    Do you mean server load balancing, or outbound gateways/routing groups? We've been using gateway group balancing in multi-WAN in 2.0 for months and it works well.



  • Hello, I think I know what is going on with this thread.  See ticket 749

    http://redmine.pfsense.org/issues/749

    Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.

    I guess they are still discussing the change, as stated in the ticket.

    Traffic shaping only works on traffic leaving an interface, so the wan queues are only for upload traffic.  There really is no way to truly shape the download traffic in a multi lan environment, since each lans queues are handled separately.  It can work most of the time depending on usage patterns... hopefully they will put it back in.
    Josh



  • @stompro:

    Hello, I think I know what is going on with this thread.  See ticket 749

    http://redmine.pfsense.org/issues/749

    Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.

    I guess they are still discussing the change, as stated in the ticket.

    Traffic shaping only works on traffic leaving an interface, so the wan queues are only for upload traffic.  There really is no way to truly shape the download traffic in a multi lan environment, since each lans queues are handled separately.  It can work most of the time depending on usage patterns... hopefully they will put it back in.
    Josh

    Actually i said the same in my post, stompro. ;) And what they've done is not solution of the problem. ALTQ shaper just can't work in this case, old school dummy net shaper (personally like it much more) required for this purpose. At this moment it must be already fixed. Didn't tested it yet, but i will in a next few hours. Wizards must be fixed. If shaper shapes only in one direction, who need it at all? It's a nonsense. But we all must understand pfSense 2.0 is not ready yet, it's still in development stage so all we can do is wait or help developers to finish it faster.



  • @stompro:

    Basically someone opened a bug about how the Traffic shaper didn't work very well in multi lan situations, so Ermal removed the ability to shape on LAN interfaces, which technically sort of fixed the problem… since intra lan traffic will no longer be shaped. So it now isn't possible to restrict the download speed while using the wizards.

    That answers my question.  Thanks.  I was wondering why the LAN side bandwidth limit (on LAN Root) didn't work.  I guess I need to find a way to perform LAN shaping via Limiter queues instead.  However, this screws up my ack queues (ack packets are getting dumped into default queue instead).



  • Thanks stompro… that makes sense. Thanks Loke too... I didn't quite understand the way you wrote it, but I understand now. I'll follow up on that ticket. We'd be happy to sponsor development on this, it's pretty crucial and like you said, a shaper that only works one way is pretty useless.



  • @Loke:

    Actually i said the same in my post, stompro. ;) And what they've done is not solution of the problem. ALTQ shaper just can't work in this case, old school dummy net shaper (personally like it much more) required for this purpose. At this moment it must be already fixed. Didn't tested it yet, but i will in a next few hours. Wizards must be fixed. If shaper shapes only in one direction, who need it at all? It's a nonsense. But we all must understand pfSense 2.0 is not ready yet, it's still in development stage so all we can do is wait or help developers to finish it faster.

    Sorry Loke, I see it now, sorry for just repeating you.  I have thought of using dummynet for this, but I haven't figured out how to use multiple dummynet pipes together yet.  I want to keep using the built in limiters for the captive portal users, along with for certain other uses, and also run all traffic through one that limits to %90 of bandwidth, so I don't trigger the buffer bloat http://www.bufferbloat.net/ problem.



  • Well you can clone the WAN config on LAN from the Queue view tab and just change bandwidth settings.
    That will enforce 'download' to be followed with the same rules.



  • Dummynet shaper is quiet simple to understand. And when you'll understand how it works it's very simple to create any configuration you need. ALTQ shaper got a great minus. It can shape traffic only on exit from interface. Dummynet can shape in both directions and can limit speed not only for diff traffic, but also for diff sources/destinations. And this is a huge advantage over ALTQ in some cases.



  • @ermal:

    Well you can clone the WAN config on LAN from the Queue view tab and just change bandwidth settings.
    That will enforce 'download' to be followed with the same rules.

    Thanks Ermal, I've done that now. The shaper limiting is now working as it was some months ago. We're still interested in sponsoring this to come to a good solution for all parties.



  • BTW i think there must be an ability in wizard to select which type of shaper you want to use. In some cases preferred ALTQ in some Dummynet. But now if you want to use dummynet you forced to create a tonne of rules manually. And this is not good. ;D



  • What is dummynet exactly? Is that an internal scheduler that I can't select… cause I don't see it in the available schedulers.



  • He is talking about limiters.
    But altq and limiters are very different in behaviour.



  • @tacfit:

    What is dummynet exactly? Is that an internal scheduler that I can't select… cause I don't see it in the available schedulers.

    Dummynet is completely different type of shaper so it's not the type of scheduler as it's not ALTQ. And ermal is right, it's called limiter in pfSense 2.0. If you want to use dummynet shaper you need to go to limiter tab and create pipes and queue manually. Than you need create corresponded rules to put traffic in queues. To understand dummynet you better look how it's done in m0n0wall. In pfSense it's very hard to understand. Or if you familiar with FreeBSD just look man ipfw and read about dummynet.



  • ok gotcha, thanks.



  • Have been working a lot with shapers and would like to add my 2 cents.

    I have been playing with the altq hfsc queues and would like to say they are very much more powerful and useful than limiters/dummynet/captive portal limits.  However it's a bit harder to understand, and I agree the wizard maybe doesn't shed much light on the workings…

    As for the problem of mulitple LANs being limited when you limit the LAN interface, I think this is the same problem I ran into when putting a slow limit on the LAN interface, it also slows down the web interface to pfSense!  The solution is to set the LAN interface queue rate at wire speed, and then create child queues to shape internet traffic, and send traffic to those queues with firewall rules.  put the default queue as a direct child of the LAN with no bandwidth Upperlimit.  Or alternatively use the default queue to shape internet (WAN) traffic, and create an unlimited queue and send inter-LAN traffic specifically to that queue.  The point is that the root LAN interface is set to wire speed.

    I know the wizards and interface doesn't make this intuitive or easily to edit... I find it much easier to edit the xml config file, cloning rules with copy and paste.  But the wizard is a lot better than nothing, and it mostly makes sense, it's just that hfsc and the whole system is not a simple subject!

    Also I found some good information about HFSC that was really useful for me to understand the configuration better, it seems to be very relevent to pfSense:  http://www.iijlab.net/~kjc/software/TIPS.txt  See section 3.

    I think it has caused a lot of confusion that the wizard doesn't create LAN interface queues, but there is a lot of confusion anyway about shaping incoming traffic!  I remember reading, "once the packets have come to your WAN interface from the internet, they have already used your bandwidth, so you can't shape that traffic."  Well this isn't true at all, because of how TCP works.  That is exactly how you shape any TCP traffic, by dropping packets when it starts to go too fast, which is what happens naturally without the shaper anyway, but with altq you can have a lot of control over which streams get slowed down by dropping packets.  I know, i hate to drop packets at all after they have come all the way over an expensive satellite link, but that's how it works. If you want to share the bandwidth, you have to limit some streams, meaning you have to get the server to stop sending you so many packets, and that is done by not sending ACKs, which is done by dropping the packets.  Still trying to understand the effects of delaying instead of dropping, but that causes even more problems so far.  ECN makes lots of sense, but not sure yet if it actually works, maybe it isn't widely implemented across the internet.

    Anyway, just wanted to post somewhere the little that I have learned lately, before i forget it!  Should make for good discussion.

    edit: UDP traffic goes to queues properly now!!!



  • Good tip, thanks!


Locked