Traffic Shaping and HFSC



  • I'm wondering if someone can give a clear, concise description of HFSC as it is implemented in pfSense.  I've been reading up on it, and the best documentation I can find is here: https://calomel.org/pf_hfsc.html

    While this seems to offer practical advice about how to set up your queues, I don't believe this is using the benefits of HFSC.

    In order to explain what I mean, let me try and explain what I understand about HFSC:

    HFSC is capable of using non-linear service curves to decouple delay and bandwidth for a given service.  This seems to be the defining difference between HFSC and other queueing algorithms.  The way HFSC accomplishes this is by using 3 parameters to describe the curve: m1, d, m2.  d is the x projection of the inflection point between the m1 line and the m2 line, which essentially defines where the curve "changes" from one bandwidth to another.  If m1 is set to 0, then HFSC essentially acts like other queueing algorithms, such as the Virtual Clock algorithm, etc.

    From what I can tell, most setups always set m1 to 0, which seems silly to me because in effect you're just using another queueing algorithm, so why not just use another (simpler) queueing algorithm?  I guess the answer to that question is flexibility, because perhaps for most people, using a linear "curve" does basically what they want (basic queueing with priorities) vs. a more complicated, actual curve which is more sophisticated but also harder to understand.

    Can someone clarify how specifying an actual curve (using all 3 params, m1, d, and m2) decouples delay from bandwidth?  And, if this isn't too much to ask, can you describe why the default pfSense queues that are generated by the wizard don't even bother?  In fact, it seems that most people who use HFSC don't even bother to specify these parameters, so I'm not trying to single out pfSense or badmouth it or anything, I'm just really trying to understand this stuff so I have a good grip on how the shaping is actually working.  The link above does a good job of describing the different parameters (bandwidth, realtime, linkshare, etc. (although the descriptions there of the relationship between linkshare and bandwidth are different than those in post #4 on the following topic from the pfSense forums http://forum.pfsense.org/index.php/topic,3050.0.html)) and how they affect the queues.

    Thanks in advance!



  • In  1.2 it does not try to use at HFSC full capability on 2.0 it is, during the wizards.

    Actually i already have described how to configure HFSC. The info is somewhat scattered around and the bounty part for the shaper has the longest discussion barried during those long posts.
    If you're looking for a quick guide there is some post in this category by me that explains the basics.

    Calomel.org is just basic and not really the best options around.

    So do your searching/learning by gathering the info in the forums.



  • Sure, thanks Ermal.  I appreciate the response.  I'll poke around the forum and see if I can find more detailed information, but do you have any specific posts I should take a look at?  When I do a search for 'hfsc' nothing is turned up, so finding the information is pretty tricky.



  • Sorry Ermal, I didn't read your response carefully enough.  I see now that you're referencing the "bounty of the shaper" topic, I'll go take a look.

    Thanks again!



  • In case anyone is looking for this stuff as I was, here is a wonderful post from Ermal from the above-mentioned thread that goes a long way towards clearing a lot of this stuff up.  This was 15 pages into that topic, btw, so a little hard to track down.

    I will explain some things but you have to wait for the next update to actually try to configure it.

    pfSense uses ALTQ for its QoS which applies to the outgoing traffic on an interface. This means that if you have 2 interfaces LAN/WAN and an internet connection of Up 256Kb/s and Down 1Mb/s than the WAN queue has the upload limit and the LAN one has the download limit.
    This is why i ask for interfaces during the wizard. Since i need to know in what interfaces the Upload/download values has to be applied. Each interface can have different schedulers (PRIQ/CBQ/HFSC for now).

    This means that if you enable the traffic shaper EVERY traffic that leaves any interfaces where the shaper is active will be shaped or better needs to be classified to a queue. Every interface needs explicitly 1 AND ONLY 1 DEFAULT QUEUE. It means that unclassified traffic by rules will go to this queue.

    The different schedulers give you flexibility on how to achieve your QoS. The best one is HFSC but it is the harder to configure right without the knowledge of it. Mos people have an hard time groking what "decoupled delay and bandwidth" means and i would rather make them choose PRIQ then have to go through the hassle of explaining that.
    PRIQ is the simplest one, you set the bandwidth to apply(this is an hard upperlimit) meaning it will not use more than that.

    NOTE: that i am just describing only one part of the configuration below. Meaning it is only the upload part which will be applied on the WAN interface. For the LAN/download one or any other interface where traffic will pass on a configuration should be applied to make it complete. Usually this configuration is just a copy of this one.

    After that you setup different priority for different queues maximum is 15, meaning you can have maximum of 15 queues.
    PRIQ queues can not have childs.
    So lets says you want to give priorities in this order(the first has the highest priority):
    VoIP
    VNC
    SSH
    HTTP
    ICMP
    Penalty
    With PRIQ you just setup this queue schema:
    VoIP priority 7
    VNC priority 6
    SSH priority 5
    HTTP priority 4
    ICMP priority 3
    Penalty (priority 1 default)

    NOTE: that i am not setting a bandwidth value anywhere here and just letting the ISP do the actual capping of the bandwidth.
    Though i strongly suggest to tweak the tbrconfig size of the interface. Later more on what this is.

    And set rules to choose the priorities to the specific traffic by choosing the queues in the rules.
    This is as simple as it can get. And is the most recommended for home uses. Since you are the only customer and have not so much need of sharing bandwidth.

    CBQ is class based scheduling. It allows you two define a tree of classes.
    Each queue can have a priority setup from 1 - 7 which will be honored and give specific queue a bandwidth value in percentage or specific value regarding to its parent. Furthermore you can have a borrow action which will give you more bandwidth than actually configured when the parent says it has some spare one.
    So lets take the same example as above and say that we want to share the bandwidth between 2 subnets.
    The following logical schema makes sense then:

    –-qTotalBandwidth (Value of upload bandidth)
    ------qSubnet1 (50% bandwidth)
    ------qSubnet2 (50% bandwidth)

    Now i setup rules that say subnet1 traffic goes to the qSubnet1 and subnet2 traffic goes to the qSubnet2
    If i wanted that subnets share available bandwidth between them just add the borrow option to both of them and it will activate the sharing.

    Now if i wanted to add priority for each subnet the logic would say:
    ---qTotalBandwidth (Value of upload bandidth borrow )
    ------qSubnet1 (45% bandwidth priority 1)
    --------------q1VoIP (priority 7 bandwidth 30% borrow )
    --------------q1VNC (priority 5 bandwidth 30% borrow )
    --------------q1HTTP (priority 4 bandwidth 30% borrow )
    ------qSubnet2 (45% bandwidth pruority 1 borrow )
    --------------q2VoIP (priority 7 bandwidth 30% borrow )
    --------------q2VNC (priority 5 bandwidth 30% borrow )
    --------------q2HTTP (priority 4 bandwidth 30% borrow )
    ------qPenalty (priority 1 bandwidth 10% default)

    Setup the rules accordingly and it should work like a charm.
    What that schema means is give priority on the 2 subnets to VoIP than VNC than HTTP than every other traffic would go to the Penalty queue and will be capped to total 10% of its parent.

    This is called whitelist policy where we choose what is friendly traffic and for the other we do not care and let the qPenalty queue handle it.

    Now HFSC is the most sophisticated one and the most confusing one to people that do not have the proper knowledge.
    It decouples delay and bandwidth.
    What that sentence means is that often you need realtime traffic that has delay(time as milliseconds or seconds) bound for which you do not want the normal limit to apply.
    I.E. i have VoIP traffic that uses UDP protocol with packet sizes of 1.2Kbit which needs a delay of 30ms to feel as normal phone call.
    But also i want a hard limit, 64Kb, on all the bandwidth that VoIP traffic consumes on my network.
    All this is exposed to the user through 3 parameters. m1 d and m2. Where:
    m1 = bandwidth needed in d time
    d = delay(in milliseconds)
    m2 = hard limit
    So if create a config as:  m1 = 1.2Kb d = 30 m2 = 64Kb
    it means that i want that in d time m1 traffic gets served without checking m2. After that m2 will get checked and if the limit has been reached backlog/queue packet.
    Now there are three such schedulers in HFSC. Realtime, Linkshare, Upperlimit.
    Realtime is the first scheduler that is run every time. Meaning if we are trying to send a packet the Realtime scheduler will be asked if it has one. After that the Linkshare scheduler takes the lead and if it exceeds some limits the Upperlimit one overrides its decision.
    So getting back from theory, when the VoIP traffic above reaches the limit m2 it will be scheduled by the linkshare service curve till VoIP traffic gets back under m2 realtime limit. That's why you have to specify always the bandwidth parameter which is the same as specifying m2 parameter of linkshare.
    When both bandwidth and linkshare m2 parameters are specified the m2 parameter is the one that prevails.

    So getting back to the example we used with PRIQ/CBQ we would have:
    ---qTotalBandwidth (Value of upload bandidth )
    ------qSubnet1 (50% bandwidth)
    --------------q1VoIP (bandwidth 30%)
    --------------q1VNC (bandwidth 30%)
    --------------q1HTTP (bandwidth 30%)
    ------qSubnet2 (50% bandwidth)
    --------------q2VoIP (bandwidth 30%)
    --------------q2VNC (bandwidth 30% )
    --------------q2HTTP (bandwidth 30%)
    ------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

    This is the same config replicating CBQ one. As you see HFSC has the borrowing of CBQ on by default and you can override it with the upperlimit parameter. Now to have really the power of HFSC server us we would better configure it as:

    ---qTotalBandwidth (Value of upload bandwidth )
    ------qSubnet1 (50% bandwidth)
    --------------q1VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
    --------------q1VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
    --------------q1HTTP (bandwidth 30%)
    ------qSubnet2 (50% bandwidth)
    --------------q2VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
    --------------q2VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
    --------------q2HTTP (bandwidth 30%)
    ------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

    I consider VoIP and VNC realtime traffic as it is Audio and Video and setup they parameters and delay.

    Now to have some bursting effects on with HFSC you can play with m1 and m2.
    Let say that we have a line that allows the upload to burst to 2Mbits/s for 5seconds and after that it goes to 1Mbit/s
    then setup the qTotalBandwidth, in the scheme above, linkshare parameters to m1 = 2Mb d = 5000 m2 = 1Mbit/s
    Here the upperlimit bursting configuration is not necessarysince the ISP infoces that.
    If we wanted to enforce a 512 hard limit with a burstable of 1 sec to 1Mbit/sfor qSubnet1 we have to add this configuration to that queue
    upperlimit m1 = 1Mb d = 1000 m2 = 512Kbit/s

    Now in pfSense there are 2 strategies that can be applied for QoS.
    1- is white listing policy which selects the traffic we are interested on and sends it to the policy(queue) we have configured for it and all the other one is sent to the default queue which in this case is configured with very low priority and low bandwidth.
    This is even the policy that the wizard tend to express.

    IE with PRIQ scheduler it means:
    qClassifiedtraffic(priority 7)
    qDefault(default priority 1)

    2- is black listing priority. This policy tries to identify traffic we do not want and send it to penalty queues. All the other traffic may be classified to other queues we are interested on or send it to the default queue, which in this policy has higher priority and more bandwidth than in the whitelisting case.

    IE with PRIQ scheduler it means:
    qDefault(default priority 7)
    qPenalty(priority 1)

    Questions? Smiley



  • I want to add a link to the quote I pulled from ermal above, because the couple of exchanges that happen after it in the original thread could also be helpful to people looking this stuff up.

    http://forum.pfsense.org/index.php/topic,2718.msg48336.html#msg48336


Log in to reply