QoS/Traffic Shaping information and tips.

[eri--]

Some basic information about traffic-shaping.

@http://forum.pfsense.org/index.php/topic:

pfSense uses ALTQ for its QoS which applies to the outgoing traffic on an interface. This means that if you have 2 interfaces LAN/WAN and an internet connection of Up 256Kb/s and Down 1Mb/s than the WAN queue has the upload limit and the LAN one has the download limit.
This is why i ask for interfaces during the wizard. Since i need to know in what interfaces the Upload/download values has to be applied. Each interface can have different schedulers (PRIQ/CBQ/HFSC for now).

This means that if you enable the traffic shaper EVERY traffic that leaves any interfaces where the shaper is active will be shaped or better needs to be classified to a queue. Every interface needs explicitly 1 AND ONLY 1 DEFAULT QUEUE. It means that unclassified traffic by rules will go to this queue.

The different schedulers give you flexibility on how to achieve your QoS. The best one is HFSC but it is the harder to configure right without the knowledge of it. Mos people have an hard time groking what "decoupled delay and bandwidth" means and i would rather make them choose PRIQ then have to go through the hassle of explaining that.
PRIQ is the simplest one, you set the bandwidth to apply(this is an hard upperlimit) meaning it will not use more than that.

NOTE: that i am just describing only one part of the configuration below. Meaning it is only the upload part which will be applied on the WAN interface. For the LAN/download one or any other interface where traffic will pass on a configuration should be applied to make it complete. Usually this configuration is just a copy of this one.

After that you setup different priority for different queues maximum is 15, meaning you can have maximum of 15 queues.
PRIQ queues can not have childs.
So lets says you want to give priorities in this order(the first has the highest priority):
VoIP
VNC
SSH
HTTP
ICMP
Penalty
With PRIQ you just setup this queue schema:
VoIP priority 7
VNC priority 6
SSH priority 5
HTTP priority 4
ICMP priority 3
Penalty (priority 1 default)

NOTE: that i am not setting a bandwidth value anywhere here and just letting the ISP do the actual capping of the bandwidth.
Though i strongly suggest to tweak the tbrconfig size of the interface. Later more on what this is.

And set rules to choose the priorities to the specific traffic by choosing the queues in the rules.
This is as simple as it can get. And is the most recommended for home uses. Since you are the only customer and have not so much need of sharing bandwidth.

CBQ is class based scheduling. It allows you two define a tree of classes.
Each queue can have a priority setup from 1 - 7 which will be honored and give specific queue a bandwidth value in percentage or specific value regarding to its parent. Furthermore you can have a borrow action which will give you more bandwidth than actually configured when the parent says it has some spare one.
So lets take the same example as above and say that we want to share the bandwidth between 2 subnets.
The following logical schema makes sense then:

–-qTotalBandwidth (Value of upload bandidth)
------qSubnet1 (50% bandwidth)
------qSubnet2 (50% bandwidth)

Now i setup rules that say subnet1 traffic goes to the qSubnet1 and subnet2 traffic goes to the qSubnet2
If i wanted that subnets share available bandwidth between them just add the borrow option to both of them and it will activate the sharing.

Now if i wanted to add priority for each subnet the logic would say:
---qTotalBandwidth (Value of upload bandidth borrow )
------qSubnet1 (45% bandwidth priority 1)
--------------q1VoIP (priority 7 bandwidth 30% borrow )
--------------q1VNC (priority 5 bandwidth 30% borrow )
--------------q1HTTP (priority 4 bandwidth 30% borrow )
------qSubnet2 (45% bandwidth pruority 1 borrow )
--------------q2VoIP (priority 7 bandwidth 30% borrow )
--------------q2VNC (priority 5 bandwidth 30% borrow )
--------------q2HTTP (priority 4 bandwidth 30% borrow )
------qPenalty (priority 1 bandwidth 10% default)

Setup the rules accordingly and it should work like a charm.
What that schema means is give priority on the 2 subnets to VoIP than VNC than HTTP than every other traffic would go to the Penalty queue and will be capped to total 10% of its parent.

This is called whitelist policy where we choose what is friendly traffic and for the other we do not care and let the qPenalty queue handle it.

Now HFSC is the most sophisticated one and the most confusing one to people that do not have the proper knowledge.
It decouples delay and bandwidth.
What that sentence means is that often you need realtime traffic that has delay(time as milliseconds or seconds) bound for which you do not want the normal limit to apply.
I.E. i have VoIP traffic that uses UDP protocol with packet sizes of 1.2Kbit which needs a delay of 30ms to feel as normal phone call.
But also i want a hard limit, 64Kb, on all the bandwidth that VoIP traffic consumes on my network.
All this is exposed to the user through 3 parameters. m1 d and m2. Where:
m1 = bandwidth needed in d time
d = delay(in milliseconds)
m2 = hard limit
So if create a config as:  m1 = 1.2Kb d = 30 m2 = 64Kb
it means that i want that in d time m1 traffic gets served without checking m2. After that m2 will get checked and if the limit has been reached backlog/queue packet.
Now there are three such schedulers in HFSC. Realtime, Linkshare, Upperlimit.
Realtime is the first scheduler that is run every time. Meaning if we are trying to send a packet the Realtime scheduler will be asked if it has one. After that the Linkshare scheduler takes the lead and if it exceeds some limits the Upperlimit one overrides its decision.
So getting back from theory, when the VoIP traffic above reaches the limit m2 it will be scheduled by the linkshare service curve till VoIP traffic gets back under m2 realtime limit. That's why you have to specify always the bandwidth parameter which is the same as specifying m2 parameter of linkshare.
When both bandwidth and linkshare m2 parameters are specified the m2 parameter is the one that prevails.

So getting back to the example we used with PRIQ/CBQ we would have:
---qTotalBandwidth (Value of upload bandidth )
------qSubnet1 (50% bandwidth)
--------------q1VoIP (bandwidth 30%)
--------------q1VNC (bandwidth 30%)
--------------q1HTTP (bandwidth 30%)
------qSubnet2 (50% bandwidth)
--------------q2VoIP (bandwidth 30%)
--------------q2VNC (bandwidth 30% )
--------------q2HTTP (bandwidth 30%)
------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

This is the same config replicating CBQ one. As you see HFSC has the borrowing of CBQ on by default and you can override it with the upperlimit parameter. Now to have really the power of HFSC server us we would better configure it as:

---qTotalBandwidth (Value of upload bandwidth )
------qSubnet1 (50% bandwidth)
--------------q1VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
--------------q1VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
--------------q1HTTP (bandwidth 30%)
------qSubnet2 (50% bandwidth)
--------------q2VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
--------------q2VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
--------------q2HTTP (bandwidth 30%)
------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

I consider VoIP and VNC realtime traffic as it is Audio and Video and setup they parameters and delay.

Now to have some bursting effects on with HFSC you can play with m1 and m2.
Let say that we have a line that allows the upload to burst to 2Mbits/s for 5seconds and after that it goes to 1Mbit/s
then setup the qTotalBandwidth, in the scheme above, linkshare parameters to m1 = 2Mb d = 5000 m2 = 1Mbit/s
Here the upperlimit bursting configuration is not necessarysince the ISP infoces that.
If we wanted to enforce a 512 hard limit with a burstable of 1 sec to 1Mbit/sfor qSubnet1 we have to add this configuration to that queue
upperlimit m1 = 1Mb d = 1000 m2 = 512Kbit/s

Now in pfSense there are 2 strategies that can be applied for QoS.
1- is white listing policy which selects the traffic we are interested on and sends it to the policy(queue) we have configured for it and all the other one is sent to the default queue which in this case is configured with very low priority and low bandwidth.
This is even the policy that the wizard tend to express.

IE with PRIQ scheduler it means:
qClassifiedtraffic(priority 7)
qDefault(default priority 1)

2- is black listing priority. This policy tries to identify traffic we do not want and send it to penalty queues. All the other traffic may be classified to other queues we are interested on or send it to the default queue, which in this policy has higher priority and more bandwidth than in the whitelisting case.

IE with PRIQ scheduler it means:
qDefault(default priority 7)
qPenalty(priority 1)

Questions? Smiley

Now back to why you need to disable the anti-lockout rule and the default LAN rule.
The pf packet filter is stateful and if it registers a state about a stream of traffic it will not check the ruleset again.
On this packet filter that is used in pfSense traffic is assigned to a queue by specifying it explicitly with the rule that matches the traffic/ the rule that creates the state.
The default anti-lockout rule is the same as the default lan rule just createt automatically for the user to prevent his from doing stupid things.
But this rule is to generic as it matches all the traffic passing from lan and nothing else in the ruleset gets executed. As such it sends all the traffic to the default queue which is not what the user wants with a QoS policy on.
The same applies to the default LAN rule pfSense ships with. Since now you have to explicitly choose the queue the traffic has to go when creating a rule there is no easy solution to this other than disable these settings and have more fine tuned rules for classifying traffic to the propper queue.

Ermal

[ttlinna]

Thousand thanks for Ermal for writing this very clarifying tip!

There are a couple of questions this raises:

• First: You set the total bandwidth in the example to 110% (50% for qSubnet1 and qSubnet2 plus 10% for qPenalty). That causes errors. I think you must have meant 45% for subnets.

• Second: pfSense doesn't allow to have hierarchical structure deeper than two levels (at least that I'm not aware of). Could you explain how would it be possible to have qTotalUp, which has children qWanFastResponse, qWANBulk and qWANP2P and these classes have children queues (qWANFastResonse has ACK, VoIP, Games, dns and so on)? I tried it, but did not succeed since if a queue is parent queue itself, it cannot have parents. That's something pfSense implementation of HFSC is missing. Of course this there's workaround so that one doesn't define qTotalUp, but then you have to play with each class' bandwidth limits as numerical value which causes problems for administering them in larger scale.

edit: I found the original message. There was stated that this type of implementation is available on version 2

Third: Stateful packet filter confuses me. Let me ask you something by example.

I have qP2P set UpperLimit 10000Kb  360000 ms (=1 hour)  500Kb. Now first user starts P2P traffic and gets 10Mb for first hour. After one hour his share is dropped to 500Kb. Then after a while new user starts P2P, is he getting also 10Mb for first hour? Or is he being punished for the earlier user's behavior?

Or if this is done by states, is it so that when a new P2P state (file changes, new download starts from another peer) is created, the time counter is reset and therefore user practically always gets 10Mb?

BTW, would it be possible to create rules based on clock so that during nighttime there would be different kind of rules?

Thanks in advance,

Tommi

[eri--]

Some useful information too.

http://www.probsd.net/pf/index.php/Hednod%27s_HFSC_explained

be aware that the limits on the m1 parameter do not apply on pfSense ie m1 can be smaller than m2.