Traffic shaper changes [90% completed, please send money to complete bounty]

sullrich

Should be soon.

eri--

For all the bounty contributors.
In the same link as before will find the updated images with several problem fixed.

eri--

Get the one with the highest date on it. as -20080324 ;)

songus

Ill add 50 to the bounty, should i send them now? when will the image be available¿?

hoba

@songus:

Ill add 50 to the bounty, should i send them now? when will the image be available¿?

All bountysupporters get exclusive access to the testingimages and are welcome to testdrive and report back. All others will have to wait for now until there are official builds including the changes. Feel free to send the money in right now.

eri--

I explained it a page before:
http://forum.pfsense.org/index.php/topic,2718.180.html

the queue wizard is really a work in progress. the first part is difficult to understand and has text labels in code style. the second part, the one with traffic type prioritization, is an heritage of the old shaper wizard but has no reason to exist, 'cause is not applied anywhere and there's no interface to edit. It seems that now the assignment of traffic type to queues is done within each firewall rule.

Well you do not need any interface to choose since it applies to all interfaces.
Read my explanation of the Floating Tab.

As for the names i will make them more friendly.

BTW, since you are a user what part of the first part you didn't understand?

k3rmit

sorry, i just found your 1st explanation, that's why i deleted my post…

i'll try apply the rules as by your tutorial and in case get back to you with a good feedback.

to answer your question, if for example i click on the "single wan multi lan" wizard, i'm asked for the number of connections: in my understanding this should be the LAN and the DMZ, but in the next step i have WAN and OPT1 (DMZ) grouped in the "setup connections speed" section, like if we were talking about two WANs, while the DMZ has to be considered like a LAN section.
i'm puzzled here because given i'm configuring multiple lans, as by wizard name, i should be asked just for the wan bandwidth and then describing the lan part. this could be a limit of my understanding of the shaping mechanism within pf, but i have to admit that the wizard isn't a lot descriptive about what am i doing with the info i'm entering and the options i'm choosing.

i just want to avoid traffic shaping between the LAN and DMZ and meanwhile shape all traffic from all interfaces to WAN: from your tutorial i understand that i just need to assign floating rules to queues. i have a solid heritage of rules assigned to each interface, so i think it will take time to make it work correctly. is there any monitoring/debugging application for pf out there?

btw, thanks for the prompt answer.

eri--

Oh for the Multi Lan wizard i might have missed some labels changes.
Though it really asks you for the number of LAN's. As i can not guess what interfaces are considered LAN in your cases.
You see WAN in there since i need to know on which interface is the internet connection connected.

If you do not want to shape traffic between DMZ and LAN, on the traffic shaper config:
1- Click the lan root node on the tree. Set its interface bandwidth to the same as you Network card speed(i.e. 100Mb)
2- Delete the traffic shaper config on both LAN and DMZ
3- Create a queue called qInternet in both the LAN and DMZ interface and setup it with the download speed of your internet connection.
If you have choosen HFSC scheduler make its linkshare m1=m2=link download speed and d =something.
4- Create a DMZ queue on both the LAN and DMZ interface. Setup its bandwidth = Lan root speed - speed of qInternet queue
5- Under the qInternet queue replicate the queues that gets created by the wizard, so that the internet shaping for LAN and DMZ works ok.

Than create a rule that matches local traffic(traffic between LAN and DMZ) and sends it to the qDMZ queue so it does not have limitations from the shaper.

I am testing this setup and will make the changes for the Multi Lan wizard, at least, to produce the above automatically.

You will get it with the next update which fixes the other reported issues.

Just a stupid text illustration of the above is:
WAN
–-qACK
---qDefault
---qP2P
---qVoIP
---qOthersHigh
LAN
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
DMZ
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ

On the floating rules tab make a rule:
1- pass
2- select LAN and DMZ interface
3- Direction any
4- from any (though you might consider only the ports to the DMZ services)
5- to any (though you might consider only the ports to the DMZ services)
6- queue qDMZ

And done.

Another more advanced scheme might be:
WAN
---qACK
---qDefault
---qP2P
---qVoIP
---qOthersHigh
LAN
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
----------qDMZACK
----------qDMZDefault
----------qDMZP2P
----------qDMZVoIP
----------qDMZOthersHigh
DMZ
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
----------qDMZACK
----------qDMZDefault
----------qDMZP2P
----------qDMZVoIP
----------qDMZOthersHigh

And propper rules in place.

sullrich

LANs are easy to determine. Walk the configuration and look for interfaces without a gateway attached to them.

SlickNetAaron

Hi Ermal,

Thanks for allowing access to the new shaper. I see you are continuing to work on it.

I'm having a very hard time trying to figure out how to set this up. I am unable to add queues to interfaces (I got it to succeed only once!) I'm totally not understanding how this shaper is laid out - it just does not seem intuitive.

My setup was explained here: http://forum.pfsense.org/index.php/topic,2718.195.html
If you can help me understand how to set this up, I would be grateful. I would even be willing to write up a HowTo to try to explain the new shaper as well as help form the GUI with you.

Regards,
Aaron

eri--

Can you please post full details of your configuration.
Bandwidths you want to use etc so i can give you a config.

The upgrade you have has 3 issues:
1- you cannot add queues other than on the Lan.
EDIT: You cannot add queues that are childs of parent interface other than LAN. But you can add childs of other queues on any interface.
2- The Status->queues is shifted to the right as for a missing line for displaying the header ok.
3- The rrd graphs has a typo which does not allow to propperly view the queues graph
4- Floating rules are generated after per tab interface rules so if you have some rules in the specific interface tabs(wan/lan tab) they will spoil the floating rules.
This are just regressions of backporting from RELENG_1. In the next update they will be ok.

In your case you should not have any problems since you want to add queues only for LAN so you should be OK.

Now from what i see you want something like this.
Create an alias with the host you want to limit.

On the wizard check the Penalty box and add this alias on this step.
Also check the catchall option of it.
You should have a scheme like this after it.

WAN
–-qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow

This should set you on for anything you want.
You limit the customers through the Alias config and no need to tweak the rules.
Also if you want a hard limit for them set the uppelimit of qOthersLow(value m2) to the required limit.

Since of issue 4 you do not need any settings on Wan apart specific things you want to block.
Disable anti lockout rule.
And replicate the LAN default pass in rule to the Floating tab and disable that one(for this upgrade you are running.

That's all you need to share all the bandwidth evenly in your setup. Since you say the AP's are limited to 6Mb that's as simple as it can get with the upper scheme.
You can optimize VoIP rules by converting the rules for VoIP to use DSCP(diffserv code point) instead of port based ones; if you know that they use a specific DSCP mark.

Tell me if this suits you.

The other scheme if you wanted to have the hard limit to 6Mb setuped on the pfSense is:
WAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
---qAP1 (m1=m2=6Mb d=line delay)
------qAP1ACK
------qAP1P2P
------qAP1VoIP
------qAP1OthersHigh
------qAP1OthersDefault
------qAP1OthersLow
---qAP2 (m1=m2=6Mb d=line delay)
------qAP2ACK
------qAP2P2P
------qAP2VoIP
------qAP2OthersHigh
------qAP2OthersDefault
------qAP2OthersLow

or
WAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
------qACK
------qP2P
---qVoIP
------qOthersHigh
------qAP1OthersHigh
------qAP2OthersHigh
---qOthersDefault
------qAP1OthersDefault
------qAP2OthersDefault
---qOthersLow
------qAP1OthersLow
------qAP2OthersLow
On this one set the limits for each AP to the specific queue using upperlimit m2 value. Though i doubt you want their Voip queues to be separate since you want both clients to have seemles VoIP.
The last scheme might give you better results but it is hard to understand for someone not knowing what he is doing.

BTW, if you could gather all my postings about the shaper to something readble and skinned :) i would greatly appriciate. I have not yet found the time to do that.

mikenl

I haven't pledged to the original bounty, but i made a contribution of $50,00 USD.
I appreciate the work done on the traffic shaper, and would love to take a look at it.

SlickNetAaron

Hi Ermal,

Thanks for taking the time to describe the config. While when you draw out the queues definitions and it makes mostly perfect sense, but I am having trouble. The shaper is simply not allowing me to add queues at all! I push ADD Queue button and fill everything out and nothing shows up! The other portion is: getting from the shaper wizard to the end outcome is very,very confusing. The labels are confusing and the interface needs a lot of help. I just went back to m0n0wall 1.3b10 to play with their shaper last night. It is MUCH more intuitive and simple. As simple as it is, it seems to have more functionality, including the ability to limit per IP bandwidth (in a very weird way, but it says it's easy LoL). I hear m0n0wall also will honor RADIUS bandwidth attributes as well? I do not mean to offend, by any means, I just think your shaper could be simplified and made a lot easier for the end user.

1 other problem - while trying to add the queues, the Service Curve options were always grayed out even after clicking the checkbox to enable the fields.

In the end it seemed that nothing would do what I told it to?

@ermal:

Can you please post full details of your configuration.
Bandwidths you want to use etc so i can give you a config.

that would be great. Details are below.

WAN: 12mb down / 2mb up (Actually, this is a dyamic WAN.. it will burst up to about 16/2.5, but it is committed to 8/1. If we could figure out a dynamic rule, that would be amazing! Otherwise, I think just setting 12/2 will work as long as low priority traffic is limited to below the 8/1 mark). I know several people who are looking for this feature.

Want VNC, SSH, HTTP, ICMP and whatever is customary as higher priority.

As mentioned, there are 2 APs and 1 direct connected router to pfSense. Each AP can have a total of 5 mb of end-user bandwidth (changed from before). Each AP should be able to burst up to the full 2mb upload speed. The 5mb of usable bandwidth on the APs is half-duplex. How do we account for that? (ie, if there is 1mb of upload, then there is only room for 4mb of download.) There will be traffic coming over the APs to my servers on the LAN or OPT1 as well. The other router attached can have equal priority as the APs for WAN bandwidth. Of course this needs to be shared. Identification of which AP or router will have to be by subnet. (10.5.x.y=AP1 and 10.6.x.y=AP2 and and 10.4.x.y=localrouter)

I don't have my OPT1 network figured out yet. It will basically be for servers and such. Severs are currently on LAN subnets. OPT1 will need to share upload/download bandwidth on the WAN - at just below HTTP LAN priority (customers surfing the web should be higher priority, but the catchall rule should be lower priority than the OPT1 servers).

@ermal:

Since of issue 4 you do not need any settings on Wan apart specific things you want to block.
Disable anti lockout rule.
And replicate the LAN default pass in rule to the Floating tab and disable that one(for this upgrade you are running.

I totally don't understand why anti-lockout should be disabled, or what you mean with the LAN rules.

@ermal:

Tell me if this suits you.

The other scheme if you wanted to have the hard limit to 6Mb setuped on the pfSense is:
WAN
–-qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
---qAP1 (m1=m2=6Mb d=line delay)
------qAP1ACK
------qAP1P2P
------qAP1VoIP
------qAP1OthersHigh
------qAP1OthersDefault
------qAP1OthersLow
---qAP2 (m1=m2=6Mb d=line delay)
------qAP2ACK
------qAP2P2P
------qAP2VoIP
------qAP2OthersHigh
------qAP2OthersDefault
------qAP2OthersLow

or

The above setup looks exactly how I thought it should look. (Wasn't sure how the last setup would work, but it makes sense on the surface.) However, I am simply unable to Add these queues in the shaper! And the queues are confusing to me. I think I am figuring out that any queues on the LAN interface actually control the UPLOAD to the WAN? And any queues on the WAN control traffic going TO the LANs? It greatly confuses the matter when we don't want traffic shaped between LANs (interfaces). How can this be simplified?

@ermal:

BTW, if you could gather all my postings about the shaper to something readble and skinned :) i would greatly appriciate. I have not yet found the time to do that.

I think if I can get a more thorough understanding of the shaper I could write an overview to get people to understand some of the basics myself and others are having difficulty with. It is sometimes hard to read your descriptions ;) I'm pretty good at documentation - as long as I have a thorough understanding myself. Are all of your posts regarding the shaper only in this thread?

Regards,
Aaron

bogus

Well, I am having some problems.

Before I get into it here my setup:

I am running pfSense on a laptop with
CPU: Intel(R) Pentium(R) III Mobile CPU 1200MHz (1196.02-MHz 686-class CPU)
256MB RAM.

The internal nic is
xl0: <3Com 3c905C-TX Fast Etherlink XL>
and the second nic is
dc0: <xircom 10="" x3201="" 100basetx="">on Cardbus.

On dc0 I have three VLANs for the ADSL links (2x1Mbps/512K and 1x2Mbps/512k) terminated
with modems/router providing 192.168.10.0/24, 192.168.20.0/24 and 192.168.30.0/24 networks.
LAN is on 192.168.100.0/24
All three ADSL links are load-balanced with failover.

So far so good. I never had any performance problems with this setup and the webgui and also ssh were pretty snappy.
CPU is never more 20% used and memory is usually around 30% usage (swap is just untouched).

The primary goal is to provide 128kbit/s garanteed bandwidth for VoIP (never more than 2-3 sim. calls).
Everything else could use the remaining bandwidth as desired but limiting P2P traffic to max. 10kbit/s (shared between all users).
Secondary goal would be to provide higher priority to Skype traffic and to integrate squid transparently into this
load-balanced/traffic-shaping environment, but that would be a bonus.
But Squid is currently not installed.

What did I do?
Updating the box to 1.2-RELEASE-20080324-1409 went without problem.
Running the "Single LAN/Multi WAN Wizard" and entering the desired values according to the goals above.
But once I press the "Finish" button the webgui stopps responding, often times out. No more internet access.
Even top on ssh does not update anymore.
Finally 5 minutes I was able to get back to "Remove Shaper" and everything went back to normal.

I tried both nominal and real values for the bandwidth (e.g. 1024/512 and 850/400)
I tried all 3 connections at once and only one connection.
All with the same result.

Does the minimum or recommended hardware requirements for the new shaper changed so much?
Do I need to wait longer until the queues have fully initialized?
Is a reboot necessary?</xircom>

eri--

Hmmm no nothing has changed as for requirements.

Another case that you can check if you have checked catchall option in the wizard and limited it to 10Kb and have the Anti-lockout option on.
Plus the default lan rule makes things worse since of the issue that update has with floating rules.

It will behave that way.

My recomandation before running the wizard make a copy of the LAN rule to the floating tabs without the quick keyword than disable the Default Lan rule altogether.
Disable the antilockout rule.
Than run the wizard.

The antilockout rule is the worse for the new shaper since it sends all lan traffic to the default queue(which in your case is the catchall=10Kb/s) and you do not see the effect of the new shaper at all. But i cannot do anything about it other than warn about it.
The default LAN rule one should be fixed with the new update you will get.

Just to let you kow: cvstrac.pfsense.com/timeline(all the fixes that went on).
I fixed all the remaining issues i have listed above plus the "By queue" view now allows cloning full interfaces to replicate those multi-level queues on multiple interfaces easily.
The wizards would generate 2 level queues by default for local interfaces:
LAN
–-qInternet
--------qACK
.
.
.
---qLocal
And the multi Lan wizard setups a rule to send the traffic between the Local interfaces to the qLocal queue.

When the new build finishes and i test the image will notify again.

eri--

I will explain some things but you have to wait for the next update to actually try to configure it.

pfSense uses ALTQ for its QoS which applies to the outgoing traffic on an interface. This means that if you have 2 interfaces LAN/WAN and an internet connection of Up 256Kb/s and Down 1Mb/s than the WAN queue has the upload limit and the LAN one has the download limit.
This is why i ask for interfaces during the wizard. Since i need to know in what interfaces the Upload/download values has to be applied. Each interface can have different schedulers (PRIQ/CBQ/HFSC for now).

This means that if you enable the traffic shaper EVERY traffic that leaves any interfaces where the shaper is active will be shaped or better needs to be classified to a queue. Every interface needs explicitly 1 AND ONLY 1 DEFAULT QUEUE. It means that unclassified traffic by rules will go to this queue.

The different schedulers give you flexibility on how to achieve your QoS. The best one is HFSC but it is the harder to configure right without the knowledge of it. Mos people have an hard time groking what "decoupled delay and bandwidth" means and i would rather make them choose PRIQ then have to go through the hassle of explaining that.
PRIQ is the simplest one, you set the bandwidth to apply(this is an hard upperlimit) meaning it will not use more than that.

NOTE: that i am just describing only one part of the configuration below. Meaning it is only the upload part which will be applied on the WAN interface. For the LAN/download one or any other interface where traffic will pass on a configuration should be applied to make it complete. Usually this configuration is just a copy of this one.

After that you setup different priority for different queues maximum is 15, meaning you can have maximum of 15 queues.
PRIQ queues can not have childs.
So lets says you want to give priorities in this order(the first has the highest priority):
VoIP
VNC
SSH
HTTP
ICMP
Penalty
With PRIQ you just setup this queue schema:
VoIP priority 7
VNC priority 6
SSH priority 5
HTTP priority 4
ICMP priority 3
Penalty (priority 1 default)

NOTE: that i am not setting a bandwidth value anywhere here and just letting the ISP do the actual capping of the bandwidth.
Though i strongly suggest to tweak the tbrconfig size of the interface. Later more on what this is.

And set rules to choose the priorities to the specific traffic by choosing the queues in the rules.
This is as simple as it can get. And is the most recommended for home uses. Since you are the only customer and have not so much need of sharing bandwidth.

CBQ is class based scheduling. It allows you two define a tree of classes.
Each queue can have a priority setup from 1 - 7 which will be honored and give specific queue a bandwidth value in percentage or specific value regarding to its parent. Furthermore you can have a borrow action which will give you more bandwidth than actually configured when the parent says it has some spare one.
So lets take the same example as above and say that we want to share the bandwidth between 2 subnets.
The following logical schema makes sense then:

–-qTotalBandwidth (Value of upload bandidth)
------qSubnet1 (50% bandwidth)
------qSubnet2 (50% bandwidth)

Now i setup rules that say subnet1 traffic goes to the qSubnet1 and subnet2 traffic goes to the qSubnet2
If i wanted that subnets share available bandwidth between them just add the borrow option to both of them and it will activate the sharing.

Now if i wanted to add priority for each subnet the logic would say:
---qTotalBandwidth (Value of upload bandidth borrow )
------qSubnet1 (45% bandwidth priority 1)
--------------q1VoIP (priority 7 bandwidth 30% borrow )
--------------q1VNC (priority 5 bandwidth 30% borrow )
--------------q1HTTP (priority 4 bandwidth 30% borrow )
------qSubnet2 (45% bandwidth pruority 1 borrow )
--------------q2VoIP (priority 7 bandwidth 30% borrow )
--------------q2VNC (priority 5 bandwidth 30% borrow )
--------------q2HTTP (priority 4 bandwidth 30% borrow )
------qPenalty (priority 1 bandwidth 10% default)

Setup the rules accordingly and it should work like a charm.
What that schema means is give priority on the 2 subnets to VoIP than VNC than HTTP than every other traffic would go to the Penalty queue and will be capped to total 10% of its parent.

This is called whitelist policy where we choose what is friendly traffic and for the other we do not care and let the qPenalty queue handle it.

Now HFSC is the most sophisticated one and the most confusing one to people that do not have the proper knowledge.
It decouples delay and bandwidth.
What that sentence means is that often you need realtime traffic that has delay(time as milliseconds or seconds) bound for which you do not want the normal limit to apply.
I.E. i have VoIP traffic that uses UDP protocol with packet sizes of 1.2Kbit which needs a delay of 30ms to feel as normal phone call.
But also i want a hard limit, 64Kb, on all the bandwidth that VoIP traffic consumes on my network.
All this is exposed to the user through 3 parameters. m1 d and m2. Where:
m1 = bandwidth needed in d time
d = delay(in milliseconds)
m2 = hard limit
So if create a config as: m1 = 1.2Kb d = 30 m2 = 64Kb
it means that i want that in d time m1 traffic gets served without checking m2. After that m2 will get checked and if the limit has been reached backlog/queue packet.
Now there are three such schedulers in HFSC. Realtime, Linkshare, Upperlimit.
Realtime is the first scheduler that is run every time. Meaning if we are trying to send a packet the Realtime scheduler will be asked if it has one. After that the Linkshare scheduler takes the lead and if it exceeds some limits the Upperlimit one overrides its decision.
So getting back from theory, when the VoIP traffic above reaches the limit m2 it will be scheduled by the linkshare service curve till VoIP traffic gets back under m2 realtime limit. That's why you have to specify always the bandwidth parameter which is the same as specifying m2 parameter of linkshare.
When both bandwidth and linkshare m2 parameters are specified the m2 parameter is the one that prevails.

So getting back to the example we used with PRIQ/CBQ we would have:
---qTotalBandwidth (Value of upload bandidth )
------qSubnet1 (50% bandwidth)
--------------q1VoIP (bandwidth 30%)
--------------q1VNC (bandwidth 30%)
--------------q1HTTP (bandwidth 30%)
------qSubnet2 (50% bandwidth)
--------------q2VoIP (bandwidth 30%)
--------------q2VNC (bandwidth 30% )
--------------q2HTTP (bandwidth 30%)
------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

This is the same config replicating CBQ one. As you see HFSC has the borrowing of CBQ on by default and you can override it with the upperlimit parameter. Now to have really the power of HFSC server us we would better configure it as:

---qTotalBandwidth (Value of upload bandwidth )
------qSubnet1 (50% bandwidth)
--------------q1VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
--------------q1VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
--------------q1HTTP (bandwidth 30%)
------qSubnet2 (50% bandwidth)
--------------q2VoIP (bandwidth 10% realtime m1 = 1.2Kb d = 30 m2 = 64Kb)
--------------q2VNC (bandwidth 10% realtime m1 = 6Kb d = 50 m2 = 128Kb)
--------------q2HTTP (bandwidth 30%)
------qPenalty (bandwidth 10% default upperlimit m2 = 10%)

I consider VoIP and VNC realtime traffic as it is Audio and Video and setup they parameters and delay.

Now to have some bursting effects on with HFSC you can play with m1 and m2.
Let say that we have a line that allows the upload to burst to 2Mbits/s for 5seconds and after that it goes to 1Mbit/s
then setup the qTotalBandwidth, in the scheme above, linkshare parameters to m1 = 2Mb d = 5000 m2 = 1Mbit/s
Here the upperlimit bursting configuration is not necessarysince the ISP infoces that.
If we wanted to enforce a 512 hard limit with a burstable of 1 sec to 1Mbit/sfor qSubnet1 we have to add this configuration to that queue
upperlimit m1 = 1Mb d = 1000 m2 = 512Kbit/s

Now in pfSense there are 2 strategies that can be applied for QoS.
1- is white listing policy which selects the traffic we are interested on and sends it to the policy(queue) we have configured for it and all the other one is sent to the default queue which in this case is configured with very low priority and low bandwidth.
This is even the policy that the wizard tend to express.

IE with PRIQ scheduler it means:
qClassifiedtraffic(priority 7)
qDefault(default priority 1)

2- is black listing priority. This policy tries to identify traffic we do not want and send it to penalty queues. All the other traffic may be classified to other queues we are interested on or send it to the default queue, which in this policy has higher priority and more bandwidth than in the whitelisting case.

IE with PRIQ scheduler it means:
qDefault(default priority 7)
qPenalty(priority 1)

Questions? :)

Now back to why you need to disable the anti-lockout rule and the default LAN rule.
The pf packet filter is stateful and if it registers a state about a stream of traffic it will not check the ruleset again.
On this packet filter that is used in pfSense traffic is assigned to a queue by specifying it explicitly with the rule that matches the traffic/ the rule that creates the state.
The default anti-lockout rule is the same as the default lan rule just createt automatically for the user to prevent his from doing stupid things.
But this rule is to generic as it matches all the traffic passing from lan and nothing else in the ruleset gets executed. As such it sends all the traffic to the default queue which is not what the user wants with a QoS policy on.
The same applies to the default LAN rule pfSense ships with. Since now you have to explicitly choose the queue the traffic has to go when creating a rule there is no easy solution to this other than disable these settings and have more fine tuned rules for classifying traffic to the propper queue.

Ermal

eri--

@slicknetaaron2:

Hi Ermal,
Thanks so much for taking the time to further explain the shaper. It helps a lot. In my ongoing quest for thorough understanding of the shaper, I would like to confirm my understanding with you and ask a few more clarifying questions. With this, I will hopefully be able to support others and write a tutorial.

I said it is somewhat difficult for a not knowledgeable person to gain thorough understanding afaik.

1. Where the queues are located: Download queue limits go on the LAN side because you do not want to limit the packets coming in from the ISP. We just gotta take them as we get them. Upload limits go on the WAN interface to reorder and shape traffic going OUT to the WAN from all combined LAN interfaces.

It is just the way ALTQ works.

2. It looks like the wizard defaults to HSFC. Somehow we need to figure out a way to make editing the wizard settings more friendly to the user? Somehow hide the complexity of HSFC, but offer the benefits in the background? Maybe shorten the regular queue config to a Basic and an advanced? And explaining how the queue that we are editing will interact with other queues?

What do you find not friendly in there.
I does not default to HFSC just that happens to be the first value in there. And preserve compatibility since it was the only thing you have on 1.2.
I only ask for connection parameters and some schedulers to apply per interface what do you find Advanced in there?!

@ermal:

I.E. i have VoIP traffic that uses UDP protocol with packet sizes of 1.2Kbit which needs a delay of 30ms to feel as normal phone call.
But also i want a hard limit, 64Kb, on all the bandwidth that VoIP traffic consumes on my network.

What does packet length of 1.2kb have to do with the shaper (realtime m1)? Isn't the shaper looking at bandwidth per second, not packet length?

My understanding of VoIP (SIP in particular) is that there is a messaging and call setup on 1 port (5060) and 2 UDP ports used for the actual audio. A typical bandwidth of 96kbps per call (for most common encoder). I have also read that several users need to have a burst of more than 96kbps (say 128kbps) for the first 5-10 seconds of the call. So I would think that if there is 1 phone on the network, m1=128kb d=10000 m2=100kb. That is my understanding of m1, d and m2. Burst speed (m1) for (d) ms and then limit to (m2) for the remainder of the connection. I do not understand where 1.2kb comes from for 30ms. 1.2kb is much less than the required 128kbps and the beginning of a call.

( i will not go into detail why since it is very deep discussion). Take it or leave it.
Or better prove me wrong after you test it ;).
follow this link to for more discussion http://forum.pfsense.org/index.php/topic,2484.0.html

3. Do the m1, d, m2 parameters operate on a PER-SESSION environment? ie. I pick up the phone and it will activate m1, d, m2. Next time I need the phone m1 starts over again? What happens in the case of 2 phones or 10 phones or when you can't know how many phones there are?

m1 and d are per packet. m2 is global.
They can be thought as per session since if you have 4 phones they send traffic at the same rate.
They all have the same delay so packets for each phone will be scheduled on a round robin manner which is approx. the same as a session.
What would be ideal is to create a queue for each phone and give the exact parameters to each queue.
Then you would have perfect/exact per session tracking but even with one queue you would have pretty much the same result.

4. And how does m1, d, m2 work for a dynamic bandwidth WAN queue? When does m1 go into effect? With new sessions? hmm.. I'm hoping so! I think I am beginning to see the power of HSFC!

They scale accordingly if you have not set hard numbers in there.

@ermal:

Now there are three such schedulers in HFSC. Realtime, Linkshare, Upperlimit.
Realtime is the first scheduler that is run every time. Meaning if we are trying to send a packet the Realtime scheduler will be asked if it has one. After that the Linkshare scheduler takes the lead and if it exceeds some limits the Upperlimit one overrides its decision.
So getting back from theory, when the VoIP traffic above reaches the limit m2 it will be scheduled by the linkshare service curve till VoIP traffic gets back under m2 realtime limit. That's why you have to specify always the bandwidth parameter which is the same as specifying m2 parameter of linkshare.
When both bandwidth and linkshare m2 parameters are specified the m2 parameter is the one that prevails.

5. This is kind of confusing.. I think the terms might be mixed up? Here is what I am thinking:
a. RealTime tries to "grab" bandwidth to try to ie. guarantee a good VoIP call
b. Linkshare monitors RealTime to make sure he doesn't get out of hand for this queue's part of the bandwidth for the whole interface? This isn't quite clear to me..? Can we borrow bandwidth if it's not being used elsewhere? There is a note in the shaper that says "Linkshare overrides priority". Can you please explain that? I think we should only use priority?
c. UpperLimit is an Arbitrary maximum for a queue - no matter if we can borrow unused bandwidth or not?

A new packet needs to be transmitted on the wire.
We first ask Realtime scheduler if it has something to transmit.
After we ask the Linkshare which cooperates with Upperlimit to follow the rules.

6. What do you mean by: "you have to specify always the bandwidth parameter which is the same as specifying m2 parameter of linkshare." Which bandwidth parameter are you referring to?

If you click "Add new queue" on top of the form there is a bandwidth parameter and that is what i refer to as "bandwidth parameter".

I'm going to head over to wikipedia to try to understand this more as well.

Good luck you need it :).

@ermal:

I will explain some things but you have to wait for the next update to actually try to configure it.

Do you have an ETA for the update? I just want to decide if I should put 1.2 back on my box and reinstall pfSense onto my network, or if it will be a day or 2 and I can just wait with my network without pfSense for a bit longer.

Default rule & Anti-lockout: Is there a way you can script to change these rules, or give a message to the user that they need to do this?

Thanks for your time!
Aaron

Probably tomorrow.

Ermal

eri--

@slicknetaaron2:

Hi Ermal,

Thanks again for the reply. I apologize, I made a couple errors and did not mean to offend.

@ermal:

I said it is somewhat difficult for a not knowledgeable person to gain thorough understanding afaik.

I was not knowledgeable about hfsc and altq, but to say that I am not knowledgeable and not able to gain thorough understanding… thats just not very nice! :) I am incredibly knowledgeable, just not in this particular area, yet. After spending some time researching last night I am well on my way to thorough understanding and the ability to explain to others how it works. I certainly do not have the knowledge and development skills you possess, but I would like to contribute to the project.

It sound badly but i didn't meant what you understood.
It simply means that without reading too much you would have an hard time with it.
BTW, read the original HFSC paper to understand more.

What do you find not friendly in there.
I does not default to HFSC just that happens to be the first value in there. And preserve compatibility since it was the only thing you have on 1.2.
I only ask for connection parameters and some schedulers to apply per interface what do you find Advanced in there?!

I apologize, I did not mean for that portion of the wizard. That portion is not advanced at all. After reading about hfsc, I totally understand why the queue gui is designed as it is. However, trying to figure out what conn0 and conn1 mean and the "number of connections" questions are very counterintuitive. Is it possible to clear up the descriptions (labels) to ask the number of local and WAN connections? It seems on at least 1-2 of the wizards when I enter "2" in for num of local connections the next screen will not even let me select my LAN port and bugs like that. I am not the only one who had trouble with that (from responses in this tread.)

Yeah i will fix the labels!

@ermal:

I.E. i have VoIP traffic that uses UDP protocol with packet sizes of 1.2Kbit which needs a delay of 30ms to feel as normal phone call.
But also i want a hard limit, 64Kb, on all the bandwidth that VoIP traffic consumes on my network.

( i will not go into detail why since it is very deep discussion). Take it or leave it.
Or better prove me wrong after you test it ;).
follow this link to for more discussion http://forum.pfsense.org/index.php/topic,2484.0.html

I remember reading a thread about VoIP service curve settings. It looks like you were very active in that, and suggested almost exact service queue as I suggested. See here:
http://forum.pfsense.org/index.php/topic,7502.msg42693.html#msg42693

After spending several hours last night reading on hfsc, it is also invalid to have a realtime service curve that is concave. m1 must be higher than m2.
In the same thread linked above, you were telling people to set m1=m2. That is not a curve, but a straight line and is redundant. Not specifying m1 and d will have the same effect. Lastly, There is never a mention of packet size for any of the altq schedulers as you are suggesting for the m1 value for VoIP queue. plus, isn't it impossible to have packet sizes of 125kb as listed in that same post?

Well you cannot really configure a convcave(or is it convex?) service curve in HFSC. Since the starting point of the second curve is in the first service curve.

3. Do the m1, d, m2 parameters operate on a PER-SESSION environment?

m1 and d are per packet. m2 is global.

In my research, I found that the service curve is basically applied during "link congestion" only. Otherwise the scheduler is not doing much. the service curve value of m1 is not on a packet size, but total bandwidth used by the queue without regard for packet size. If m1 was packet size and m2 is global, wouldn't they be different variables instead of the same variable at different time spans?

Yeah every discipline is non-work conserving in ALTQ. Does it need not to be?!
Though if you want the discipline to behave as congested take a look at the tbrconfig/tbrsize parameter.
It might even help more in high speed links to lower it from what ALTQ/pf calculates automatically so the discipline acts propperly.
Actually m1 and m2 are different parameters since they define different service curves.
I can use it as packet size since i know the details as:
m1 * d converts to bytes approximately ;). Anyway long discussion but you can configure m1 < m2 with this shaper since i patched ALTQ/pf to allow that.

4. And how does m1, d, m2 work for a dynamic bandwidth WAN queue? When does m1 go into effect? With new sessions? hmm.. I'm hoping so! I think I am beginning to see the power of HSFC!

@ermal:

They scale accordingly if you have not set hard numbers in there.

So what settings would I use if I have a WAN that will burst all the way up to about 15mb download but it's guaranteed 8mb down and upload burst to 3mb and guarantee 1mb? I am thinking set bandwidth to 15mb/3mb and then use one of the service curves (not sure which one yet) to m1=15mb d=30000 m2=8mb?
Nailing this will help a lot of Comcast or other cable customers that have bursts that they are not able to take advantage of with the standard shaper wizard. In fact, if you could put this as an option in the wizard all the better!

Well i suggested it previously. Though you need the time of this bursting to pass to d parameter.

As for m1 = m2 try it if you find any difference or not!

@ermal:

Good luck you need it :).

Nah, I'll just use my brain. I learn quickly.

I'm looking forward to the updated today! Thanks so much for your hard work!

Good that's what i meant since the start :D.

Aaron

eri--

I have sent new links for the updated shaper to most of you.

The others will get a PM after an hour or so since there's a limit to how many PMs can be sent.

k3rmit

Thanks for the new update, however once installed and followed trough the revised (great thanks) multi lan wizard, i got stuck at "Generating ALTQ queues…" in the filter reload page.

It's not going forward and cannot get back to the shaper page, i have this error:

Fatal error: Call to a member function on a non-object in /usr/local/www/firewall_shaper.php on line 321

Thanks for any help

albe