Traffic shaper changes [90% completed, please send money to complete bounty]

[eri--]

Yeah it can do multiple level of queues and all of what you describe.

[SlickNetAaron]

Great! Thank you!  I just sent $75 to Chris.

@ermal:

Yeah it can do multiple level of queues and all of what you describe.

[SlickNetAaron]

So I guess I need to know how to access and install this.  I will get a PM?  This is an embedded install on ALIX.2C3

Regards,
Aaron

@SlickNetAaron:

Great! Thank you!  I just sent $75 to Chris.

[eri--]

@SlickNetAaron:

So I guess I need to know how to access and install this.  I will get a PM?  This is an embedded install on ALIX.2C3

Regards,
Aaron

@SlickNetAaron:

Great! Thank you!  I just sent $75 to Chris.

Yes, pretty soon.

[NateDavis]

If this is the place for tech support questions with the new shapper than great. Otherwise, please direct me where these should go.

I have been playing aorund with the new shapper and either I am really dense, and can't figure it out or I don't understand QoS Properly…  Who Knows...

Anyway, I am trying to prioritize VoIP traffic. This traffic runs over my OpenVPN connection setup in the pfSense. I am having a real problem getting the traffic to register in the voip queue (using the wizard and then modifying the floating tab in rules). Is there anything special I am suppoed to do? I thought about trying to prioritize the openvpn traffic, but couldn't get that to work either. Everything just goes to the default queue.

This is an Avaya ip office setup. I have traffic being tagged with difserv-  DSCP 46, DSCP Mask 63, and SIG DSCP as 0. I tried setting the diffserv in the floating rule to 46, but it still didn't put that traffic in the queue. Any help would be appreciated.

Thanks!
Nate

[hoba]

Shaping inside openvpn tunnels is not yet supported afaik, inside IPSEC should work though.

[eri--]

It is the default LAN rule that is botching it.
Just make it specific or create the rules for the in the LAN tab over the default one supplied by pfSense.
And please try disabling the antilockout rule.

With the new update things should be better(a matter of days since some issues have been fixed).

[SlickNetAaron]

Hi, I don't mean to be impatient.  Just wondering when I may get access to the new shaper.  I can wait for the new update if it is just a couple days.

Regards,
Aaron

@ermal:

With the new update things should be better(a matter of days since some issues have been fixed).

[sullrich]

Should be soon.

[eri--]

For all the bounty contributors.
In the same link as before will find the updated images with several problem fixed.

[eri--]

Get the one with the highest date on it. as -20080324 ;)

[songus]

Ill add 50 to the bounty, should i send them now? when will the image be available¿?

[hoba]

@songus:

Ill add 50 to the bounty, should i send them now? when will the image be available¿?

All bountysupporters get exclusive access to the testingimages and are welcome to testdrive and report back. All others will have to wait for now until there are official builds including the changes. Feel free to send the money in right now.

[eri--]

I explained it a page before:
http://forum.pfsense.org/index.php/topic,2718.180.html

the queue wizard is really a work in progress. the first part is difficult to understand and has text labels in code style. the second part, the one with traffic type prioritization, is an heritage of the old shaper wizard but has no reason to exist, 'cause is not applied anywhere and there's no interface to edit. It seems that now the assignment of traffic type to queues is done within each firewall rule.

Well you do not need any interface to choose since it applies to all interfaces.
Read my explanation of the Floating Tab.

As for the names i will make them more friendly.

BTW, since you are a user what part of the first part you didn't understand?

[k3rmit]

sorry, i just found your 1st explanation, that's why i deleted my post…

i'll try apply the rules as by your tutorial and in case get back to you with a good feedback.

to answer your question, if for example i click on the "single wan multi lan" wizard, i'm asked for the number of connections: in my understanding this should be the LAN and the DMZ, but in the next step i have WAN and OPT1 (DMZ) grouped in the "setup connections speed" section, like if we were talking about two WANs, while the DMZ has to be considered like a LAN section.
i'm puzzled here because given i'm configuring multiple lans, as by wizard name, i should be asked just for the wan bandwidth and then describing the lan part. this could be a limit of my understanding of the shaping mechanism within pf, but i have to admit that the wizard isn't a lot descriptive about what am i doing with the info i'm entering and the options i'm choosing.

i just want to avoid traffic shaping between the LAN and DMZ and meanwhile shape all traffic from all interfaces to WAN: from your tutorial i understand that i just need to assign floating rules to queues. i have a solid heritage of rules assigned to each interface, so i think it will take time to make it work correctly. is there any monitoring/debugging application for pf out there?

btw, thanks for the prompt answer.

[eri--]

Oh for the Multi Lan wizard i might have missed some labels changes.
Though it really asks you for the number of LAN's. As i can not guess what interfaces are considered LAN in your cases.
You see WAN in there since i need to know on which interface is the internet connection connected.

If you do not want to shape traffic between DMZ and LAN, on the traffic shaper config:
1- Click the lan root node on the tree. Set its interface bandwidth to the same as you Network card speed(i.e. 100Mb)
2- Delete the traffic shaper config on both LAN and DMZ
3- Create a queue called qInternet in both the LAN and DMZ interface and setup it with the download speed of your internet connection.
If you have choosen HFSC scheduler make its linkshare m1=m2=link download speed and d =something.
4- Create a DMZ queue on both the LAN and DMZ interface. Setup its bandwidth = Lan root speed - speed of qInternet queue
5- Under the qInternet queue replicate the queues that gets created by the wizard, so that the internet shaping for LAN and DMZ works ok.

Than create a rule that matches local traffic(traffic between LAN and DMZ) and sends it to the qDMZ queue so it does not have limitations from the shaper.

I am testing this setup and will make the changes for the Multi Lan wizard, at least, to produce the above automatically.

You will get it with the next update which fixes the other reported issues.

Just a stupid text illustration of the above is:
WAN
–-qACK
---qDefault
---qP2P
---qVoIP
---qOthersHigh
LAN
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
DMZ
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ

On the floating rules tab make a rule:
1- pass
2- select LAN and DMZ interface
3- Direction any
4- from any  (though you might consider only the ports to the DMZ services)
5- to any (though you might consider only the ports to the DMZ services)
6- queue qDMZ

And done.

Another more advanced scheme might be:
WAN
---qACK
---qDefault
---qP2P
---qVoIP
---qOthersHigh
LAN
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
----------qDMZACK
----------qDMZDefault
----------qDMZP2P
----------qDMZVoIP
----------qDMZOthersHigh
DMZ
---qInternet
----------qACK
----------qDefault
----------qP2P
----------qVoIP
----------qOthersHigh
---qDMZ
----------qDMZACK
----------qDMZDefault
----------qDMZP2P
----------qDMZVoIP
----------qDMZOthersHigh

And propper rules in place.

[sullrich]

LANs are easy to determine.  Walk the configuration and look for interfaces without a gateway attached to them.

[SlickNetAaron]

Hi Ermal,

Thanks for allowing access to the new shaper.  I see you are continuing to work on it.

I'm having a very hard time trying to figure out how to set this up.  I am unable to add queues to interfaces (I got it to succeed only once!) I'm totally not understanding how this shaper is laid out - it just does not seem intuitive.

My setup was explained here: http://forum.pfsense.org/index.php/topic,2718.195.html
If you can help me understand how to set this up, I would be grateful.  I would even be willing to write up a HowTo to try to explain the new shaper as well as help form the GUI with you.

Regards,
Aaron

[eri--]

Can you please post full details of your configuration.
Bandwidths you want to use etc so i can give you a config.

The upgrade you have has 3 issues:
1- you cannot add queues other than on the Lan.
EDIT: You cannot add queues that are childs of parent interface other than LAN. But you can add childs of other queues on any interface.
2- The Status->queues is shifted to the right as for a missing line for displaying the header ok.
3- The rrd graphs has a typo which does not allow to propperly view the queues graph
4- Floating rules are generated after per tab interface rules so if you have some rules in the specific interface tabs(wan/lan tab) they will spoil the floating rules.
This are just regressions of backporting from RELENG_1. In the next update they will be ok.

In your case you should not have any problems since you want to add queues only for LAN so you should be OK.

Now from what i see you want something like this.
Create an alias with the host you want to limit.

On the wizard check the Penalty box and add this alias on this step.
Also check the catchall option of it.
You should have a scheme like this after it.

WAN
–-qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow

This should set you on for anything you want.
You limit the customers through the Alias config and no need to tweak the rules.
Also if you want a hard limit for them set the uppelimit of qOthersLow(value m2) to the required limit.

Since of issue 4 you do not need any settings on Wan apart specific things you want to block.
Disable anti lockout rule.
And replicate the LAN default pass in rule to the Floating tab and disable that one(for this upgrade you are running.

That's all you need to share all the bandwidth evenly in your setup. Since you say the AP's are limited to 6Mb that's as simple as it can get with the upper scheme.
You can optimize VoIP rules by converting the rules for VoIP to use DSCP(diffserv code point) instead of port based ones; if you know that they use a specific DSCP mark.

Tell me if this suits you.

The other scheme if you wanted to have the hard limit to 6Mb setuped on the pfSense is:
WAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
---qAP1 (m1=m2=6Mb d=line delay)
------qAP1ACK
------qAP1P2P
------qAP1VoIP
------qAP1OthersHigh
------qAP1OthersDefault
------qAP1OthersLow
---qAP2 (m1=m2=6Mb d=line delay)
------qAP2ACK
------qAP2P2P
------qAP2VoIP
------qAP2OthersHigh
------qAP2OthersDefault
------qAP2OthersLow

or
WAN
---qACK
---qP2P
---qVoIP
---qOthersHigh
---qOthersDefault
---qOthersLow
LAN
------qACK
------qP2P
---qVoIP
------qOthersHigh
------qAP1OthersHigh
------qAP2OthersHigh
---qOthersDefault
------qAP1OthersDefault
------qAP2OthersDefault
---qOthersLow
------qAP1OthersLow
------qAP2OthersLow
On this one set the limits for each AP to the specific queue using upperlimit m2 value. Though i doubt you want their Voip queues to be separate since you want both clients to have seemles VoIP.
The last scheme might give you better results but it is hard to understand for someone not knowing what he is doing.

BTW, if you could gather all my postings about the shaper to something readble and skinned :) i would greatly appriciate. I have not yet found the time to do that.

[mikenl]

I haven't pledged to the original bounty, but i made a contribution of $50,00 USD.
I appreciate the work done on the traffic shaper, and would love to take a look at it.