My Shaper Config File For All To See! Works great with VoIP!
-
See http://forum.pfsense.org/index.php?topic=13.msg225#msg225 for a description of how I have set up my traffic shaping configuration.
My 4 rule approach came to be after looking at what ports FireFox was using. Port Explorer shows what programs listen on what ports and what connections are being used by a paticular program such as FF. I compared that to the connection states in pfSense and started seeing some correlations. I noticed that most if not all connections both incoming and outgoing from FF only had either the source or the destination port being used as 80. I also noticed that similiar rules for my software firewall (Look 'n Stop) on my WindozeXP machine were similiar. (Please no hate mail - there was a time I wanted to put heart and soul into Unix/BSD/Linux; XP does what I need right now and its a custom stripped-down version that's much, much leaner than when it left MS) So I decided to make 2 rules for outgoing and 2 rules for incoming and use the LAN interface shape all outgoing traffic and WAN interface shape all incoming traffic and have each rule only match one of 4 variables, etc…Outgoing Source Port, Outgoing Destination Port, Incoming Source Port, Incoming Destination Port. That way, any traffic that has anything to do with port 80 gets the priority I want it to.
Bittorent traffic does much the same thing because individual users disignate different ports for their clients to listen on. But, bittorent clients will also use other ports to switch a connection to if a port is already too "noisy" with traffic. This isn't a security issue because SPI FW's like pfSense will allow traffic back in on a particular port if it was initiated from the inside. So if I have changed my bittorent client to listen on port 25555 then it could possibly be connecting to another client on the WAN side on port 25555 or another port. This dynamic use of ports for both VoIP and P2P seems to make traffic-shaping so difficult. Thus, my approach of using a "bigger net" to catch more of certain type of traffic.
I just use the IP address of my phone adapter (I give it a static LAN IP) for the source on all outgoing VoIP traffic and the destination for incoming so it doesn't matter what port is being used.
Remember that shaping rules do not affect whether traffic can get in or out, they just determine the "speed" of the traffic.
-
when you make doubled up rules like below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 LAN out TCP * * qOthersDownH/qOthersUpH outbound on Port 80would this really do any justice … wouldn't they fight each other? then in sense wouldn't you just make a rule that states as shown below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 on Port 80Is the above single rule the same as making 2 different rules? I would essential believe it to be the same thing.
If thats not the case .. how will doubling the rules affect the overall speed of the machine the pfsense is running on, well I guess that would also pertain to the amount of users that were on the network using the internet. I myself do not know a whole lot about traffic shaping. So I am trying to figure this out. -
when you make doubled up rules like below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 LAN out TCP * * qOthersDownH/qOthersUpH outbound on Port 80would this really do any justice … wouldn't they fight each other? then in sense wouldn't you just make a rule that states as shown below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 on Port 80Is the above single rule the same as making 2 different rules? I would essential believe it to be the same thing.
If thats not the case .. how will doubling the rules affect the overall speed of the machine the pfsense is running on, well I guess that would also pertain to the amount of users that were on the network using the internet. I myself do not know a whole lot about traffic shaping. So I am trying to figure this out.Uhhh, combining that rule will match NO traffic. Source ports are 99.999% chosen randomly.
-
when you make doubled up rules like below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 LAN out TCP * * qOthersDownH/qOthersUpH outbound on Port 80would this really do any justice … wouldn't they fight each other? then in sense wouldn't you just make a rule that states as shown below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 on Port 80Is the above single rule the same as making 2 different rules? I would essential believe it to be the same thing.
If thats not the case .. how will doubling the rules affect the overall speed of the machine the pfsense is running on, well I guess that would also pertain to the amount of users that were on the network using the internet. I myself do not know a whole lot about traffic shaping. So I am trying to figure this out.Uhhh, combining that rule will match NO traffic. Source ports are 99.999% chosen randomly.
So then making rules like the set of 2 like the above is essentially pointless as far as the source port goes?
-
when you make doubled up rules like below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 LAN out TCP * * qOthersDownH/qOthersUpH outbound on Port 80would this really do any justice … wouldn't they fight each other? then in sense wouldn't you just make a rule that states as shown below:
IF in/out Proto Source Destination Target LAN out TCP * * qOthersDownH/qOthersUpH outbound on port 80 on Port 80Is the above single rule the same as making 2 different rules? I would essential believe it to be the same thing.
If thats not the case .. how will doubling the rules affect the overall speed of the machine the pfsense is running on, well I guess that would also pertain to the amount of users that were on the network using the internet. I myself do not know a whole lot about traffic shaping. So I am trying to figure this out.Uhhh, combining that rule will match NO traffic. Source ports are 99.999% chosen randomly.
So then making rules like the set of 2 like the above is essentially pointless as far as the source port goes?
No, its absoultely necessary.
-
I can understand why there needs to be two rules thats simple. But what happends when both the source and destination port are port 80? Wouldn't they then "fight each other" or conflict?
I assume there is another third rule in the shapper that deals with this and shapes http traffic? blah confused :)
-
I can understand why there needs to be two rules thats simple. But what happends when both the source and destination port are port 80? Wouldn't they then "fight each other" or conflict?
I assume there is another third rule in the shapper that deals with this and shapes http traffic? blah confused :)
Again, the source port is randomized 99.9999% of the time. The chances of this situation coming up are very very very slim.
-
I can understand why there needs to be two rules thats simple. But what happends when both the source and destination port are port 80? Wouldn't they then "fight each other" or conflict?
I assume there is another third rule in the shapper that deals with this and shapes http traffic? blah confused :)
Again, the source port is randomized 99.9999% of the time. The chances of this situation coming up are very very very slim.
Back to the 2 rules being made .. why wouldn't this have been done in the wizard … is there pros or cons to this?
-
Just to put some light why using source ports in general is not a very good idea:
Clients are initiating connections mostly from a random source port as there are usually a lot of services/applications running at the same time at the same IP. It just picks randomly free port. Think of running your own webserver at your machine. It will listen at port 80 but you still can Browse as your browser chooses another free port.
The source port IS NOT the destination port. So a real life connection to a webserver looks like this:
Client IP:tcp random port <-> Server IP:tcp 80
The server answers back to the random port. Usually random ports are picked from above 1024 as below 1024 are mostly so called "well known ports" ( http, ftp control session, smtp, …).
So a sourceport 80 rule would be nearly never ever matched unless you run a webserver or another service at your site originationg from port 80 (there are 65535 ports available to choose randomly from !!! ).
Why is it good to have sourceports then? It's good for blocking well known ports or services. Additional to that there are pure hardwaredevices that use always the same ports (as they don't have to deal with other "unknown" applications that are running at the same IP) like voip-phones for example.
If you want to have some examples look in the webgui at Diagnostic>states. The number behind the ":" is the port.
-
Just to put some light why using source ports in general is not a very good idea:
Clients are initiating connections mostly from a random source port as there are usually a lot of services/applications running at the same time at the same IP. It just picks randomly free port. Think of running your own webserver at your machine. It will listen at port 80 but you still can Browse as your browser chooses another free port.
The source port IS NOT the destination port. So a real life connection to a webserver looks like this:
Client IP:tcp random port <-> Server IP:tcp 80
The server answers back to the random port. Usually random ports are picked from above 1024 as below 1024 are mostly so called "well known ports" ( http, ftp control session, smtp, …).
So a sourceport 80 rule would be nearly never ever matched unless you run a webserver or another service at your site originationg from port 80 (there are 65535 ports available to choose randomly from !!! ).
Why is it good to have sourceports then? It's good for blocking well known ports or services. Additional to that there are pure hardwaredevices that use always the same ports (as they don't have to deal with other "unknown" applications that are running at the same IP) like voip-phones for example.
If you want to have some examples look in the webgui at Diagnostic>states. The number behind the ":" is the port.
So if I am understanding you correctly …
if i am running bittorrent on port 49152... everything going out is random ... but everything coming in is on port 49152?
-
Run bittorrent or something else and look at the states in the webgui. Most time it get's obvious by looking at real life examples.
-
psychosematic
In the example you cited above, any traffic that matches port 80 at the source OR destination headers will be given a certain priority. Only traffic (given that you haven't designated a different port than the default) that has anything to do with web browsing will have port 80 associated with it whether its the source or destination. The rules don't fight with each other because the first rule that a packet matches is used to prioritize it.
Billm can only incorporate this into the wizard in a very generic way that might not make much sense to someone new to traffic-shaping. Also, I use port 32172 for bittorrent and you use 49152. The only way to make it work is to adapt it to what you use.
The more I've learned about this by studying the the connection state table, I've even started to add additional rules that cover the default ports for bittorrent and emule. My bittorrent client makes about 40%-50% of its connections on port 32172 (source OR destination but not both for the same connection). About another 30% of its connections are in the range of 6870-6999 because clients on the other end are using that port, the default range for bittorrent. However, my client is not using 32172 for those connections, it's using something random. The rest of the connections just seem to be random at both the source and destination port (although I'm sure the majority are just the port other people have chosen).
The reason I have a total of four rules for each port or range of ports for a is because just figuring out traffic-shaping fries my brain and I don't have anything left over for figuring out if my browser is only using port 80 as the source for all outging traffic or such. So, this way, I make sure that all traffic that has to do with web browsing get treated the same, because it will have port 80 show up somewhere in the packet and it will receive the right priority whether it's coming or going. One other thing to keep in mind is bittorrent uses UDP also so I use 4 more rules for that. pfSense will have errors if the "any" setting for the protocol is used.
I also understand that maybe this approach might cause huge queues in DSL modems on the download side. My modem is set up as a bridge so I don't have this problem since it does no packet handling of any kind.
Even if some of my shaping rules are unnecesary (and I'm willing to admit that some may very well be), I have not noticed any speed drops. AFAIK performance only drops when the number of connections are more than the cpu can handle. But, Sullrich and hoba would know more about that.
-
psychosematic
In the example you cited above, any traffic that matches port 80 at the source OR destination headers will be given a certain priority. Only traffic (given that you haven't designated a different port than the default) that has anything to do with web browsing will have port 80 associated with it whether its the source or destination. The rules don't fight with each other because the first rule that a packet matches is used to prioritize it.
Billm can only incorporate this into the wizard in a very generic way that might not make much sense to someone new to traffic-shaping. Also, I use port 32172 for bittorrent and you use 49152. The only way to make it work is to adapt it to what you use.
The more I've learned about this by studying the the connection state table, I've even started to add additional rules that cover the default ports for bittorrent and emule. My bittorrent client makes about 40%-50% of its connections on port 32172 (source OR destination but not both for the same connection). About another 30% of its connections are in the range of 6870-6999 because clients on the other end are using that port, the default range for bittorrent. However, my client is not using 32172 for those connections, it's using something random. The rest of the connections just seem to be random at both the source and destination port (although I'm sure the majority are just the port other people have chosen).
The reason I have a total of four rules for each port or range of ports for a is because just figuring out traffic-shaping fries my brain and I don't have anything left over for figuring out if my browser is only using port 80 as the source for all outging traffic or such. So, this way, I make sure that all traffic that has to do with web browsing get treated the same, because it will have port 80 show up somewhere in the packet and it will receive the right priority whether it's coming or going. One other thing to keep in mind is bittorrent uses UDP also so I use 4 more rules for that. pfSense will have errors if the "any" setting for the protocol is used.
I also understand that maybe this approach might cause huge queues in DSL modems on the download side. My modem is set up as a bridge so I don't have this problem since it does no packet handling of any kind.
Even if some of my shaping rules are unnecesary (and I'm willing to admit that some may very well be), I have not noticed any speed drops. AFAIK performance only drops when the number of connections are more than the cpu can handle. But, Sullrich and hoba would know more about that.
Thank you for the extensive reply … do you know of any good sources for traffic shaping, so that I can read up more on it?
-
Sorry, I can't help you there except to try to google for it. Everything I know (which is little enough) is from getting my hands dirty.
-
when i use you config file once i reboot i am no longer able to browse websites.
but i can still access the pfsense box via browser.i get a notification scrolling along the top saying
"There were error(s) loading the rules:
/tmp/rules.debug:16: queue qWANRoot has no parent
/tmp/rules.debug:16: errors in queue definition
/tmp/rules.debug:17: queue qWANdef has no parent
/tmp/rules.debug:17: errors in queue definition
/tmp/rules.debug:18: queue qLANRoot has no parent
/tmp/rules.debug:18: errors in queue definition
/tmp/rules.debug:19: queue qLANdef has no parent
/tmp/rules.debug:19: errors in queue definition
/tmp/rules.debug:20: queue qLANacks has no parent
/tmp/rules.debug:20"they can also be found in system log
i have no idea what any of this means (first day using pfsense)
you config sense to work well until you reboot.second quick question
1000 Kb qWANRoot
1500 Kb qLANRoot
are those to the speed your your dsl line up/down?thanks
mentalinc -
when i use you config file once i reboot i am no longer able to browse websites.
but i can still access the pfsense box via browser.i get a notification scrolling along the top saying
"There were error(s) loading the rules:
/tmp/rules.debug:16: queue qWANRoot has no parent
/tmp/rules.debug:16: errors in queue definition
/tmp/rules.debug:17: queue qWANdef has no parent
/tmp/rules.debug:17: errors in queue definition
/tmp/rules.debug:18: queue qLANRoot has no parent
/tmp/rules.debug:18: errors in queue definition
/tmp/rules.debug:19: queue qLANdef has no parent
/tmp/rules.debug:19: errors in queue definition
/tmp/rules.debug:20: queue qLANacks has no parent
/tmp/rules.debug:20"they can also be found in system log
i have no idea what any of this means (first day using pfsense)
you config sense to work well until you reboot.second quick question
1000 Kb qWANRoot
1500 Kb qLANRoot
are those to the speed your your dsl line up/down?thanks
mentalincYou most likely need to rerun the traffic shaping wizard.
-
Thank you for your reply.
but if i run the wizard i will loss the current config (which i downloaded from this thread)
i am also experiencing from the system log
"pftpx[706]: #6 abnormal server error: 65" -
Thank you for your reply.
but if i run the wizard i will loss the current config (which i downloaded from this thread)
i am also experiencing from the system log
"pftpx[706]: #6 abnormal server error: 65"They are no longer compatible.
pftpx issues is cosmetic, will be fixed in beta3.
-
SO there has been a change to the shaper works from what is posted above and beta 2?
-
Yes of course.