Still having problems with shaping…



  • Hi all,
    There are allot of posts about traffic shaping, but I have not found any that can help me understand why my shaper dos not work.

    First off, running Beta2 and made a clean hard disk install first thing.
    Created all my necessary NAT rules with automatic adding of firewall rules (in most cases the ports forwarded in the NAT rule shall also be shaped) and then run the traffic shaper wizard.
    After this, I added some new rules in the traffic shaper for the shaping to be done at the correct ports (or so I thought anyway).

    In short:
    I have Quake 3 game traffic coming in and out at the "default" ports that shall have high priority.

    I have traffic coming in and out from a port range (let's say 10000 to 10050) that shall have lower priority then game traffic but higher priority then P2P traffic.

    I have P2P traffic coming in and out over the normal ports and torrent traffic coming in and out on (let's say on port 50000).

    I have run testing software trying out my connection and I have figured out my maximum in and output that way. I have run multiple tests to and from multiple server at different occasions and thereby determine my line up and down capacity.

    I use the traffic shaper wizard to create the "normal" queues and pipes, entering the "average value" of my tests in the capacity fields.
    Not using VOIP
    Using games, tagging Q3
    Using some streaming, tagging MP3 Streaming.
    Using P2P, tagging DCPlusPlus, DirectConnect and Torrent.
    Using prioritising of VNC, WEB, SMTP and POP3.
    Done.
    Now edit the Torrent rules to change the ports from the standard to my 50000 (I only use one port, utorrent dos not need more then one (and that may be the source of the problem, but I don’t know).
    Adding 2 new rules by copying ex. the VNC rules but using my port range for my out and incoming traffic (10000-10050) that I want prioritized over P2P but under the games.

    Done, save and apply.

    Now, I tell my friend to start downloading on my prioritized port rage and then when he is downloading I change the "upload limit" to a higher value in my torrent software my friends download instantly reduces to nearly nothing.

    What am I doing wrong here? Is there something I have forgotten?
    I can send my configuration file if someone want to take a look at it.

    I hope it's some what clear what I'm trying to do, otherwise let me know.



  • There was some queue fixes recently.  Now all traffic goes to correct queues.

    Upgrade to http://www.pfsense.com/~sullrich/RELENG_1_SNAPSHOT_03-10-2006/



  • Hi again,
    I have now upgraded, rerun the wizard and added the necessary rules for my needs. This time it works better, my friends download (upload out from me) don't completely die, but still it slowly degrades to a low rate, it seams that it's almost working at first but then the P2P traffic again takes the overhand even now but it takes a longer time.

    I also noticed another strange thing, it may not have anything to do with this but it feels like it does. I am using a "active desktop" with 10 different news RSS feeds updating every 10th minute, when traffic shaping is enabled the updates fail more often then when it's turned off. It either times out (page can not be displayed) or I get a parse error. I have been using pfSense for almost a year and I remember that I had a similar problem before when using the traffic shaper. I stopped using the shaper due to other issues with it at the time and have not encounter the RSS problem since then, I awaited the Beta2 (and now the upgrade as well) but now the strange error it's back again when the shaper is enabled. I can mention that I use my own PHP script on my server on the LAN to parse the external RSS feeds. I have reflection ON, but have also tried with it off with the same result. I use my LAN web server IP address for the PHP script and a FQDN for the actual RSS feed.

    If you want my config, just let me know.



  • Hi again,
    I finally figured out why my shaping dos not work as I thought it would do.
    The fact is that the Bit Torrent application I'm using (and probably allot of other Bit Torrent applications as well) selects a random port for outgoing connections. This means that the "handshake" is made on the specified port but the actual data transfer is made on any port randomly selected. That is why my "outgoing" traffic gets jammed up when letting my Bit Torrent client use "all" bandwidth (with the intent to let the traffic shaper in pfSense handle the limitation. It actually don't shape the outgoing traffic because it goes on "random ports". :(

    So, the next question then becomes…
    Is there by any chance a possibility that you will add the application layer to the traffic shaper? 8) That would basically solve all the problems. But, of course create a tremendous amount of work for you guys... :/



  • Yeah there is layer 7 traffic shaping work going on for pfSense so you can look forward to that in the future.
    In the meantime what I can suggest as a work around is to prioritize the traffic for well know ports (VNC, WEB, SMTP, POP3, etc..)
    then create a low priority queue where everything else gets pushed into regardless of port number. Everything that is
    not explicitly tagged by the shaper goes to the default queue, so giving that (qWANdef) a lower priority should make
    other traffic smoother.



  • @Snille:

    Hi again,
    I finally figured out why my shaping dos not work as I thought it would do.
    The fact is that the Bit Torrent application I'm using (and probably allot of other Bit Torrent applications as well) selects a random port for outgoing connections. This means that the "handshake" is made on the specified port but the actual data transfer is made on any port randomly selected. That is why my "outgoing" traffic gets jammed up when letting my Bit Torrent client use "all" bandwidth (with the intent to let the traffic shaper in pfSense handle the limitation. It actually don't shape the outgoing traffic because it goes on "random ports". :(

    So, the next question then becomes…
    Is there by any chance a possibility that you will add the application layer to the traffic shaper? 8) That would basically solve all the problems. But, of course create a tremendous amount of work for you guys... :/

    At some point…maybe.  Here's an interesting hack I did to catch all P2P on my network.

    Run the wizard selecting options as usual (ensure that you select "common" items like HTTP even if you want them "default" - you'll see why in a minute)
    Modify both BitTorrent rules to any port
    Move both BitTorrent rules to the LAST rule in your ruleset

    The shaper is first match..by putting an "any" rule at the bottom you have just added a catch-all rule.  This will nail ALL your P2P along with ALL other traffic that doesn't have explicit rules for it.  Works like a champ (until some putz runs hit BT on port 80 - which does happen from time to time).

    --Bill



  • As of the 04-02-2006 snapshot (todays) this option is now included on the p2p screen called p2pCatchAll.



  • Ah, nice! :)
    Is this also included in the 04-03-2006 snapshot or should I wait with that one?



  • Yep, it's already included in this snapshot.



  • @Snille:

    Ah, nice! :)
    Is this also included in the 04-03-2006 snapshot or should I wait with that one?

    As of the 04-02-2006 snapshot (todays) this option is now included on the p2p screen called p2pCatchAll.  (ps: could this post have been any more clear!?!?!?!)



  • Well, sorry, I am not native English….
    For me it could also meant that it was included in (todays only) snapshot for test purposes. Because of the trouble we (I) have had. Sorry, I will try to shape up! :)



  • @Snille:

    Well, sorry, I am not native English….
    For me it could also meant that it was included in (todays only) snapshot for test purposes. Because of the trouble we (I) have had. Sorry, I will try to shape up! :)

    Heh…I love how dates get swapped :)  Is 04-03-2006 April 3rd or March 4th? :)  It's obvious to both Brits and Yanks...it's obvioulsy April 3rd to me duck..

    --Bill



  • Heh, I see the disconnect now.

    Over here we go by MM-DD-YYYY.



  • one question about the catch-all idea…

    doesnt this screw with http traffic?

    don't most browsers use a random high port to initiate data transfer of page contents...?
    e.g.
    myPC:52345 -> google.com:80  (matches QoS rule for http)
    then remote server responds to the request on the port initiated with the actual page contents.
    google.com:?? ->  myPC:52345  (matches catch-all rule and gets treated as P2P?)

    so the QoS works only one way...?

    probably other things besides http work in a similar way... i am no expert though so i'm hoping you will tell me i'm wrong and traffic will be recognized fine and shaped. of course, my choke point is the outgoing b/w not the incoming so i still added the catch-all rule.  but if i am right then perhaps i should only add the outgoing catch-all? err.. hmm.. now i think i confused myself.



  • That connection belongs to the same state and will be treated the same way  like the outgoing request.


Log in to reply