Bandwidth saturated by lowest priority queue?



  • I'm trying to configure the traffic shaper to, among other things, give HTTP traffic priority over P2P traffic.  My queues look like this:

    Flags Priority Default Bandwidth Name
    0 No 400 Kb qWAN_Root
    0 No 2500 Kb qLAN_Root
    ACK 7 No 30% qWAN_ACK
    ACK 7 No 30% qLAN_ACK
    6 No 20% qWAN_DNS
    6 No 20% qLAN_DNS
    5 No 15% qWAN_Games
    5 No 15% qLAN_Games
    3 No 30% qWAN_HTTP
    3 No 30% qLAN_HTTP
    2 Yes 1% qWAN_Default
    2 Yes 1% qLAN_Default
    RED, ECN 1 No 1% qWAN_P2P
    RED, ECN 1 No 1% qLAN_P2P

    I'm testing this on a Windows XP workstation using Azureus and Firefox while looking at the Queues status page in the webConfigurator.  My procedure and expected results look like this:

    • Start Azureus with several torrents downloading and its bandwidth limiter disabled.

    • Confirm that the qLAN_P2P queue saturates the connection.

    • Change Azureus' bandwidth limiter to an extremely low value ( < 10 KB/s).

    • Confirm that traffic on the qLAN_P2P queue drops to nearly nothing.

    • In Firefox, browse to YouTube and begin loading several videos.

    • Confirm that the qLAN_HTTP queue saturates the connection.

    • Disable Azureus' bandwidth limiter.

    • Confirm that the majority of the bandwidth still goes to the qLAN_HTTP queue.

    (Note that I'm only adjusting Azureus' download bandwidth.  The entire time I have Azureus throttling its upload speed to 50% of my upload bandwidth.)  That last item is where everything goes wrong.  With Azureus's bandwidth limiter disabled and bandwidth allocation now entirely pfSense's responsibility, not only does the qLAN_HTTP queue not get more bandwidth than the qLAN_P2P queue, it appears that the qLAN_HTTP queue isn't getting any bandwidth at all!  The video downloads from YouTube are basically being starved of bandwidth while the torrents downloads are allowed to run at full blast.  I've tried tweaking this configuration every way I can think of and still get the same result: the lower priority queue gets all the bandwidth while the higher priority queue gets none.

    I thought maybe I had the priorities backwards (that a higher number means lower priority), but I tried changing them around and it didn't make a difference.  It's not an issue of my rules not matching the traffic correctly and it being dumped into the default queue, either; I can see that the only queues that are getting any significant traffic during these tests are the inbound HTTP and P2P queues and the outbound ACK and P2P queues, and the bandwidth readings for each queue seem to the correctly reflect the amount of traffic I'm seeing each application getting.  Also, the qLAN_P2P is dropping a lot of packets (~60 every time the status page refreshes, even with no HTTP downloads going, oddly), but it's not enough to give the HTTP queue any bandwidth.

    I just can't figure out where my configuration is wrong.  I have the 3 megabit DSL plan from SBC/AT&T and I'm running it through the CDROM version of pfSense 1.2 Final which is handling PPPoE itself.  I created my configuration by first running the shaper wizard and then adjusting the settings from there.  I have disabled the Service Curve options for both of the queues in question but with no effect.  Curiously, if I reset the states table and then disable traffic shaping altogether, I see the exact same behavior (if I start a video download while torrents are downloading at full speed, the video barely loads at all while the bandwidth usage as reported by Azureus barely drops).  Any ideas?



  • @BACON:

    I'm trying to configure the traffic shaper to, among other things, give HTTP traffic priority over P2P traffic.  My queues look like this:

    Flags Priority Default Bandwidth Name
    3 No 30% qLAN_HTTP
    2 Yes 1% qLAN_Default

    I have disabled the Service Curve options for both of the queues in question but with no effect.  Curiously, if I reset the states table and then disable traffic shaping altogether, I see the exact same behavior

    Meaning that traffic shaper doesn't take effect for qlan_p2p? Try to check that it takes at least some effect. Try massive downloads at the same time through qlan_http and qlan_default (nntp for example), record about 10 queue load samples, check that qlan_http load is about 30 times larger than qlan_default.

    p/s. For a HFSC traffic shaper, Priorities make (almost) no sense, only Bandwidths matter.



  • I think the problem is with rules and that the videos are not downloaded on port http and that's why you get anything on default queue.



  • @dusan:

    @BACON:

    I'm trying to configure the traffic shaper to, among other things, give HTTP traffic priority over P2P traffic.  My queues look like this:

    Flags Priority Default Bandwidth Name
    3 No 30% qLAN_HTTP
    2 Yes 1% qLAN_Default

    I have disabled the Service Curve options for both of the queues in question but with no effect.  Curiously, if I reset the states table and then disable traffic shaping altogether, I see the exact same behavior

    Meaning that traffic shaper doesn't take effect for qlan_p2p? Try to check that it takes at least some effect. Try massive downloads at the same time through qlan_http and qlan_default (nntp for example), record about 10 queue load samples, check that qlan_http load is about 30 times larger than qlan_default.

    p/s. For a HFSC traffic shaper, Priorities make (almost) no sense, only Bandwidths matter.

    Ok, I just tried simultaneous HTTP and FTP downloads of some Linux .iso files.  First, I tried each seperately to confirm the traffic was going to the correct queues, and it did; the HTTP download went to the qLAN_HTTP and the FTP download went to qLAN_Default.  Then I tried starting the HTTP download, letting that run for 10-15 seconds, and then starting the FTP download.  For the first update of the queue status page the bandwidth was split pretty evenly, but from then on the qLAN_HTTP queue fluctuated between 2.0 and 2.4 mbps and the qLAN_Default got whatever was left.  I then tested it again but in the reverse order: first starting the FTP download and letting that run for a few minutes, then starting the HTTP download.  The results were pretty much the same, with the HTTP download being given the majority of the bandwidth.  So, that's the result I would expect.  The question is, why doesn't my qLAN_P2P queue behave in similar fashion as my qLAN_Default queue?

    The other thing I've noticed in the last few days that's a little weird is that there were a couple instances where, during a large HTTP download, Azureus indicated that it was starved of bandwidth.  The difference there is that, at the time, my torrents weren't downloading very fast, compared to before where they were going at full blast.  I'm wondering if perhaps my ACK queues are the culprit here.  Is it possible that the P2P ACKs are somehow displacing the HTTP ACKs, so the HTTP connection effectively gets cut off?

    @ermal:

    I think the problem is with rules and that the videos are not downloaded on port http and that's why you get anything on default queue.

    That's what I thought at first, too, but a netstat showed that the YouTube server from which videos are downloaded does, in fact, use the standard port 80.  Like I said, all of the traffic appears to be going to the correct queues, it's just the bandwidth isn't being allocated to the queues as I would expect.  Plus, even if the YouTube videos were going to the default queue, given my configuration I'd still expect them to be given at least as much if not more bandwidth than the torrents, as opposed to the none at all that I'm seeing.



  • I don't know what may cause the problem. But I suggest you not to use the Queue Status page.
    Instead, use pftop (a command-line tool available in pfsense standard installation), as it displays queue load in real time. Double check that all traffics go through the correct queues. HTTP download, for example, should use four queues (possibly qwan_http + qlan_ack to send requests and qlan_http + qwan_ack to receive data).



  • What commandline switches would you use for pftop to view the queues?

    I'm having a similiar problem with P2P traffic, but I won't post the details yet since I'm new to pfsense, and it could be user error :)



  • I don't need any switch. I use keys to switch between views.



  • @JustinHoMi:

    What commandline switches would you use for pftop to view the queues?

    I'm having a similiar problem with P2P traffic, but I won't post the details yet since I'm new to pfsense, and it could be user error :)

    Log to console, start PFtop ("9") and the press "8" key. This will show Queue live view.



  • Hi,

    I have faced your problem too, so i digged myself into the issue a bit.
    The core problem:
    "The question is, why doesn't my qLAN_P2P queue behave in similar fashion as my qLAN_Default queue?"

    I think the reason is: Because a P2P traffic usually consist of many connections, while normal HTTP or FTP only few. This makes huge difference on traffic shaping.
    Freebsd uses queues, meanwhile it can only work if there is always empty space in each queue. So, when you try to prioritize your P2P traffic by assigning the lowest priority to it, what will happen? PF will drop packets from P2P queue first, as it has the lowest prio. And what will happen after this? One of your many-many P2P connections will be slowed down a bit, but nobody knows which one, and you still have a lot of others, which all tries to use your connection. That is why you feel: traffic shaper does not limit the bandwidth. It tries, but it can not.
    In other words: freebsd can not shape traffic with overloaded queues if there are a lot of connections belong to it, because it simply can not know which connection uses the most bandwidth within the a queue. It simply drops packages by random from the respective queue - I guess.
    You can check this by going out to shell, and type: pfctl -vsq
    You see, the default queue size of freebsd is 50. This is ok for low number of connections, but for P2P it is nothing.

    Solution:
    You have to increase the size of your P2P queues manually in order to avoid the queue saturation.
    See my post at:
    http://forum.pfsense.org/index.php/topic,9427.0.html
    It works for me perfectly since weeks. Now i can prioritize P2P traffic without any problem. I have queue size for P2P of 2000! (of course the latency is higher with such big queue, but for P2P this is not a problem at all)
    Unfortunatelly the current webgui does not support this setting, but with 'vi' you can modify the setting in '/tmp/rules.debug' in seconds.
    Then you reload the modified ruleset by 'pfctl -A -f /tmp/rules.debug'.
    I think it would have reason to include queue setting into the Webgui.
    Hope this helps.
    Viktor


Log in to reply