Basic question on setting bandwidth



  • I was just wondering how all of you who are much more knowledgable than I am would set this up.

    I have a WISP that has a very unstable connection and the speed is all over the place. Sometimes it runs as fast as 8Mb/3Mb and sometimes as slow as 1.7Mb/600Kb. I have followed the traffic shaping setup described here -> https://forum.pfsense.org/index.php?topic=63531.msg364520#msg364520 and it works wonderfully when the connection speed stays above the set limits.

    What I'm wondering is what would be the best approach to set the limits with a connection that is all over the place at any given time so that I can provide bandwidth to users who need it? I'm running a small private discussion forum from my home and I want to ensure that users will be able to connect without issue. We've never had more than 14 people on line at one time so having a ton of bandwidth isn't really an issue but I need some available all the time.

    Changing ISP isn't an option due to location. They are the ONLY option I have.


  • LAYER 8 Netgate

    That sucks.  I think you're pretty much saddled with garbage internet.

    Get a VPS and host your forum in a datacenter somewhere.



  • You could use HFSC's link-sharing parameter, which uses ratios to determine which queue will be granted how much throughput, and when throughput is unused any link-sharing queue can use the maximum available.

    There is also HFSC's upper-limit parameter which you could use to limit the maximum bandwidth of non-forum-related traffic, but this is a hard-limit that does not change with the connection speeds.

    Edit: You could also additionally set the real-time parameter to guarantee a minimum speed, which may be wise. Use it only when necessary though. I think real-time may interact with link-share in ways that are not easily predicted because of real-time's "unfair" nature.



  • I would have to research a bit to understand how to do what you first suggested. I'm pretty new to pfsense so I've still got a lot to learn.

    As far as the second suggestion, I definitely don't want to starve any other connection of what little bandwidth is available with a hard limit. That's kind of why I'm struggling with my settings currently. I can set an artificially low limit and ensure there is available bandwidth at the expense of overall speed or hope that my ISP's connection doesn't get too slow that shaping isn't happening on my end.



  • The effectiveness of any traffic shaping you do with be based on the quality of your bandwidth. PFSense can't know how much bandwidth you have, but you will still benefit by rate limiting your upload.

    Technically you're supposed to rate limit yourself to a value slightly less than your minimum rate, but with huge differences between your best and worst, that won't be practical. You could try setting it to 80% of your normal rate. A directional antenna, like a parabolic dish may help with stability of bandwidth.



  • @Jailer:

    I would have to research a bit to understand how to do what you first suggested. I'm pretty new to pfsense so I've still got a lot to learn.

    As far as the second suggestion, I definitely don't want to starve any other connection of what little bandwidth is available with a hard limit. That's kind of why I'm struggling with my settings currently. I can set an artificially low limit and ensure there is available bandwidth at the expense of overall speed or hope that my ISP's connection doesn't get too slow that shaping isn't happening on my end.

    It would most likely have worse delay during high utilization because your ISP/modem will be doing the limiting/buffering instead of your intelligent traffic-shaping queue, but you may be able to counter-act that by enabling Codel or maybe RED/RIO (RIO = RED In and Out).

    You can either tune for throughput or latency. Both are hard to achieve.



  • @Harvy66:

    A directional antenna, like a parabolic dish may help with stability of bandwidth.

    Already have one, direct line of site, less than a mile, flat ground, zero obstructions.  :'(


  • LAYER 8 Netgate

    @Nullity:

    You could use HFSC's link-sharing parameter, which uses ratios to determine which queue will be granted how much throughput, and when throughput is unused any link-sharing queue can use the maximum available.

    That won't do any good because if the bandwidth is set at 8Mb and there's only 1.6Mb actually available, there will be no shaping done.  Linkshare 25% will mean 2Mbit, which is already over what's available.

    You could use priq so at least the traffic you mark as priority is sent first.

    What does the WISP say when you complain?



  • @Derelict:

    @Nullity:

    You could use HFSC's link-sharing parameter, which uses ratios to determine which queue will be granted how much throughput, and when throughput is unused any link-sharing queue can use the maximum available.

    That won't do any good because if the bandwidth is set at 8Mb and there's only 1.6Mb actually available, there will be no shaping done.  Linkshare 25% will mean 2Mbit, which is already over what's available.

    Link-share chooses which packet to send based on a "virtual time" scale derived from the ratio between the queues. You could set queue-A to 7Kb and queue-B to 1Kb or queue-A to 700Kb and queue-B to 100Kb. Since the ratio is the same, link-share would act exactly the same for both of the previous examples. Link-share has no limit, hence the need for the "upper-limit" parameter.

    The ratio, not the actual bitrate, is what HFSC bases it's link-share algorithm on.



  • @Derelict:

    What does the WISP say when you complain?

    Yeah they are on a first name basis with me. They always give the same old "we can see your modem so it must be your router or possibly malware on your computer".

    This was just taken a few minutes ago. This is what I 'm dealing with. Tonight is an especially bad case but it still reflects what I'm talking about.



  • And I just ran it again and get this:


  • LAYER 8 Netgate

    @Nullity:

    @Derelict:

    @Nullity:

    You could use HFSC's link-sharing parameter, which uses ratios to determine which queue will be granted how much throughput, and when throughput is unused any link-sharing queue can use the maximum available.

    That won't do any good because if the bandwidth is set at 8Mb and there's only 1.6Mb actually available, there will be no shaping done.  Linkshare 25% will mean 2Mbit, which is already over what's available.

    Link-share chooses which packet to send based on a "virtual time" scale derived from the ratio between the queues. You could set queue-A to 7Kb and queue-B to 1Kb or queue-A to 700Kb and queue-B to 100Kb. Since the ratio is the same, link-share would act exactly the same for both of the previous examples. Link-share has no limit, hence the need for the "upper-limit" parameter.

    The ratio, not the actual bitrate, is what HFSC bases it's link-share algorithm on.

    But until the total set bandwidth is reached, I don't believe any drops occur, which is why it is imperative for the top-end to be set a little lower than what's actually available.  There is no reason for the shaper to drop any packets if there is bandwidth available (absent upperlimit on the queue), regardless of link-share ratio.



  • You could ask your ISP how much bandwidth they see you using. Since their argument is that you must have malware using your bandwidth, then they should be able to see that traffic. Better yet, have them bring a laptop of their choosing over and do some speed tests. When my ISP installed my connection, they first checked it with their computer plugged directly in, then did a speedtest with one of ours.


  • LAYER 8 Netgate

    Curious what your WAN RRD graphs look like.  Status > RRD Graphs



  • @Derelict:

    But until the total set bandwidth is reached, I don't believe any drops occur, which is why it is imperative for the top-end to be set a little lower than what's actually available.  There is no reason for the shaper to drop any packets if there is bandwidth available (absent upperlimit on the queue), regardless of link-share ratio.

    Maybe it makes more sense to consider link-share a traffic-policer instead a traffic-shaper.
    I know the HFSC paper mentions a test where they explicitly stated they were imitating the throughput limits of a 10Mbit connection, which says to me that allowing the link to define the limits is perfectly fine.

    Or… someone could just try it. Too much theory in here. :P

    Edit: Interpret the quote for yourself:

    The sessions at level one all have bandwidth reservation of 1.5Mbps, and the sessions at level two have bandwidth reservations of 80Kbps, 480Kbps, 1.44Mbps and 2Mbps respectively. The total aggregate bandwidth reservation is 10Mbps – Ethernet's theoretical maximum throughput.

    That is from the "Link-Sharing" section of the paper. Seems if maxing out the link was a poor choice when using link-sharing, they would have either limited it below or stated that this was a bad practice.


  • LAYER 8 Netgate

    Yeah, someone could.



  • @Derelict:

    You could use priq so at least the traffic you mark as priority is sent first.

    Damn me and my HFSC tunnel vision… :( I kinda forgot that PRIQ could achieve his goal.
    Using PRIQ is probably the most convenient and predictable method. One drawback is that any of the higher priority queues can completely block lower priority queues, so "fairness" is impossible.



  • Well, I just tested my theory and it failed. HFSC needs to be configured as the throughput bottle-neck with the root queue's "Bandwidth" parameter.

    I will be testing more scenerios, like perhaps moving the actual link's queue from the root (root queue cannot change it's bandwidth dynamically, apparently) to an interior leaf queue (a leaf's bandwidth is dynamic). I will report back if find anything new. How the hierarchy of the queues is laid out is actually important, so I have some hope it may work. :)

    It may be interesting to note that HFSC was not originally implemented with an "upper-limit".
    https://github.com/freebsd/freebsd/blob/428b45aa532260e8c6ddf0217ec31db2234d29a8/sys/contrib/altq/altq/altq_hfsc.c#L39

    Anyway, here are my results:

    WAN Bandwidth set to 640Kb. (Less than real-world throughput of 666Kb)

    
    pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                        16:55:57
    
    QUEUE               BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S
    root_pppoe0       640K hfsc  0     0     0      0      0    0             0    0
     qBulk            1000 hfsc      214 76697      0      0    1             1  695
     qACK             500K hfsc     1697 94153      0      0    0             3  211
     qNTP            25000 hfsc        0     0      0      0    0             0    0
     qTEST1           **1000** hfsc     1285 1500K     19   7521   38             9  **11K**
     qTEST2           **6000** hfsc     3067 3792K      0      0   35            52  **67K**
    
    

    You can see that the ratios between the 2 test queues are almost exactly 1:6.
    These are the speeds recorded in my ftp client, "lftp", during the above test. The speeds are not pasted exactly as I observed, because I had to freeze the terminal with Ctrl-S.
    tmp/linux-3.18.5.tar.xz.1' at 5651256 (11%) **63.2K/s** eta:11m [Sending data]progit.en.pdf' at 3180520 (56%) 11.0K/s eta:2m [Sending data]

    WAN Bandwidth set to 250Mb.

    
    pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                        17:01:25
    
    QUEUE               BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S
    root_pppoe0       250M hfsc  0     0     0      0      0    0             0    0
     qBulk            1000 hfsc       14  2667      0      0    0             0    0
     qACK             500K hfsc       34  1874      0      0    0             0    0
     qNTP            25000 hfsc        0     0      0      0    0             0    0
     qTEST1           **1000** hfsc     1592 1921K      0      0    0            27  **34K**
     qTEST2           **6000** hfsc     1551 1940K      0      0    0            35  **46K**
    
    

    As you can see, this test did not function as I expected.

    Edit: "Bandwidth" and link-share's "m2" are set to the same value.


Log in to reply