Question regarding bufferbloat mitigation and lan-to-lan shaping



  • Preface:  I did search all around the forums and on google and couldn't find a how-to, FAQ or example of what i'm trying to achieve here..  So, if this exact question has been asked before, forgive me.

    My goal is simple:  I'm trying to mitigate bufferbloat on my comcast extreme 150/20 connection.

    I'm running PFSense 2.2.4 with the following setup:  WAN,  LAN_WIFI (int/vlan for wireless clients) and LAN_WIRED (int/vlan for wired clients).

    All I'm trying to do is use traffic shaping to throttle back upstream and downstream WAN traffic about 10 percent.  All while NOT throttling LAN to LAN traffic.  If i go to the traffic shaper settings in PFsense and enable basic, per interface shaping (using CODELQ) I can easily throttle WAN (egress) traffic (lets say to 20Mbps since comcast over provisions).

    If I do the same type of shaping on either of my LAN interfaces, throttling it to 150Mbps,  it not only throttles inbound/ingress traffic from the WAN but it also throttles LAN to LAN traffic to that same 150Mbps (I'm using iperf on my LAN to verify this – my LAN is all 1Gbps, when i have the traffic shaper off, i get ~930Mbps iperf on the LAN).

    So, my question is, considering i'm using the latest version of PFsense, what is the recommended/best practice method for mitigating bufferbloat (throttling WAN approx 10%) while not throttling LAN-to-LAN traffic?

    I just need some pointers, tips, ideas, anything.  Surely someone has accomplished this exact task because nearly every home user has a gigabit LAN these days but a much smaller pipeline to the internet.  It should be simple to throttle ingress/egress WAN traffic without negatively affecting in-house LAN speeds.

    thanks!

    -ryan



  • I have been learning how to do traffic shaping the last week so I am inexperienced.  With the caveat that there may be a more elegant way to do this, I think you can accomplish what you'd like to do by creating a couple of floating rules on the firewall.

    Edit:  I tested exactly what I initially proposed and it doesn't appear to work.  My setup is actually a little different in that I have QLink and QInternet on each LAN interface.  I throttle QInternet to 33 Mbs and QLink to 960 Mbs.  I have a default Queue under QInternet called QBulk.  Internal Lan/Lan traffic is diverted to QLink using the method below.

    First, none of your existing floating match rules should be designated quick.

    1. Create a new floating rule one with:
      Action–>Match
      Interface: Select both Lan_Wired & Lan_Wireless
      Direction: Out
      Protocol: Any
      Source: Lan_Wired net
      Destination: Any

    Set Ackqueue / Queue both to none/QLink.

    1. Create a similar second floating rule with the only difference being Source: Lan_Wireless net  (you could probably get by with one rule if you create an alias that covers both of your local lan subnets).

    2. Ensure both of these rules are at the bottom of list on the floating rules as the firewall will apply the last floating rule that matches if rule processing isn't immediately stopped by Quick designation on an earlier rule.

    Best of luck,
    Tim



  • The first thing you want to do is shape your WAN and use CoDel as the queue discipline.

    The next part is shaping your download. This is difficult with multi-LAN because you can't shape the data coming in, only going out. Since you have one coming in and two going out, you're only real way to properly shape is to split the bandwidth 50/50 between both LANs. I do make the assumption that you're routing directly from WAN to LAN. If you have an interface between your WAN and LAN, you'll then have a common choke-point where you can split the bandwidth 50/50, but allow one LAN segment to share bandwidth with the other LAN segment.

    If there is a way to create a virtual interface in PFSense that can be shaped, then you can properly share and split bandwidth among more than one LAN. Without that, all you can do is split bandwidth, but not share it.



  • Harvy, I got you on the WAN (egress/outbound) shaping.

    for the sake of argument, let's assume I had a single LAN interface (I set up a separate wired and wireless vlan/interface more as a learning experience back in the day – i dont have to keep it this way, necessarily)

    so, if i had a single LAN,  what would the best practice be for shaping inbound traffic from outside, yet allow LAN to LAN clients full gigabit speeds?

    -ryan



  • Since this is bandwidth shaping and not priority, HFSC. Creating a qInternet and shove your Internet traffic in it, and a qLink that is a sibling in the root along with qInternet. qInternet can have an upper limit set, but qLink needs no such limit. Assign all traffic where the destination is to a local LAN into qLink, and all other traffic goes into qInternet.

    Of course you can make the network more complicated than two queues, but it depends on how complicated you want it.



  • @Harvy66:

    Since this is bandwidth shaping and not priority, HFSC. Creating a qInternet and shove your Internet traffic in it, and a qLink that is a sibling in the root along with qInternet. qInternet can have an upper limit set, but qLink needs no such limit. Assign all traffic where the destination is to a local LAN into qLink, and all other traffic goes into qInternet.

    Of course you can make the network more complicated than two queues, but it depends on how complicated you want it.

    so, just so that I got this all right,  figuring a single LAN and WAN interface setup,  to mitigate bufferbloat:

    WAN (egress/outbound) – simple CODELQ shaper,  set the bandwidth to the appropriate limit.

    LAN (ingress/inbound from outside) -- HFSC would be recommended method, using your above example as one way to shape inbound from outside but not LAN to LAN.  you wouldnt use CODELQ because you're working with multiple queues:  one (qInternet -- upper limit set) that is in a rule that forces inbound internet traffic into it, and another queue (qLink -- no upper limit) that has a rule that forces local LAN traffic/subnet through it.

    so, sounds like 2 rules for the inbound:

    internet to LAN -->  qInternet
    LAN to LAN -->  qLink

    (LAN to internet would be covered by the other CODELQ traffic shaper on WAN interface)

    hope i'm not over complicating this.  I wanted to make sure it's documented out in this thread because I'm having a hell of a time trying to find a straight-up how-to on this subject and this thread could prove helpful to other people.

    -ryan


  • LAYER 8 Netgate

    Try this:

    Delete all your shapers and remove anything in any rules attempting to steer traffic into shapers.  Delete any floating match rules you have defined attempting the same.

    WAN
      Enable
      HFSC
      Bandwidth 20Mb
    qInternet
        Enable
        Default Checked
        CoDel Checked
        Upperlimit Checked m2 = 90%

    You shouldn't need any firewall rules since you only have one queue on WAN and it's the default queue. No LAN - LAN traffic will be shaped since this will only affect WAN out.

    Chances are all you really need to do is keep from saturating your upstream.

    Sure there is a lot more you can do. The next step would probably be to prefer certain upstream traffic over others (such as TCP ACKs, VoIP, DNS, etc) but that is outside your stated goal.

    Nothing you do on LAN/Downloads will help with bufferbloat since you can probably send out your gigabit LAN faster than your ISP can send to you so that, too, is another exercise.

    You can tweak the qInternet upperlimit higher until you start seeing symptoms then back it off.



  • Ok, so i think my entire hold-up was probably how i have a multi-LAN setup.  IF i set a traffic shaper on one of the LAN interfaces, with the goal being to throttle downstream internet traffic, it would also have the side effect of shaping any LAN to LAN traffic that passes through that interface (like from LAN wired to LAN wireless).    Traffic going from one host to another on the same LAN (two hosts on wired LAN, for example), since those hosts are connected off a switch,  and are in the same subnet, they aren't routing to a different subnet and their traffic isnt being throttled.

    this was probably my hold-up the entire time as I was testing from my wireless laptop to a wired server.

    If i put every thing on my LAN on the same subnet and turn on the shaping on the LAN and WAN interface, i'll get my expected throttling of internet traffic (Because i'm just dealing with a single WAN and single LAN interface).

    Anyways, i think my multi-LAN setup had me tripped up and i was missing the obvious.

    Thanks everyone for your responses and tips/tricks.

    I think i'll just set up a basic CODELQ shaper (unless there is a better scheduler to use) for WAN and one LAN and have all my hosts on the same LAN – then i'll get full gigabit between hosts on the LAN and throttled back internet from WAN <> LAN


Log in to reply