Complex setup (4 WAN, multiple VLANS) rate limiting



  • Hi,

    I'm taking my first steps on QoS/Traffic Shaping and for the first task i wanted to solve the following problem:

    When a VPN client saturates a WAN link or sometimes a client downloads at maximum speed the links start to have horrible latency, so bad that the MultiWAN kicks it from the routing group. (not a problem since the VPN link stays active and the routing group that kicks it uses it as tier3 only).
    I wanted to know if somehow limiting or otherwise shaping that traffic would allow me to keep the link's latency under control even if it means a little less speed for those using it.
    I currently have the following:
    pfSense 2.2.6 (i'm waiting for a good weekend to upgrade to 2.3 since it's a big change) running inside a VMWare ESXi 6.0U2 host inside an HS22 Blade and Cisco 3110G stacked switches with LACP to Dell 5548 stacked core.
    4 virtual ethernets working as WAN (2x cablemodems, 30down/2up, 20/2 and 2x ADSL 10/1 and 6/1)
    1 virtual ethernet to the internal server vlan (there is no tagging done at the pfSense level, all tagging or untagging is done either by the switchs or the VMWare host)
    pfSense has 2 OpenVPN servers listening on all interfaces (one for my users and the other for a client that is housed here and only sees it's subnet)

    The links that usually have latency problems are the ADSL ones, but i've seen the cablemodems from time to time having long RTTs and some packet drop. If shaping or limiting the interfaces to a bit under their max could help keep the links stable i'm all for it but i don't really understand how to apply the queues to the entire interface and not a firewall rule, since i want it to affect ALL traffic from all groups and subnets…



  • Try CODELQ (a scheduling discipline like PRIQ, HFSC, CBQ, FAIRQ) on each WAN. Set the bandwidth to 98-93% of the link's real-world bitrate.

    Packet drop is normal; that is how most TCP congestion avoidance algorithms register a congested link.



  • Forgot to say, from what i've recorded most of my problems come from the low upload on the WAN links, but the " by interface shaper " doesnt seem to have a way to differentiate traffic coming or going…
    EDIT, I just noticed the "MultiWan Shaper Wizard", is it worth it? do i use nominal or 93~98% of bandwith where it asks for the bw? HFSC, CBQ or PRIQ? bah, i'll start reading the manual before asking questions...



  • @Raiker:

    Forgot to say, from what i've recorded most of my problems come from the low upload on the WAN links, but the " by interface shaper " doesnt seem to have a way to differentiate traffic coming or going…
    EDIT, I just noticed the "MultiWan Shaper Wizard", is it worth it? do i use nominal or 93~98% of bandwith where it asks for the bw? HFSC, CBQ or PRIQ? bah, i'll start reading the manual before asking questions...

    I prefer manual setup with traffic-shaping. Create a firewall rule to catch the traffic then use that same firewall rule to assign the traffic to a queue.

    Regarding the queue bitrate, set it to the approximately the lowest common maximum that you observe during a speedtest.



  • Ok, i've got various gateway definitions (2 cablemodems being T1, one being T1, ADSL being T1, etc) as a way of distributing traffic between interfaces. To catch all traffic for a given interface, without messing all the other rules, how should i build the rule? And being that not a single one of those wan links is symmetrical, where CODELQ asks for bandwith, does it mean download or upload?


Log in to reply