Newbie banging against the wall High Latency HFSC



  • Hi Group

    I'm now bussy doing other stuff where I work so I've couldn't make some tests for my pfsense purposes. Reading in forum seems there's lot of people who's giving a lot of resources to understand and to running up pfsense. So here's my request for help:

    What do you think is the best resource (web/book) for getting started and to understand HFSC and pfsense?

    I mean, i.e: first you need to read this, then this and later this…

    I know that many times people doesn't have the time for help, so we the beginners need to make our best effort and try to learn for ourself. So please, links, links, but above all a guidance... :)

    Thanks in advance



  • Sorry I took so long.  The thread was pushed down a little too far.

    For starters, can you state the WAN connection limits (what you can really get up and down) and what you need prioritised (ie. specific voip applications) or penaltied (ie. torrent)?

    Note that most torrents these days are encrypted and the L7 won't do much to catch the traffic.  It's better to do a catch all and penalty then manually select what you want and prioritise it.



  • The rule may be working for the l7, but there is also something else that will prevent it from working properly:  pfSense doesn't like it when you select TCP/UDP.  You need two rules, one TCP and one UDP.  It's a long-standing issue that I've often been annoyed with.



  • Hi group

    Thanks for the answers. I was losing my faith  :o. My results:

    @dreamslacker:

    Sorry I took so long.  The thread was pushed down a little too far.

    For starters, can you state the WAN connection limits (what you can really get up and down) and what you need prioritised (ie. specific voip applications) or penaltied (ie. torrent)?

    Note that most torrents these days are encrypted and the L7 won't do much to catch the traffic.  It's better to do a catch all and penalty then manually select what you want and prioritise it.

    OK, for lab purposes we have a cable modem with 4M Down / 1M up, one laptop and the pfsense box with 2 nics. But on wizards I've been working with 1024k DW/512k UP. What I'm trying to prioritize for lab purposes is HTTP and VoIP (firefox and skype) with the percentages posted before giving the highest priority possible. What I'm trying to penalize is torrents (bittorrent and Ares). I know L7 can't stop encrypted torrents but most of our users are unaware how to convert torrents into encrypted (by default torrents are unencrypted) so I think is good to try.

    On tests posted, I think is better using HFSC instead of PRIQ because torrents were penalize while I was surfing but I felt navigation slow.

    @Liath.WW:

    The rule may be working for the l7, but there is also something else that will prevent it from working properly:  pfSense doesn't like it when you select TCP/UDP.  You need two rules, one TCP and one UDP.  It's a long-standing issue that I've often been annoyed with.

    Let me do the tests so I can give you some results.

    I'm not giving up until my pfsense box is completely working.  ;)

    If you guys have some traffic shaping "recipes" would be a great help for me.

    Thanks in advance.



  • @cabo81:

    OK, for lab purposes we have a cable modem with 4M Down / 1M up, one laptop and the pfsense box with 2 nics. But on wizards I've been working with 1024k DW/512k UP. What I'm trying to prioritize for lab purposes is HTTP and VoIP (firefox and skype) with the percentages posted before giving the highest priority possible. What I'm trying to penalize is torrents (bittorrent and Ares). I know L7 can't stop encrypted torrents but most of our users are unaware how to convert torrents into encrypted (by default torrents are unencrypted) so I think is good to try.

    Ok.  Forget the Wizard then.  With single WAN, single LAN, I find it better to manually create queues.

    For starters, under WAN root (HFSC 512Kbps), create the following queues:
    qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
    qAck (Priority 6; BW 10%; Realtime M2 1%)
    qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
    qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)

    Under LAN root (HSFC), create the following queues:
    qInternet (Upperlimit 1024Kb; Priority 1; Bandwidth 1024Kb)
    qLink (Upperlimit = Interface bandwidth; Priority 2; Bandwidth = Interface B/w - 1024Kb)

    And under qInternet:
    qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
    qAck (Priority 6; BW 2%; Realtime M2 1%)
    qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
    qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)

    Note that these rules need to be duplicated on both LAN tab and floating.  It is better to do a Quickmatch for floating rules and make sure the order of the rules is correct.
    i.e.  Rules with specific ports at the top, catchall with L7 after then catchall for default is at the bottom.

    Use Catchall rule with L7 container for FTP to have rules redirect to qAck/ qOtherHigh.
    Use catchall rule with L7 container for Skype to have rules redirect to qVoip.
    Use Firewall rules to match ICMP traffic to qAck.
    Use Firewall rules to match HTTP, HTTPS, POP3, SMTP etc. to qAck/ qOtherHigh.

    Use a catchall rule to pipe to qDefault.  This will catch all traffic that isn't explicitly prioritized including encrypted traffic.  Technically, it's not required but it can be used if you need to add more rules in future.



  • Hi group. Sorry for the delay

    Thanks dreamslacker. It's working now. HTTP went good, btt was slow and skype was good. I think that the real approach here is to permit (with some priority) important protocols and the rest goes to default (less priority).

    I want to attach the rules so you can give me some advices if I configured them wrong or if they are good. Is in excel format. What you find in there is all rules configured as I thought was ok. Also if someone likes it, feel free to download it.

    I also want to ask you:

    1- Do I have to disabel/erase the anti-lockout/default rules in LAN firewall?
    2- Can I put it to work my pfsense in transparent mode with the shaping rules you gave me?
    3- I've been testing also the rules with 4M and the only thing I've changed in configuration has been the WAN root, WAN qVoip, LAN qInternet, LAN qVoip and LAN qLink. Is this correct?

    4- You gave me this rules to make it manually, so is there something wrong with wizard? I've noticed that many people in forum has the same issues.

    I want to thank you a lot for the time and now I have some lights.

    Thanks again.








  • Now the excel format.

    pfsense_queues_2.xls



  • The floating rules can be modified so that it has:

    Source port:  80/ 443 (web servers serve out of these ports; the destination port for inbound traffic is dynamic)
    Destination:  Lan Subnet

    This will reduce the amount of inspection since it's only concerned with traffic that is bound for the LAN subnet (that is, inbound traffic).

    1)  There is no need to disable the anti-lockout unless there is a specific need to harden the firewall in that segment.  Just be careful not to lock yourself out of the admin interface if you do disable them.

    2)  I've not tried transparent mode but there is no reason to believe that it will not work.  Some changes will be needed since the firewall no longer sees different network segments.

    3)  More or less correct.

    4)  The traffic shaper wizard doesn't seem to create rules nicely.  And most certainly doesn't create inbound rules as expected.  I personally prefer manually creating them because I tend not to have symmetric links and need to adjust all the queues accordingly.



  • dreamslacker and cabo81, thanks for the VERY informative discussion.

    I'm trying to replicate this setup but I am running into some issues.  Could one of you please turn this into a How-to with screenshots?

    1.  What's the purpose of the qLink queue?  I never saw any packets go through it,  and qInternet would just ram up against it's limit (set artificially low).

    2.  How do you apply the Floating rules?  To all interfaces or exclude LAN?  Should they be Pass or Queue rules?  In, out or any?

    3.  pfSense seemed to struggle with having the same queue names (especially qDefault) created on different interfaces.  Should the queues have interface-specific names (like qDefault_WAN, qDefault_LAN) or am I doing something wrong?

    4.  As you scale up the bandwidth  (ex.  20mDL/4mUL) should the percentages of the queues change?

    5.  When I enabled the L7 rule for Skype my CPU load went crazy,  and Skype wasn't even running (pfSense 2.0 on ALIX)

    Thanks in advance!



  • Hi irvingpop

    I'm glad this topic has guided you at least a little. I have to tell you that my knowledge on pfsense is still very reduced. Not for the group which have helped me a lot but for the time I can't spend working on my pfsense box. The last thing I was trying to do is to convert it into transparent mode and I had some issues I'm checking on web to solve them. Some of your questions are same for me. So let me response and then let's hope we get an answer:

    1- This is an excellent question. I don't have a clue what for is this queue.
    2- As dreamslacker told me we apply the floating rules exactly as in lan rules. I used the Pass attribute and the default in.
    3- I didn't rename the queues.
    4- As dreamslacker told me, Is more or less correct.
    5- I did not see that behaviour on my box.

    As you can see I can't tell you more than I know. What I can make for you is to give you my config file so you can see it and test with it. Please any advance you have share it with us. Is in txt. Please convert it to xml

    Hope this helps.

    PD: when I had some time, I'll post a how-to

    config_file.txt



  • The qLink is for the interface.  Basically should stop things being classified under your 'net download rate when it should be the interface rate (in the case of Squid proxy, LAN<->OPT traffic, etc.)

    You have to make your own rules for the qLink queue though.



  • Does anyone tried to recreate dreamslacker shaper for multiple WAN ?



  • Sorry to post in an older thread but it's hard to find helpful info on traffic shaping in 2.0 with an asymmetric Internet connection.  The wizard apparently isn't much help yet.

    What changes (if any) would you suggest to the below quoted suggestion from earlier in this thread if the WAN connection is 10megs down and 2 megs up (allows short bursting, PowerBoost for Cox)?  Between 100 and 200 users on the LAN at a location that houses international students.  Running transparent Squid proxy with class 2 delay pool (delay_parameters 1 1310720/1310720 393216/393216).  Want to keep upload from being saturated.  Prioritize ACKs.  Prioritize HTTP (goes through Squid), HTTPS.  Maybe prioritize Skype.  Lower priority Netflix.  Already using Snort to try to block LAN addresses that run P2P but for the ones I can't block I want to go to a catchall that will throttle them when the pipe is almost full.

    Also, already using Captive Portal with a per user bandwidth limit set of 3000 down and 550 up.

    @dreamslacker:

    @cabo81:

    OK, for lab purposes we have a cable modem with 4M Down / 1M up, one laptop and the pfsense box with 2 nics. But on wizards I've been working with 1024k DW/512k UP. What I'm trying to prioritize for lab purposes is HTTP and VoIP (firefox and skype) with the percentages posted before giving the highest priority possible. What I'm trying to penalize is torrents (bittorrent and Ares). I know L7 can't stop encrypted torrents but most of our users are unaware how to convert torrents into encrypted (by default torrents are unencrypted) so I think is good to try.

    Ok.  Forget the Wizard then.  With single WAN, single LAN, I find it better to manually create queues.

    For starters, under WAN root (HFSC 512Kbps), create the following queues:
    qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
    qAck (Priority 6; BW 10%; Realtime M2 1%)
    qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
    qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)

    Under LAN root (HSFC), create the following queues:
    qInternet (Upperlimit 1024Kb; Priority 1; Bandwidth 1024Kb)
    qLink (Upperlimit = Interface bandwidth; Priority 2; Bandwidth = Interface B/w - 1024Kb)

    And under qInternet:
    qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
    qAck (Priority 6; BW 2%; Realtime M2 1%)
    qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
    qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)

    Note that these rules need to be duplicated on both LAN tab and floating.  It is better to do a Quickmatch for floating rules and make sure the order of the rules is correct.
    i.e.  Rules with specific ports at the top, catchall with L7 after then catchall for default is at the bottom.

    Use Catchall rule with L7 container for FTP to have rules redirect to qAck/ qOtherHigh.
    Use catchall rule with L7 container for Skype to have rules redirect to qVoip.
    Use Firewall rules to match ICMP traffic to qAck.
    Use Firewall rules to match HTTP, HTTPS, POP3, SMTP etc. to qAck/ qOtherHigh.

    Use a catchall rule to pipe to qDefault.  This will catch all traffic that isn't explicitly prioritized including encrypted traffic.  Technically, it's not required but it can be used if you need to add more rules in future.



  • @Liath.WW:

    The qLink is for the interface.  Basically should stop things being classified under your 'net download rate when it should be the interface rate (in the case of Squid proxy, LAN<->OPT traffic, etc.)

    You have to make your own rules for the qLink queue though.

    What would cause download traffic from the internet to the LAN to be routed to the qLink queue when qLink is NOT the default LAN (interface) queue nor are there any floating rules written for the qLink queue?



  • @miles267:

    @Liath.WW:

    The qLink is for the interface.  Basically should stop things being classified under your 'net download rate when it should be the interface rate (in the case of Squid proxy, LAN<->OPT traffic, etc.)

    You have to make your own rules for the qLink queue though.

    What would cause download traffic from the internet to the LAN to be routed to the qLink queue when qLink is NOT the default LAN (interface) queue nor are there any floating rules written for the qLink queue?

    Nothing.  If there are no rules referencing qLink, then no traffic is sent through qLink queue if it is not a default queue.

    The idea of qLink queue is to use it for traffic that passes through interfaces on the pfSense box or originates from the pfSense box itself.

    For example, a VPN connection is terminated as a virtual interface on the pfSense box.
    Since the VPN tunnel is already shaped by the WAN connection traffic shaping, you do not want to limit the rate at which traffic between LAN and this VPN connection to your internet speed.
    Hence, the artificially high bandwidth queue (qLink) serves to provide an effectively unrestricted queue for passing traffic between the LAN and VPN connection.  The traffic is ultimately still shaped by the floating rules on WAN for the VPN tunnel.

    Alternatively, if we consider services like Squid - the http traffic from the perspective of the pfSense box actually originates from the pfSense box itself.  It is also likely to be cached in memory or from the harddrive.  Both of which are likely to be capable of much higher speeds than the WAN connection.
    If it is allowed to be caught by the default queue, then it will saturate the default queue on WAN even if there is no wan traffic.  By piping it to the qLink queue, it does not affect the qinternet queue which is used to limit and shape actual download traffic on your internet connection.



  • Hi group
    Here my thoughts
    We have configured pfsense for traffic shaping looking for P2P restrictions. The good thing is PFsense achieves controlling P2P, but we are not aware for a real increase of bandwidth for HTTP, HTTPS, DNS, ICMP and Telnet which is our final goal.

    On Images, you can see queues created by wizard. The following are percentages for each queue on LAN and WAN

    QAck:    Priority 6,  Bandwidth 12%,  LinkShare 12%.

    Qp2p:    Priority 1,  Default queue,  Bandwidth 5%,    Upperlimit: 5%,    LinkShare 5%.

    qOthersHigh:    Priority 5,    Bandwidth 82 % ,  LinkShare 82%.

    qOthersLow:      Priority 3,  Bandwidth 1%,      LinkShare 1%.

    Wizard creates all rules Floating as you can see on image.
    Traffic goes for each queue with 100mbps as total BW.
    For P2P queue, traffic goes until 5M which is the 5% of the 100M available

    –---------------------------------------------------------------------------------
    In search for knowledge, we made the traffic shapping in other way, in manual way, without the wizard as you can see on image.
    With this approach, we create only 3 queues with respective percentages.

    Qhhtp Priority 6, Bandwidth 92%, Link share: 92%

    Qack Priority 7, Bandwidth 3%, Link share: 3%

    QDefault Priority 1,  Default queue, Bandwidth 3%, Upperlimit: 3%, Link share: 3%.

    Then we applied those rules only on LAN interface. This is for testing purposes pluging a cable modem with 4M as total BW.

    Making some tests with a pc connected to LAN, the box does aplies restriction over P2P apps like BTT and ares, decreasing BW for them, due to this kind of traffic goes to the queue named QDefault which has an Upperlimit of 3% over the 4M available.

    We have several questions for this:

    1. If I have 100Mb as total BW and I’m able to lower P2P apps wich has only 5% of total BW, and I don’t see a real HTTP (web surfing) improvement, then what can I do? What others tests can I do? How can I assign real improvement over HTTP?
    2. On which interfaces do I have to apply the rules? LAN, WAN or Floating? I think I not entirely understood where to apply them properly.
    3. Rules order does matter?

    Thanks in advance

    ![Firewall -Traffic Shaper- Wizards.png](/public/imported_attachments/1/Firewall -Traffic Shaper- Wizards.png)
    ![Firewall -Traffic Shaper- Wizards.png_thumb](/public/imported_attachments/1/Firewall -Traffic Shaper- Wizards.png_thumb)
    ![Traffic Shaper Manual.png](/public/imported_attachments/1/Traffic Shaper Manual.png)
    ![Traffic Shaper Manual.png_thumb](/public/imported_attachments/1/Traffic Shaper Manual.png_thumb)
    ![Reglas en la Lan.png](/public/imported_attachments/1/Reglas en la Lan.png)
    ![Reglas en la Lan.png_thumb](/public/imported_attachments/1/Reglas en la Lan.png_thumb)





  • Hi group

    I'll post some graphs to understand my network how is configured. Also send you some issues with ping I'm having since Pfsense is in bridge mode.

    Thanks in advance










  • Pfsense's traffic shaping subsystem is in dire need of better documentation and tutorials.

    Oddly, there seemed to be much more substantive conversation about the subject several years ago…



  • Btw I noticed your MBUF Usage 25558/25600 i.e. at the upper limit, you should look into this…



  • Hi group

    Thanks dhatz for your reply. Is very sad to read this. I supposed it because every how-to or manual for TS is in detail for version prior to 2.0. I'm making many tests to post to the group wich contains speed test with differents queue approach. This is to make a little contribution to people in this group.

    Also want to ask you something about the MBUF. You're right didn't realize about the increase in values. Is that a possible reason for the continueos pings being dropped?

    Thanks in advance.




  • When there’s no any free mbuf clusters available FreeBSD enters the zonelimit state and stops to answer to any network requests. You can see it as the zoneli state in the output of the top command.

    The state of used mbuf clusters can be checked with 'netstat -m'

    You can increase quantity of the mbufs clusters through the kern.ipc.nmbclusters parameter:

    sysctl kern.ipc.nmbclusters=65536


Locked