Newbie banging against the wall High Latency HFSC
-
Hi group
As I posted in the subject, I'm newbie here so please be patient for my questions and my english ;)
I'm trying to make traffic shaping with latest release. I have one pc (dell optiplex with 2 nics) and my laptop and my cable modem with 4M/2M. Only for testing purposes while I deploy for large LAN.
As newbie I followed the wizard (single LAN multi WAN) with HFSC. So put it to work and inmediatly the ping to my pfsense box is high when I make any move on the GUI. Reading on google, when I get off P2P from being the default queue, latency gets normal. I don't know why. (that's the first question).
If I put qOthersLow as default queue everything goes OK and traffic shaping with L7 filtering for bittorrent becomes online. Once is active I probed L7 filtering and bittorrent goes down as I surf on web or download anything by HTTP (which is excellent!!) but there's high latency when I ping to pfsense box. (that's the second question)
So,
1- why P2P as default queue puts so slow pfsense?
2- why once the traffic shaping goes online and makes the job, there's high latency?Hopes I can explain myself
Thanks in advance.
-
That is because ping traffic is being caught in the P2P queue. It is not specifically sent to any higher level queue and is treated as P2P traffic.
Try adding a rule to classify ICMP traffic from any to any and send it to say, qOthersHigh or qACKs.
-
OK, I will do it and post the results!
Thanks in advance!
-
Thanks for your help dreamslacker!
Ok, I did what you told me. I created a firewall rule ICMP which sends ICMP traffic to qOthersHigh/qVoIP. It seems it worked fine until I put bittorrent to work. After that the ping went to high to pfsense box. Internet surfing slow and gmail chat goes offline. I'm attaching the screen shots.
This is my config summary
Wizard
WAn Scheduler type:HFSC 512Kb BW
–qACK Priority 6 ECN BW 17.95% LinkShare m2 17.95%
--qOthersDefault Priority 3 Default queue ECN BW 8.975%
--qP2P Priority 1 Queue limit 500 ECN BW 2% Upperlimit m2 2% Linkshare m2 2%
--qVoIP Priority 7 ECN BW 32kb RealTime m2 32kb
--qOthersHigh Priority 4 ECN BW 8.975% LinkShare m2 8.975%
--qOthersLow Priority 2 ECN BW 2% Linkshare m2 2%LAN Scheduler type:HFSC
-qInternet ECN BW 1048.576Kbps Upperlimit m2 1048.576kbps Linkshare m2 1048.576
--qACK Priority 6 ECN BW 18.59% Linkshare m2 18.59%
--qP2P Priority 1 ECN BW 2% Upperlimit m2 2% Linkshare m2 2%
--qVoIP Priority 7 ECN BW 32kbps realtime m2 64kb
--qOthersHigh Priority 4 ECN BW 9.295% Linkshare m2 9.295%
--qOthersLow Priority 3 Default queue ECN BW 2% Linkshare m2 2%Firewall rules
LAN
ICMP any/any Queue: qOthersHigh/qVoIP
TCP/UDP any/any L7 filtering for BTT
any LAN net defaultLayer 7
Protocol
Bittorrent queue qP2PI believe there's recipes for everyone needs so I would like if you guys can help me with this. I suspect this high latency is by any rule which is badly configured. So please check my summary config to find something wrong.
Thanks in advance
-
ICMP has no TCP component so you should not populate the ACK queue. Just send it to qACK or qVOIP.
Next, remove ECN on the ACK/ VOIP and root queues.
Also, what is the hardware you're running on? L7 uses a fair bit of processing power. I find that upperlimits tend to create the problems you've stated. Try using Limiters instead to limit p2p traffic.
-
Hi Group
Ok, I think I did my homework right. ;)
1- I created a firewall rule on LAN side which sends (I guess) ICMP to qVoIP only.
–-- New rule,
---- Action: Pass
---- Protocol: ICMP
---- Source: any / Destination: any
---- Advanced feature: Ackqueue / Queue: none/qVoip2- I removed ECN on ACK/ VoIP.
---- What I did was unchecked ECN from the next queues:
---- WAN: qACK, qVoIP
---- LAN: qACK, qVoIPOk, the hardware, Pentium 4 Dell Optiplex, 2 NICS (fxp, rl) with 512Gb Ram. On dashboard seems ok on the CPU processing.
Until now with those changes, I saw an increase on ping latency as attached. And also I think is more permisive than before config.
I will try to configure the limiters ( I don't know how, I will make some search) and will disable L7 as you told me. I will post my results later so apologize for that. I'm trying to priorize this effort to make it work.
So thanks again and see you later.
-
Hi Group
Ok, This now what I got
WAn Scheduler type:HFSC 512Kb BW
–qACK Priority 6 BW 18.35% LinkShare m2 18.35%
--qDefault Priority 3 Default queue ECN BW 9.175%
--qP2P Priority 1 ECN BW 4.5875% Upperlimit m2 4.5875% Linkshare m2 4.5875%
--qVoIP Priority 7 ECN BW 32kb RealTime m2 32kb
--qOthersHigh Priority 4 ECN BW 9.175% LinkShare m2 9.175%
--qOthersLow Priority 3 ECN BW 2% Linkshare m2 2%LAN Scheduler type:HFSC
--qLink Priority 2 Queue limit: 500 Default queue ECN BW 20%
--qInternet ECN BW 1048.576Kbps Upperlimit m2 1048.576kbps Linkshare m2 1048.576
---qACK Priority 6 BW 18.99% Linkshare m2 18.99%
---qP2P Priority 1 ECN BW 4.7475% Upperlimit m2 4.7475% Linkshare m2 4.7475%
---qVoIP Priority 7 BW 32kbps realtime m2 64kb
---qOthersHigh Priority 4 ECN BW 9.495% Linkshare m2 9.495%
---qOthersLow Priority 3 ECN BW 2% Linkshare m2 2%Limiters
P2P_limiter_down 20kbps
P2P_limiter_up 10kbpsLayer 7
btt_limiter_L7
-Protocol: bittorrent
-Structure: limiter
-Behaviour: P2P_limiter_downRules
LAN
-Proto: TCP/UDP
-Src/Dst: any/any
-Action: Pass
-In/out: P2P_limiter_up/P2P_limiter_down
-Layer7: btt_limiter_L7Results:
Excelent ping latency, gmail chat offline, skype offline, slow internet surfing.I put limiters (I think that's the way is configured) but I noticed no change on btt limiting. I didn't know how to put limiters without L7 filtering so please some guidance would be very appreciated.
Hope this serves
Thanks in advance.
-
Hi group
I just tried PRIQ with limiters. 512up/1024down. I found limiters work only by setting a rule and saw the floating rules so I put them there.
I think is better HFSC :-). (at least Bittorrent was controlled a little bit more, while with PRIQ was very permisive).
This is my configuration
WAN Scheduler PRIQ BW 512kbps
–qACK Priority 6, ECN
--qOthersDefault Priority 3, ECN
--qP2P Priority 1, Queue Limit:500, Default Queue, ECN
--qVoIP Priority 7, ECN
--qOthersHigh Priority 4, ECN
--qOthersLow Priority 2, ECNLAN Scheduler PRIQ
--qACK Priority 6, ECN
--qP2P Priority 1, Default queue, ECN
--qVoIP Priority 7, ECN
--qOthersHigh Priority 4, ECN
--qOthersLow Priority 3, ECNFloating rules
m_P2P BitTorrent outbound
Action: queue
Interface: LAN
Direction: in
In/out: limiters 10Kup/limiters 10KdownResults: BTT download at 400KB/s, ping latency ok except when BTT is online.
Also backed to HFSC and limiters but the result is the same. Altough when HFSC is active, btt is given less priority than http so when I surf the web, btt reduce their download speed.
I'm trying to make differents combinations, but I need some help over here... :o :(
My plans are to implement in small-medium LAN (let's say 1000 users) to controll bandwith and P2P. Also want to check protocols by users, but I guess I need to learn to walk before run.
So hope I explained correctly.
Thanks in advance.
-
Hi Group
I'm now bussy doing other stuff where I work so I've couldn't make some tests for my pfsense purposes. Reading in forum seems there's lot of people who's giving a lot of resources to understand and to running up pfsense. So here's my request for help:
What do you think is the best resource (web/book) for getting started and to understand HFSC and pfsense?
I mean, i.e: first you need to read this, then this and later this…
I know that many times people doesn't have the time for help, so we the beginners need to make our best effort and try to learn for ourself. So please, links, links, but above all a guidance... :)
Thanks in advance
-
Sorry I took so long. The thread was pushed down a little too far.
For starters, can you state the WAN connection limits (what you can really get up and down) and what you need prioritised (ie. specific voip applications) or penaltied (ie. torrent)?
Note that most torrents these days are encrypted and the L7 won't do much to catch the traffic. It's better to do a catch all and penalty then manually select what you want and prioritise it.
-
The rule may be working for the l7, but there is also something else that will prevent it from working properly: pfSense doesn't like it when you select TCP/UDP. You need two rules, one TCP and one UDP. It's a long-standing issue that I've often been annoyed with.
-
Hi group
Thanks for the answers. I was losing my faith :o. My results:
Sorry I took so long. The thread was pushed down a little too far.
For starters, can you state the WAN connection limits (what you can really get up and down) and what you need prioritised (ie. specific voip applications) or penaltied (ie. torrent)?
Note that most torrents these days are encrypted and the L7 won't do much to catch the traffic. It's better to do a catch all and penalty then manually select what you want and prioritise it.
OK, for lab purposes we have a cable modem with 4M Down / 1M up, one laptop and the pfsense box with 2 nics. But on wizards I've been working with 1024k DW/512k UP. What I'm trying to prioritize for lab purposes is HTTP and VoIP (firefox and skype) with the percentages posted before giving the highest priority possible. What I'm trying to penalize is torrents (bittorrent and Ares). I know L7 can't stop encrypted torrents but most of our users are unaware how to convert torrents into encrypted (by default torrents are unencrypted) so I think is good to try.
On tests posted, I think is better using HFSC instead of PRIQ because torrents were penalize while I was surfing but I felt navigation slow.
The rule may be working for the l7, but there is also something else that will prevent it from working properly: pfSense doesn't like it when you select TCP/UDP. You need two rules, one TCP and one UDP. It's a long-standing issue that I've often been annoyed with.
Let me do the tests so I can give you some results.
I'm not giving up until my pfsense box is completely working. ;)
If you guys have some traffic shaping "recipes" would be a great help for me.
Thanks in advance.
-
OK, for lab purposes we have a cable modem with 4M Down / 1M up, one laptop and the pfsense box with 2 nics. But on wizards I've been working with 1024k DW/512k UP. What I'm trying to prioritize for lab purposes is HTTP and VoIP (firefox and skype) with the percentages posted before giving the highest priority possible. What I'm trying to penalize is torrents (bittorrent and Ares). I know L7 can't stop encrypted torrents but most of our users are unaware how to convert torrents into encrypted (by default torrents are unencrypted) so I think is good to try.
Ok. Forget the Wizard then. With single WAN, single LAN, I find it better to manually create queues.
For starters, under WAN root (HFSC 512Kbps), create the following queues:
qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
qAck (Priority 6; BW 10%; Realtime M2 1%)
qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)Under LAN root (HSFC), create the following queues:
qInternet (Upperlimit 1024Kb; Priority 1; Bandwidth 1024Kb)
qLink (Upperlimit = Interface bandwidth; Priority 2; Bandwidth = Interface B/w - 1024Kb)And under qInternet:
qVoip (Priority 7; BW 64Kb; Realtime M1 32Kb D 30 M2 64Kb)
qAck (Priority 6; BW 2%; Realtime M2 1%)
qDefault (Priority 3; BW 2%; Upperlimit M2 4%; ECN RED Default)
qOtherHigh (Priority 4; BW 10%; Realtime M2 5%)Note that these rules need to be duplicated on both LAN tab and floating. It is better to do a Quickmatch for floating rules and make sure the order of the rules is correct.
i.e. Rules with specific ports at the top, catchall with L7 after then catchall for default is at the bottom.Use Catchall rule with L7 container for FTP to have rules redirect to qAck/ qOtherHigh.
Use catchall rule with L7 container for Skype to have rules redirect to qVoip.
Use Firewall rules to match ICMP traffic to qAck.
Use Firewall rules to match HTTP, HTTPS, POP3, SMTP etc. to qAck/ qOtherHigh.Use a catchall rule to pipe to qDefault. This will catch all traffic that isn't explicitly prioritized including encrypted traffic. Technically, it's not required but it can be used if you need to add more rules in future.
-
Hi group. Sorry for the delay
Thanks dreamslacker. It's working now. HTTP went good, btt was slow and skype was good. I think that the real approach here is to permit (with some priority) important protocols and the rest goes to default (less priority).
I want to attach the rules so you can give me some advices if I configured them wrong or if they are good. Is in excel format. What you find in there is all rules configured as I thought was ok. Also if someone likes it, feel free to download it.
I also want to ask you:
1- Do I have to disabel/erase the anti-lockout/default rules in LAN firewall?
2- Can I put it to work my pfsense in transparent mode with the shaping rules you gave me?
3- I've been testing also the rules with 4M and the only thing I've changed in configuration has been the WAN root, WAN qVoip, LAN qInternet, LAN qVoip and LAN qLink. Is this correct?4- You gave me this rules to make it manually, so is there something wrong with wizard? I've noticed that many people in forum has the same issues.
I want to thank you a lot for the time and now I have some lights.
Thanks again.
-
Now the excel format.
-
The floating rules can be modified so that it has:
Source port: 80/ 443 (web servers serve out of these ports; the destination port for inbound traffic is dynamic)
Destination: Lan SubnetThis will reduce the amount of inspection since it's only concerned with traffic that is bound for the LAN subnet (that is, inbound traffic).
1) There is no need to disable the anti-lockout unless there is a specific need to harden the firewall in that segment. Just be careful not to lock yourself out of the admin interface if you do disable them.
2) I've not tried transparent mode but there is no reason to believe that it will not work. Some changes will be needed since the firewall no longer sees different network segments.
3) More or less correct.
4) The traffic shaper wizard doesn't seem to create rules nicely. And most certainly doesn't create inbound rules as expected. I personally prefer manually creating them because I tend not to have symmetric links and need to adjust all the queues accordingly.
-
dreamslacker and cabo81, thanks for the VERY informative discussion.
I'm trying to replicate this setup but I am running into some issues. Could one of you please turn this into a How-to with screenshots?
1. What's the purpose of the qLink queue? I never saw any packets go through it, and qInternet would just ram up against it's limit (set artificially low).
2. How do you apply the Floating rules? To all interfaces or exclude LAN? Should they be Pass or Queue rules? In, out or any?
3. pfSense seemed to struggle with having the same queue names (especially qDefault) created on different interfaces. Should the queues have interface-specific names (like qDefault_WAN, qDefault_LAN) or am I doing something wrong?
4. As you scale up the bandwidth (ex. 20mDL/4mUL) should the percentages of the queues change?
5. When I enabled the L7 rule for Skype my CPU load went crazy, and Skype wasn't even running (pfSense 2.0 on ALIX)
Thanks in advance!
-
Hi irvingpop
I'm glad this topic has guided you at least a little. I have to tell you that my knowledge on pfsense is still very reduced. Not for the group which have helped me a lot but for the time I can't spend working on my pfsense box. The last thing I was trying to do is to convert it into transparent mode and I had some issues I'm checking on web to solve them. Some of your questions are same for me. So let me response and then let's hope we get an answer:
1- This is an excellent question. I don't have a clue what for is this queue.
2- As dreamslacker told me we apply the floating rules exactly as in lan rules. I used the Pass attribute and the default in.
3- I didn't rename the queues.
4- As dreamslacker told me, Is more or less correct.
5- I did not see that behaviour on my box.As you can see I can't tell you more than I know. What I can make for you is to give you my config file so you can see it and test with it. Please any advance you have share it with us. Is in txt. Please convert it to xml
Hope this helps.
PD: when I had some time, I'll post a how-to
-
The qLink is for the interface. Basically should stop things being classified under your 'net download rate when it should be the interface rate (in the case of Squid proxy, LAN<->OPT traffic, etc.)
You have to make your own rules for the qLink queue though.
-
Does anyone tried to recreate dreamslacker shaper for multiple WAN ?