Playing with fq_codel in 2.4
-
I came back here to say thanks because it works well. I completed my setup differently than some of what has just been posted.
I setup limiters just as seen in the screenshots. (post 121)(upload, download, wan, lan)
I ran the single command for IPFW pipes. (ipfw sched 1 config pipe 1 type fq_codel && ipfw sched 2 config pipe 2)
I installed shellcmd and added the single IPFW statement.
Modified the two stock LAN firewall rules (IPV4 and IPV6 advanced configuration) so that wan and lan would be used just as seen in the screenshots.
I restarted the firewall.That is all I have done. Prior my buffer bloat was a D to F. Post I get an A each time. I may/may not be setup correctly but whatever it is works. I originally used the wizard for setup of traffic shaping which used HFSC and which gave @425 upload on my gigabit connection. This new setup gives @750. So, good for me.
-
Definitely I am blind what screenshots are you all talking about? :D
-
@w0w:
Definitely I am blind what screenshots are you all talking about? :D
Reply 121 of this thread.
-
Thanks. :)
-
Floating rules vs interface rules won't make a difference. It will also work well on VPN clients. VPN traffic will always have higher latency relative to the same traffic not routed through a VPN. fq_codel can't fix that, but it will still work with fairly queuing the traffic and reducing bufferbloat.
I tested floating rules vs. lan rules and they both give excellent results. Latency results in bufferbloat tests seemed to be just slightly lower with the lan rules, but that's just splitting hairs.
I had very poor bufferbloat results when testing through my OpenVPN connection as a client connected to the OpenVPN server in pfSense. Is there any way to fix this? Should I be creating limiters to apply to the OpenVPN interface rules in the firewall and then selecting fq_codel on those limiters, as well?
-
Yes you would need to apply limiters to your openvpn interface in order to queue your clients traffic. However, you can only fix your end, if the client is connecting to you via a poor connection then you can't get any better than the worst link.
-
Yes you would need to apply limiters to your openvpn interface in order to queue your clients traffic. However, you can only fix your end, if the client is connecting to you via a poor connection then you can't get any better than the worst link.
Thanks, that makes sense.
I’ll try it out and see how much it helps. -
Finally got around to trying this again, and everything worked great! John's screenshots in reply 121 are spot on and there is no need to edit any files if one uses shellcmd.
I actually recently changed to a 100/100 Fiber connection - here are results (using the DSL Reports speed test which has a nice Bufferbloat check):
Before (no shaping):
Using ALTQ FAIRQ + Codel Active Queue Management; 100Mbit Limit on Both WAN and LAN:
Using fq_codel and 100Mbit Limit on Both Upload and Download:
What's interesting to me here is that fq_codel appears to perform a bit better than the ALTQ emulation of fq_codel (using FAIRQ + Codel) - I find this very interesting. Anyone have any thoughts as to why?
I also ran a more intense FLENT test on another system with fq_codel enabled and the results looked great as well (stable ping and stable download/upload over the course of the test).
Given the relatively little effort required to get this to work on pfSense, it's a fantastic way to improve the stability of a connection.
-
Finally got around to trying this again, and everything worked great! John's screenshots in reply 121 are spot on and there is no need to edit any files if one uses shellcmd.
I actually recently changed to a 100/100 Fiber connection - here are results (using the DSL Reports speed test which has a nice Bufferbloat check):
Before (no shaping):
Using ALTQ FAIRQ + Codel Active Queue Management; 100Mbit Limit on Both WAN and LAN:
Using fq_codel and 100Mbit Limit on Both Upload and Download:
What's interesting to me here is that fq_codel appears to perform a bit better than the ALTQ emulation of fq_codel (using FAIRQ + Codel) - I find this very interesting. Anyone have any thoughts as to why?
I also ran a more intense FLENT test on another system with fq_codel enabled and the results looked great as well (stable ping and stable download/upload over the course of the test).
Given the relatively little effort required to get this to work on pfSense, it's a fantastic way to improve the stability of a connection.
As I understand it, the biggest difference between FAIRQ + CoDel and fq_codel is that fq_codel individually applies codel to each per-flow pseudo-queue while FAIRQ + CoDel applies codel to the entire queue. There are also other subtle differences between codel and fq_codel, like the "fq" in fq_codel being a bit smarter than standard "fair queueing".
Either way, the 4ms difference you observed in best-case latency could just be a fluke.
Thanks for sharing the comparisons, btw.
-
I really don't get much difference. I was using OPNSense and fq_codel prior as it seemed to just work better for me.
With the new release, I changed back and just use HFSC queues with codel checked and some very basic rules to make sure my gaming traffic is first and my non important (downloads for media and other odd plex related download stuff) is limited. Works like a champ.
Only thing for me always comes back to making sure my upload and download limits match close to reality what I expect out of my link so I use 940 down and 880 on Verizon's Gigabit FIOS with 1000 queue. No drops and no bufferbloat that I've been able to make happen.
-
Thanks all for the feedback. i do have a quick follow up question as I think that I may have misconfigured something:
I actually ended up creating two limiters, one at 100Mbit up/down, the other at 25Mbit up/down to use on a guest network. Went through the same process and enabled fq_codel on the second set of limiters. Applied the limiters inside the firewall rules on the guest network, but for some reason when I try to test out the configuration with a machine on the guest network I'm able to go faster than the limited speed of 25Mbit. However, the interesting thing is that does not seem to be consistent - for instance:
- When running a speedtest on speedtest.net I'm limited to just 25Mbit (as expected)
- When running a speedtest on DSLReports I'm able to go well beyond 25Mbit (almost to full speed).
I haven't been able to try an iperf3 test yet unfortunately. Could it be that something is misconfigured and that the 25Mbit limit is applied per flow vs. the queue as a whole?
Thanks in advance for any insight you might have.
P.S. Some thoughts regarding fq_codel vs. FAIRQ + Codel: At least in my case, using fq_codel consistently results in a bufferbloat average (for both upload/download) under 10ms. Using FAIRQ + Codel it often goes beyond that, but never higher than 15-20ms. Ultimately, I suppose it's not really a big deal, but I found it interesting nonetheless.
-
Thanks all for the feedback. i do have a quick follow up question as I think that I may have misconfigured something:
I actually ended up creating two limiters, one at 100Mbit up/down, the other at 25Mbit up/down to use on a guest network. Went through the same process and enabled fq_codel on the second set of limiters. Applied the limiters inside the firewall rules on the guest network, but for some reason when I try to test out the configuration with a machine on the guest network I'm able to go faster than the limited speed of 25Mbit. However, the interesting thing is that does not seem to be consistent - for instance:
- When running a speedtest on speedtest.net I'm limited to just 25Mbit (as expected)
- When running a speedtest on DSLReports I'm able to go well beyond 25Mbit (almost to full speed).
I haven't been able to try an iperf3 test yet unfortunately. Could it be that something is misconfigured and that the 25Mbit limit is applied per flow vs. the queue as a whole?
Thanks in advance for any insight you might have.
P.S. Some thoughts regarding fq_codel vs. FAIRQ + Codel: At least in my case, using fq_codel consistently results in a bufferbloat average (for both upload/download) under 10ms. Using FAIRQ + Codel it often goes beyond that, but never higher than 15-20ms. Ultimately, I suppose it's not really a big deal, but I found it interesting nonetheless.
Looks like the issue I was experiencing has to do with the Squid Proxy running on the guest network. Similar to what was described here:
https://forum.pfsense.org/index.php?topic=132960.0
I'll go ahead and start a separate thread as I may need some help configuring the proper rules to get this work.
-
Implementing fq_codel improved the dsl reports TO A AND B but USING hfsc and CODEL I get better results ALL A+, I tried a linux distro with fq_codel got same A,B and sometime C but again with Pfsense HFSC and codel I get all A+, so for me I am getting better results with HFSC and Codel
-
Implementing fq_codel improved the dsl reports TO A AND B but USING hfsc and CODEL I get better results ALL A+, I tried a linux distro with fq_codel got same A,B and sometime C but again with Pfsense HFSC and codel I get all A+, so for me I am getting better results with HFSC and Codel
Did you configure manually or use the wizard? I used the wizard with HFSC selected and received better grades on dslreports but speed was much lower overall. The scores were better because the throttle was more aggressive. Would you be willing to share your config? Screenshots maybe. I would like to compare what I get using fq_codel as described in this thread.
-
It's possible that HFSC+ALTQ gives better rate limiting characteristics compared to IPFW.
-
Implementing fq_codel improved the dsl reports TO A AND B but USING hfsc and CODEL I get better results ALL A+, I tried a linux distro with fq_codel got same A,B and sometime C but again with Pfsense HFSC and codel I get all A+, so for me I am getting better results with HFSC and Codel
Did you configure manually or use the wizard? I used the wizard with HFSC selected and received better grades on dslreports but speed was much lower overall. The scores were better because the throttle was more aggressive. Would you be willing to share your config? Screenshots maybe. I would like to compare what I get using fq_codel as described in this thread.
Sure in dsl buffer bloat test I get half the speed but thats cos if it goes over that speed I get buffer bloat , but running a normal speed test with same setup I get my full speed, so I only get half with dsl reports so for me HFSC and codel are doing a fine Job but I am sure many more experts here can correct me. A other thing using ipfw limiters when using the full upload speed it does not give example enough bandwidth plex remote users need, in hfsc it takes bandwidth from example the upload backup to the cloud and gives plex itS full 5mbps it needs
-
I have fq_codel working on my system without issue. I followed the screenshots from post #121.
Question:
If I apply the same lan / wan queues to the In / Out on my IPsec interface rule will bandwidth then be shared evenly between multiple IPsec clients?
I have several people that access server resources and it would be great if the bandwidth was shared evenly when everyone was trying to perform a get operation.
Thanks
-
to the guys saying they only had to enable in cli and "nothing" else.
You didnt do this step?
Start with a recent 2.4 snapshot. Create two root limiters, Download and Upload, and put 95% your maximum values in bandwidth. Create two queues under each, say LAN and WAN. For LAN, selection destination addresses for mask and source addresses for WAN. Modify the default outgoing firewall rule to use WAN under "in" pipe and LAN under "out" pipe.
Also the limiter is surviving all filter reload's?
-
to the guys saying they only had to enable in cli and "nothing" else.
You didnt do this step?
Start with a recent 2.4 snapshot. Create two root limiters, Download and Upload, and put 95% your maximum values in bandwidth. Create two queues under each, say LAN and WAN. For LAN, selection destination addresses for mask and source addresses for WAN. Modify the default outgoing firewall rule to use WAN under "in" pipe and LAN under "out" pipe.
Also the limiter is surviving all filter reload's?
Yes I did that step. When I say I only used the command line I mean I did not install a patch of any kind. I use Shellcmd package to run the command line again each time my system boots.
-
Part of the challenge is trying to figure out what gives better performance is your ISP and what may or may not be going on with your local network.
I've got a 1Gb FIOS line and a pretty 'quiet' neighborhood so I tend to get a very consistent speed for up and download when I'm testing. Since it's not a pure 'lab' scenario, you can't really be sure of the variables in your testing.
I've noticed:
- FQ_Codel seems to have a bit less overhead than HFCS/Codel
- If I get my upload and download speeds set properly, I can straight A+s on any buffer bloat tests
- If I have multiple things going on or something not configured correctly, I tend to get problems
- If you are using a straight up limiter and equally sharing bandwidth across all LAN connections for an example, you won't see your max upload/download as you have it shared equally. To that point, in OPNSense, you would configure a limiter and "weight" your FW rules to prioritize what you wanted.
My rules would look like something like:
Limiters: 10000: 940.000 Mbit/s 0 ms burst 0 q75536 50 sl. 0 flows (1 buckets) sched 10000 weight 0 lmax 0 pri 0 droptail sched 75536 type FIFO flags 0x0 0 buckets 0 active 10001: 880.000 Mbit/s 0 ms burst 0 q75537 50 sl. 0 flows (1 buckets) sched 10001 weight 0 lmax 0 pri 0 droptail sched 75537 type FIFO flags 0x0 0 buckets 0 active Queues: q10002 50 sl. 0 flows (1 buckets) sched 10001 weight 100 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms NoECN q10003 50 sl. 0 flows (1 buckets) sched 10001 weight 10 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms NoECN q10000 50 sl. 0 flows (1 buckets) sched 10000 weight 100 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms NoECN q10001 50 sl. 0 flows (1 buckets) sched 10000 weight 10 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms NoECNWhich created some buckets and than weighted by my firewall rules.
I try to use the concept simple is better as I have very limited rules and only really lower my plex download traffic and prioritize my gaming traffic. Everything else just falls into the defaults.
