Playing with fq_codel in 2.4
-
@netblues said in Playing with fq_codel in 2.4:
@Larrikin And the parent queue to tail drop too.
However I think that the fq-codel will always overshoot initially
I've just done that and tested.
So to be clear, the parent now has Queue Algorithm Management set to Tail Drop, the scheduler as FQ_CODEL and the child Queue Algorithm Management is also set to Tail Drop.
Again, I get the same result.
-
@Larrikin And just make sure you have the needed floating rule in place too
-
@netblues said in Playing with fq_codel in 2.4:
@Larrikin And just make sure you have the needed floating rule in place too
Yep I do. Without it, I get 250mbps, with it I get 300mbps.
When I used a switch (instead of pfsense) to do the shaping I got over 360mbps. Don't know how that switch implemented shaping though and that switch was a loan.
-
Hey mate,
I had best results with gargoyle based on openwrt with it's superior qos and throttling... But at these speeds only a few routers can handle it's intensive cpu use.
With pfsense you may need to tweak more as the way they handle qos is kind of picky.
I have 1000/1000 these days, so qos is off.Maybe try a limiter without fq codel for the upload.
-
@robnitro said in Playing with fq_codel in 2.4:
Hey mate,
Maybe try a limiter without fq codel for the upload.
Yeah, I tried all the other ones. I get about a 5% to 10% improvement, but still not where it should be. Is it possible that pfsense just doesn't have the capability I'm looking for?
-
It definitely should be capable. Do you know what NICs are in the pfSense machine you're running? If they're Realtek, I would not be shocked if that's about all you'll get from them despite being "gigabit". Also, I can't tell from the screen shots, but I believe that generally you only want to enable ECN on the download limiter, not upload. That shouldn't be causing your issue, just something I thought I'd mention.
-
What was the switch you were using on loan that gave the 360mbps?
-
@TheNarc said in Playing with fq_codel in 2.4:
It definitely should be capable. Do you know what NICs are in the pfSense machine you're running? If they're Realtek, I would not be shocked if that's about all you'll get from them despite being "gigabit". Also, I can't tell from the screen shots, but I believe that generally you only want to enable ECN on the download limiter, not upload. That shouldn't be causing your issue, just something I thought I'd mention.
It's an intel quad card and they have no problem reaching gigabit speeds. The issue is the NBN policer given my upload is limited to 400mbps. It hits hard and drops my speed to 250mbps. I'm trying to just get under the radar of the NBN policer so I can get the near maximum speed of the 400mbps without hitting the NBN policier.
I have disabled ECN and appreciate the advice!
-
@q54e3w said in Playing with fq_codel in 2.4:
What was the switch you were using on loan that gave the 360mbps?
Ubiquity.
-
Here is some additional information from the NBN in terms of how they deal with their policy enforcer on uploads (ignore downloads as that is fine).
*******The PBS defines the length of a burst of Layer 2 traffic (either in bytes or milliseconds as set out below) that may be received at ingress to the NBN Co Network for a burst of traffic that pushes the average Information Rate above the configured bandwidth profile for a PIR traffic class. Traffic in excess of the PBS will be discarded by the NBN Co Network. The PBS is set by NBN Co for each PIR specification, and cannot be modified.
The PBS is used by the policing functions of the NBN Co Network at ingress to the NBN Co Network to determine whether a stream of ingress data complies with the subscribed PIR. Customer is responsible for ensuring that all ingress traffic is shaped to comply with the PIR/PBS as specified for the required traffic class and interface, before presentation to the UNI-D or NNI as relevant.
It goes on to define the PBS as:
Downstream at the NNI: 10ms
Specific PBS setting in Bytes is dependent on the TC-4 PIR (bandwidth profile) selected
Upstream at the UNI-D: 40,000 bytes
The AVC TC-4 PBS is set by NBN Co and cannot be modified by Customer******* -
Apologies in advance if you've already provided this information - I read back through but may have missed it - but are you saying that with the pfSense machine you can never get above ~300Mbps regardless of whether you use limiters at all? Or are you saying that you've tried different limiter settings but never get above that speed? In other words, through your testing have you isolated this apparent speed ceiling specifically to when you use limiters?
Another interesting test (maybe, I'm not sure whether it would really provide useful information) might be to set up a download limiter. I know that you don't need one, and it would only be temporary. But may be interesting to know whether setting up a download limit with the same parameters as your upload limiter would "over-limit" your download in the same manner.
I can say that I haven't seen anything like this and I use a roughly identical shaping configuration, but also with dramatically different bandwidth limits. I only have 10Mbps upstream, so it's nowhere near the same throughput.
One other thought, and this is really off the wall and should make no difference, but it's also trivial to try: maybe specify the bandwidth in different units. For example, 380,000 Kbps.
-
@TheNarc said in Playing with fq_codel in 2.4:
Apologies in advance if you've already provided this information - I read back through but may have missed it - but are you saying that with the pfSense machine you can never get above ~300Mbps regardless of whether you use limiters at all? Or are you saying that you've tried different limiter settings but never get above that speed? In other words, through your testing have you isolated this apparent speed ceiling specifically to when you use limiters?
Yes, I am able to get over 360+mbps when I put a switch between pfsense and the NBN connection (where the switch is doing the shaping). I don't own that switch and reluctant to buy one to fix this when hopefully we can resolve this within pfsense.
Another interesting test (maybe, I'm not sure whether it would really provide useful information) might be to set up a download limiter. I know that you don't need one, and it would only be temporary. But may be interesting to know whether setting up a download limit with the same parameters as your upload limiter would "over-limit" your download in the same manner.
I've tried this - no difference in results.
One other thought, and this is really off the wall and should make no difference, but it's also trivial to try: maybe specify the bandwidth in different units. For example, 380,000 Kbps.
I'll give this a shot and revert.
-
@Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.
I can also say that on my upload limiter, I have a much lower limit value (1024 as opposed to 10240) based on information from the following sources:
https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/
Also based on those sources I have my upload limiter quantum set at 300, but then you have a much higher upload than the 100Mbps for which the setting of 300 is recommended.
-
@TheNarc said in Playing with fq_codel in 2.4:
@Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.
I'm unfamiliar with ALTQ. How do I go about setting that up? I can't find anything in the GUI.
-
@Larrikin Sorry, it's not referred to as such in the GUI. It's set up on the "By Interface" tab from the "Firewall > Traffic Shaper" page. But typically it's easiest to set it up using the wizard (far right tab, Wizards, on the same page). It's been a while since I've set it up myself, but there's good information in the pfSense book:
Must head to bed here but good luck! Will check back in.
-
@TheNarc said in Playing with fq_codel in 2.4:
@Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.
Boom - you are right. Using ALTQ and switching off Explicit Network Congestion on qLink has fixed the problem. I've disabled the limiters and use this shaping and it works! Thank you good sir.
-
For those who want to know how exactly I got it working, you can find the instructions here:
https://whirlpool.net.au/wiki/pfsense_traffic_shaping
A big thanks to @TheNarc for pointing me in this direction.
Also a big thanks to all the others who contributed to helping me troubleshoot. I am most grateful you took your own personal time to help me.
-
I have a bit of a problem with my setup, and was hoping to gain some insight into why this may be happening. I will say that I'm no traffic shaping expert, but I have played around with it a bit with varying success. The biggest issue for me is trying to solve bufferbloat and have a better zoom experience.
I followed the instructions to get it up and running based on the google hangout post and slideshare instructions here: https://www.slideshare.net/NetgateUSA/pfsense-244-short-topic-miscellany-pfsense-hangout-august-2018
In my particular example, when running a speedtest from dslreports, I'm seeing 350Mbps down and about 22Mbps up. I have since used the speeds when configuring pfsense to 340/20. I have also set a 1700 Queue Length (5 * 340 [speed]). My internal network consists of about 11 different networks; 5 of them using VLAN's, the other are using dedicated NIC's in my pfsense box.
This does improve my bufferbloat score tremendously from a "D" to an "A". The problem I have are icmp packets dropping. When running a speedtest and pinging something (google.com, 8.8.8.8, Comcast's DNS Servers), I never get a response back while running the speedtest. After the test is over, I start seeing responses again. When somebody is on a Zoom conference, I'm seeing about 2% packet loss on the WAN connection.
I came across this bug:
https://redmine.pfsense.org/issues/9024
This sounds similar to what I've been seeing on my end. I was wondering if anybody else has had a similar experience, and if there's a better way to control this than setting a rule up on every interface.
-
While reviewing that bug, there was a comment made: “ If a separate floating match rule is created for ICMP, then packets will not be dropped.”
Have you tried that?
-
@fabrizior said in Playing with fq_codel in 2.4:
While reviewing that bug, there was a comment made: “ If a separate floating match rule is created for ICMP, then packets will not be dropped.”
Have you tried that?
Yup. I first tried to create a regular rule on every interface but was still having problems. I then thought hey, ping is ICMP so let's change this floating rule from "all protocols" to just TCP/UDP. I did that and it fixed my packet loss.
Will this reported bug affect anything else or is this fq_codel setup good to use in production?
Thank You!
