FQ_CoDel Optimizations for cable connections

[kaj]

I have a cable connection with Vodafone in Germany with a 1 Gbit/s download and 50 Mbit/s upload volume.
My cable router is switched to bridge mode and a pfsense is connected behind it.
I receive a WAN IPv4 and a WAN IPv6 address from the provider in dual-stack mode

With the standard settings of the FQ_CoDel Sceduler :

ECN: checked

• Firewall > Traffic Shaper > Limiters:
  * WANDownIPv4/WANDownIPv4Q
  * bandwidth: 600Mbps
  * Queue mgmt algo: Tail Drop
  * Scheduler: FQ_CODEL (5/100/1514/10240/4096)
  * Queue length: 3000
  * ECN: unchecked

Especially with the target parameter 5 ms and the interval value of 100 ms, I always have the problem that the latency in the download only improves when I limit the download bandwidth from 1 Gbit/s to around 600 Mbit/s

I have found a document at Cablelabs in relation to Docsis and AQM, under the website: https://www-res.cablelabs.com/wp-content/uploads/2019/02/28094021/DOCSIS-AQM_May2014.pdf

In chapter 4.2 the author has made tests with modified target and interval parameters, which are more suitable for use in cable environments. In his tests, he increased the Targer value from 5 ms to 20 ms and increased the Interval value from 100 ms to 150 ms.

I did the same and was very positively surprised by the result.

here my changes:

• Firewall > Traffic Shaper > Limiters:
  * WANDownIPv4/WANDownIPv4Q
  * bandwidth: 600Mbps
  * Queue mgmt algo: Tail Drop
  * Scheduler: FQ_CODEL (20/150/1514/10240/4096)
  * Queue length: 3000
  * ECN: unchecked

The result was so amazingly good that I upgraded my download limiter to 900 mbit/s and still achieved excellent results:

I have been testing for days and have further adjusted the target and the interval.

As we use both Docsis 3.0 and Docsis3.1 in Hibrid mode in Germany, I have set the target value to 10 ms for Docsis 3.0, as described in this document in section 3.2: https://www-res.cablelabs.com/wp-content/uploads/2019/02/28094118/Active_Queue_Management_Algorithms_DOCSIS_3_0.pdf

This is my final configuration at the moment:

• Firewall > Traffic Shaper > Limiters:
  * WANDownIPv4/WANDownIPv4Q
  * bandwidth: 960Mbps
  * Queue mgmt algo: Tail Drop
  * Scheduler: FQ_CODEL (10/150/1514/10240/4096)
  * Queue length: 4000
  * ECN: unchecked

The results speak for themselves:

Here are a few more results:
https://www.waveform.com/tools/bufferbloat?test-id=69bb2004-77cc-4d1d-867f-a17de3ac2a1d
https://www.waveform.com/tools/bufferbloat?test-id=4f70df70-2058-49b6-b900-b346d5e6ce3f
https://www.waveform.com/tools/bufferbloat?test-id=041d3bc3-64da-4145-920a-9b3cf51d5deb
https://www.waveform.com/tools/bufferbloat?test-id=56b7bfad-aa20-435d-9981-505cf77d747d

The amazing thing is that with FQ_Codel I always had problems with the download latency never going down, so I even thought that the FQ_Codel implementation was buggy. Since a few days I know that I have to adjust the parameters target and interval for a cable internet connection. The default values for target 5 ms and interval 100 ms, as you can read everywhere on the Internet, are not optimal for cable connections based on my findings.
It would be nice if other users could also test these values.

Best regards kaj

[artenpie]

@kaj Thanks for sharing. I'm on DOCSIS 3.1 in the US, rated at 400/10 (provisioned more like 425/11). Based on your results, I changed just the Target and Interval values to the CableLabs suggested 20/150, and saw significantly better bandwidth. My BW previously struggled to hit ~375 Mbps, now it's comfortably over 400 Mbps. (My latency numbers were already pretty good with pfSense's defaults, so not much change there.)

I am not sure how the AQM that is supposedly part of my modem's firmware affects these results. There's no way to turn it off, so I guess there's no way to test.

[artenpie]

@artenpie Update: Further testing showed my connection struggling again to hit 375 Mbps on Waveguide's test. I expect there's a lot of variability in the ISP network itself, by time of day, etc.

[kaj]

@artenpie
Yes, I have the same experience and this is typical for a shared medium based cable connection. Here is a 24-hour evaluation of my provider router regarding the occupancy of my cable segment:

The screenshot clearly shows how the upload in the segment reaches a saturation of over 90% towards the evening.
This naturally has an extreme effect on the download bandwidth and latency.
This example clearly shows that the download to upload ratio is too high. In my case it is a factor of 20 (1000mbit/s : 50 mbit/s). The more symmetrical the download to upload ratio is chosen, the lower the latency will be.
Here you can see impressively how the cable segment to which my CMTS is connected is overprovisioned. I therefore have to share the provider line from my cable modem to the CMTS remote station with around 750 subscribers. Poor Germany :-)

[provels]

Interesting reading. I have a nominal 150/20 line with Comcast and a DOCSIS 3.0 modem (Netgear CM600/Broadcom chipset). FWIW, I setup my limiters according to the pfSense doc, left the Queue Length blank and ECN checked and get this result on a Sunday evening when things should be pretty busy. Not really sure why the "Gaming" category would warn, as it's often A+/A+. ¯_(ツ)_/¯

Anyway, seems to work pretty well and just posting this should it help anyone with a similar setup.