Playing with fq_codel in 2.4

Zeny001

@larrikin said in Playing with fq_codel in 2.4:

For those who want to know how exactly I got it working, you can find the instructions here:

https://whirlpool.net.au/wiki/pfsense_traffic_shaping

A big thanks to @TheNarc for pointing me in this direction.

Also a big thanks to all the others who contributed to helping me troubleshoot. I am most grateful you took your own personal time to help me.

Thanks for this. I was running the dummynet version and after a few months I started to get my bandwidth fucked all the time... I figured something must have bugged out and did a fresh install and kept having the same issue.... Without dummynet and a ton of bufferbloat I was getting 990mbps download and 40 mbps upload.... My package from isp is 1gb/30mbps upload. Dummynet was giving me anywhere from 150mbps to 300mbps download and 0.1mbps upload.

I decided to go the wizard ALTQ version which is listed in your link and now I have A for bufferbloat and 980mbps download and 35 mbps upload. I'm satisfied with this thanks!

NinthWave

@ninthwave said in Playing with fq_codel in 2.4:

@uptownvagrant
When you made the post with the 100/100 connection, how did you come with the bandwidth values for IN and OUT ?

I have tried your settings but the upload is now very bad.

Sometimes it even gets down to zero.

Any idea ?

Maybe I should point out that I have a VOIP service which the vast majority of those having great result don't use.

Plus, I have enabled OpenVPN that I rarely use to check my IP cameras from my cell when I out. But from I have read, the OpenVPN service might be having an effect.

andresmorago

@uptownvagrant said in Playing with fq_codel in 2.4:

WAN-Out FQ-CoDel queue

Hello to all
I have been trying to configure my limiters based on @uptownVagrant tutorial. Im having some issues with the upload speeds as the Bandwidth parameter under the FQ_CODEL_OUT doesnt seem to correctly work for me.

i have a 150/10 cable connection which, without traffic shaper provides the following speed results. they are according to what i pay for.

Server: Movistar - Barranquilla (id = 17577)
             ISP: Telmex Colombia S.A.
         Latency:    41.25 ms   (2.60 ms jitter)
        Download:   156.14 Mbps (data used: 152.1 MB)
          Upload:    11.07 Mbps (data used: 11.5 MB)
     Packet Loss:     0.0%

Setting the upload to 9 Mbits/s will completely block all uploads from my LAN clients. internet access is pretty much dead with this setup.

  Server: Movistar - Barranquilla (id = 17577)
     ISP: Telmex Colombia S.A.
 Latency:    36.99 ms   (3.52 ms jitter)
Download:   140.77 Mbps (data used: 171.6 MB)
Upload:     FAILED
[error] Protocol error: Did not receive HELLO

so i decided to "illogically" increase the upload. test with 50 Mbits/s

  Server: Movistar - Barranquilla (id = 17577)
     ISP: Telmex Colombia S.A.
 Latency:    43.55 ms   (4.42 ms jitter)
Download:   142.25 Mbps (data used: 193.6 MB)
  Upload:     4.07 Mbps (data used: 7.0 MB)
Packet Loss:     0.0%

i increased the upload parameter one more time. this time to 100 Mbit/s (which is 10x larger than my real upload speed)

  Server: Movistar - Barranquilla (id = 17577)
     ISP: Telmex Colombia S.A.
 Latency:    42.05 ms   (3.65 ms jitter)
Download:   139.95 Mbps (data used: 189.6 MB)
  Upload:    10.10 Mbps (data used: 16.5 MB)
Packet Loss:     0.0%

is there anything im missing or omitting on my setup?
why does the upload parameter seem to divide the upload speed by 10?

Here are my parameters so far:

DOWNLOAD

mind12

@andresmorago Seems like you have switched the limit and flows parameter values.
Limit should be 10240 and flows 20480

Ricardox

@mind12 For me so it works perfectly

mind12

@ricardox You also have 10240 configured for the limiter not 20480.

Can you achieve your max speed with such a low queue lengths?
I lost about 15Mbit/s from my 150Mbit download even with a 10K queue length.

Why is the gateway empty for the In queue fw rule? I thought it's a must.
And what's that 100 Weight for in the child queue? Never saw that elsewhere.

Thx

zwck

@mind12 is there a general rule of thumb how to choose target interval quantum limit and flow ?

mind12

@zwck
Idk, I have just used the same working config as others here from this post: https://forum.netgate.com/topic/112527/playing-with-fq_codel-in-2-4/815

Pentangle

@andresmorago Check out your floating firewall rules in/out pipes - are they switched?

Ricardox

@mind12 For my 200/100 MB network I have no loss of speed. X86 PC

Ricardox

@zwck I believe not, change the values and test, for my network these values work well.

zwck

@ricardox whats your advertised line speed?

KOM

This post is deleted!

TheNarc

I don't mean to hijack the thread, but has anyone else seen any catastrophic issues with adjusting fq_codel parameters since upgrading to 2.5.0? I was playing with one of my systems that had limit and flows both set to 1024. The consensus - as much as there is one - seems to be that 10240 and 20480, respectively, may yield better results so long as you're not memory constrained. I have 4GB and it was rarely more than 20 to 30% utilized so I thought I'd try.

Now, for full disclosure, there was some negligence on my part and I was following @andresmorago's post which accidentally had these values flipped (so 20480 for limit and 10240 for flows). When I set those values and applied, the pfSense system became unresponsive (even to pings). I eventually had to resort to hard powering it off, but it didn't come back when I turned it back on either. So I connected a monitor and was able to observe that at some point in the boot process, it began rapidly spamming the period character (.), and did so at such a rate that it was impossible to view the last boot message before this happened. If I were better versed in FreeBSD I may have known what to do to glean more useful information, but I had unhappy users so I just resorted to doing a fresh 2.5.0 installation and restoration of a config backup.

Also of note, after that config backup, I threw caution to the wind and tried to update the parameters again, but this time to limit 10240 and flows 20480. That time, which I clicked apply, the system spontaneously rebooted. It did come back, and the new values had been applied, but I don't know what happened there.

So this isn't really a support request, more just wondering if anyone else has seen any weirdness along these lines. I'm wary of adjusting these parameters any more now as well lest I need to perform a full reinstallation again. I also can't directly implicate 2.5.0 specifically here, although I believe this was the first time I changed the fq_codel params since upgrading, and I know that prior to the upgrade I had done a lot of experimentation with changing them without any issues.

Pentangle

@thenarc Not seen anything like that, but I was aware that the traffic shaping in earlier pfSense instances could play havoc with the connection if it changed for some other reason. I have recently built a v2.5.0 fresh instance and configured it with FQ_CoDel with no issues.

TheNarc

@pentangle Thanks for the input. I'd feel better had I not seen the spontaneous reset after adjusting these parameters following a fresh install; although it was a fresh install plus a config restore, so perhaps I pulled in some invalid configuration along with it. Just didn't have the stamina at the time to re-configure everything from scratch ;)

mind12

I have applied the same settings for my 150/10 Mb connection but my download speed wont move above 130Mb. Upload is fine. Checked CPU usage also during the speedtest but it's fine abou 30% utilization at all.

These are my config, similar to @Ricardox 's:
Pfsense VM with Intel NICs 2CPU 4GB RAM (about 60% utilized)
All network hardware offload off because of suricata inline mode.

DownLimiter:
147Mb, Tail Drop - FQ_CODEL (5,100,300,10240,20480), Queue 10000, ECN off
DownQueue:
Taildrop, ECN off

Any idea/tweak I could try?

Ricardox

@mind12 Installed Open-VM-Tools? For my 200/100 MB network I have no loss of speed. X86 PC!
realtek gigabit network card

mind12

@ricardox
Sure, without the limiters I get maximum speed too.

fabrizior

@mind12 @Ricardox

I think I still have a bit off tuning to do... Any recommendations?

Have Comcast 400/25 service.
Getting ~380/23 with my limiter config and bufferbloat lags of 56ms/41ms respectively, but with max download bufferbloat lag spiking up to ~230ms.

DSLReports SpeedTest (limiters on)
DSLReports SpeedTest with limiters

WANDown limiter @ 400mbit/s
Queue: CoDel, target:5 interval:100
Scheduler Config: FQ_CODEL, target:5, interval:100, quantum: 1514, limit: 5120, flows 1024, QueueLength: 1001, ECN: [checked]

WANUp limiter @ 25Mbit/s
Queue: CoDel, target:5 interval:100
Scheduler: FQ_CODEL, target:5, interval:100, quantum: 1514, limit: 10240, flows 1024, QueueLength: 1001, ECN: [checked]

EDIT: added detail with limiters disabled.
Perhaps I should just turn them off??? Am I really getting any benefit?

448/24 MBit/s and 51/67 ms bufferbloat with limiters disabled

[DSLReports SpeedTest (no limiters)]
DSLReports SpeedTest Results without limiters)

SW:
pfSense v. 2.4.5-RELEASE-p1
pfBlockerNG-devel (2.2.5_37), ntopng, bandwidthd, telegraf
Openvpn server active, no connections at time of test.

HW:
Protectli Vault FW6C
Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
Current: 2400 MHz, Max: 2601 MHz
AES-NI CPU Crypto: Yes (active)
CPU Utilization: ~5%
Memory Usage: ~17% of 8GB
Network HW Offloading: [edit] ~~disabled~~ enabled