Playing with fq_codel in 2.4

w0w

I am not sure if it's IPFW/dummynet problem, I think it's should be reported on redmine as soon as new Limiters GUI is merged with master branch.
Do you really need this floating rule?

dennypage

@strangegopher:

it was reported on here earlier by someone else: https://forum.pfsense.org/index.php?topic=126637.msg765566#msg765566
could it be issue with the way I have set up the firewall rules as floating (same as person in the post I linked to above)
If I do traceroute I see all the hops show up as the destination ip.

It's unclear that this is a problem with limiters themselves. It only happens if you use a floating rule to assign the queue. If you use the default allow rule on the interface to assign the queue, the traceroute problem does not occur.

strangegopher

Only reason I have it as floating is because I have 6 vlans. I will set it one by one in each vlan and hope this goes away.

dennypage

@strangegopher:

Only reason I have it as floating is because I have 6 vlans. I will set it one by one in each vlan and hope this goes away.

Yea, I hear you. Floating is much more convient. I have 5 local nets, although using interface specific rules I only bother with 4 of them. The last network is very low traffic and I don’t want to drop packets there anyway.

strangegopher

Just finished setting up the rules and traceroute is fixed now. :)

matt_

https://github.com/pfsense/pfsense/pull/3941

PR was accepted and needs to be tested now.

zwck

! amazing !

strangegopher

@matt_:

https://github.com/pfsense/pfsense/pull/3941

PR was accepted and needs to be tested now.

Sorry I am not on development releases but has limiter info page also been updated to show schedulers information?

like suggested earlier in this thread:

@w0w:

Patch for Limiter Info page with schedulers information and refresh interval of 500ms


--- diag_limiter_info.php	Wed Sep 07 00:26:47 2016
+++ diag_limiter_info.php	Sun Oct 01 08:20:33 2017
@@ -40,5 +40,5 @@
 	echo $text;
-	$text = `/sbin/ipfw queue show`;
+	$text = `/sbin/ipfw sched show`;
 	if ($text != "") {
-		echo "\n\n" . gettext("Queues") . ":\n";
+		echo "\n\n" . gettext("Shedulers") . ":\n";
 		echo $text;
@@ -72,3 +76,3 @@
 	events.push(function() {
-		setInterval('getlimiteractivity()', 2500);
+		setInterval('getlimiteractivity()', 500);
 		getlimiteractivity();

matt_

Not yet, that's gonna need to be a separate PR. I'm thinking of maybe making a Queue-like page for that instead of text-based readouts.

strangegopher

any idea when this will show up in 2.4 development snapshots?

areynot

Can someone please guide me how to setup the shellcmd if Im using more than 1 queue per pipe as in:
(I have placed different weights to identify where queues are placed)

pipe 1 config  bw 5000Kb
queue 1 config pipe 1 weight 10 mask src-ip6 /128 src-ip 0xffffffff
queue 2 config pipe 1 weight 20 mask src-ip6 /128 src-ip 0xffffffff
 

pipe 2 config  bw 1024Kb
queue 3 config pipe 2 weight 30 mask dst-ip6 /128 dst-ip 0xffffffff
queue 4 config pipe 2 weight 40 mask dst-ip6 /128 dst-ip 0xffffffff

using the setup above i get this when running ipfw sched show

[2.4.3-RELEASE][root@pfTest.localdomain]/root: ipfw sched show
00001:   5.000 Mbit/s    0 ms burst 0 
q00001  50 sl. 0 flows (256 buckets) sched 1 weight 10 lmax 0 pri 0 droptail
    mask:  0x00 0xffffffff/0x0000 -> 0x00000000/0x0000
 sched 1 type FQ_CODEL flags 0x0 0 buckets 0 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 2 1 
00002:   1.024 Mbit/s    0 ms burst 0 
q00002  50 sl. 0 flows (256 buckets) sched 1 weight 20 lmax 0 pri 0 droptail
    mask:  0x00 0xffffffff/0x0000 -> 0x00000000/0x0000
 sched 2 type FQ_CODEL flags 0x0 0 buckets 0 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 4 3

If you pay close attention you can see that the queues 1 and 2 from pipe 1 are assigned to sched 1 and 2... look for the weight value and youll see better.

im using this command:

ipfw sched 1 config pipe 1 type fq_codel && ipfw sched 2 config pipe 2 type fq_codel

hope you guys can help me!

cplmayo

@matt_ said in Playing with fq_codel in 2.4:

https://github.com/pfsense/pfsense/pull/3941

PR was accepted and needs to be tested now.

Could I download this to my box for testing, or to a test VM? I am super excited to have a less hacky way to implement fq_codel.

w0w

@cplmayo
Yes, you can. What is your pfSense version?

cplmayo

@w0w I am on the latest release

w0w

@cplmayo
Install System_Patches package, go to System->Patches, Press Add New Patch button and copy-paste URL https://github.com/pfsense/pfsense/compare/RELENG_2_4_3...mattund:RELENG_2_4_3.diff into URL/Commit ID field.
Also, write something into description field, press SAVE button. After that you will see you newly created patch in the list and Fetch button on the right, press it, then press appeared Test button, you should see

Patch can be applied cleanly (detail)
Patch can NOT be reverted cleanly (detail)

If you see that, then you can press Apply button and enjoy your new Limiters

SlowGrind

So after applying the patch do you just fill in the settings under limiters?

New here to traffic shaping so just looking for an easy way to decrease latency. Thanks.

w0w

@slowgrind
Generally yes, but I prefer to share bandwidth between all users evenly.
Unfortunately looks like foxale08 guide is broken, it was here but now it's completely gone :(
I'll provide some guide later, if you want it.

SlowGrind

@w0w

Thank you and I just have a simple network setup. 1 wan and 1 lan with 1000 Mbps download and 50 Mbps upload.

mattund

I'm testing my patch, and I am seeing some unexpected behavior (maybe intentional).

pipe 2 config  bw 122800Kb queue 100 buckets 256 droptail
sched 2 config pipe 2 type fq_codel target 35ms interval 35ms quantum 1514 limit 10240 flows 1024 noecn
queue 1 config pipe 2 queue 100 droptail

pipe 1 config  bw 12000Kb queue 100 droptail
sched 1 config pipe 1 type fq_codel target 5ms interval 10ms quantum 1514 limit 10240 flows 1024 noecn
queue 2 config pipe 1 queue 100 droptail

Using the above generated ruleset, if I start a speedtest, and have an ICMP ping up at the same time, upon reaching the ceiling of ~120Mbps, pings drop completely. Not bufferbloat, but full drops. Now, of course that would happen, the limiter is supposed to limit the bandwidth. However, is there a way around this? I figured burst might help, but it doesn't appear to -- FQ_CoDel assumes the new burst value as part of the total bandwidth share and doesn't kick in.

It's almost as though FQ_CoDel isn't active, and the limiter is putting itself into the drop state before CoDel can say, "hey, this teeny tiny little ICMP packet isn't part of a busy flow, so let's let that through". Unless I am mistaken on something.

What I want:

Small flow traffic to not get dropped (think my little ping)
Busy flow traffic to be shaped (dropped) appropriately
Not allow the limiter pipe drop packets before FQ_CoDel has a chance to process them (?)

I was under the impression FQ_CoDel was designed around those objectives. Am I doing this right?

cplmayo

@mattund said in Playing with fq_codel in 2.4:

I'm testing my patch, and I am seeing some unexpected behavior (maybe intentional).
pipe 2 config  bw 122800Kb queue 100 buckets 256 droptail
sched 2 config pipe 2 type fq_codel target 35ms interval 35ms quantum 1514 limit 10240 flows 1024 noecn
queue 1 config pipe 2 queue 100 droptail

pipe 1 config  bw 12000Kb queue 100 droptail
sched 1 config pipe 1 type fq_codel target 5ms interval 10ms quantum 1514 limit 10240 flows 1024 noecn
queue 2 config pipe 1 queue 100 droptail
Using the above generated ruleset, if I start a speedtest, and have an ICMP ping up at the same time, upon reaching the ceiling of ~120Mbps, pings drop completely. Not bufferbloat, but full drops. Now, of course that would happen, the limiter is supposed to limit the bandwidth. However, is there a way around this? I figured burst might help, but it doesn't appear to -- FQ_CoDel assumes the new burst value as part of the total bandwidth share and doesn't kick in.

It's almost as though FQ_CoDel isn't active, and the limiter is putting itself into the drop state before CoDel can say, "hey, this teeny tiny little ICMP packet isn't part of a busy flow, so let's let that through". Unless I am mistaken on something.

Am I doing this right?

I applied your patch to my install and so far it is working as intended. I haven't noticed any huge difference between it and manual approach I have been using for months.

I did noticed that when I tried setting weights for different queues and if I set the scheduler in the queues no traffic would pass. I went back and reread your instructions on GitHub and once I removed the weights and scheduler options from the queue it works.

I am still doing some testing to see results in dslreports but so far your interface is simple and just seems to work.

Great job!