CoDel does NOT work on limiter queues in 2.4.4?

uptownVagrant

It appears that limiter queues are using the default queue size (50) instead of CoDel managing the queue size. Functionally, based on observed latency, it appears that TailDrop is managing the queue and queue size is being used.

Example configuration:

Outgoing NAT is not enabled in pfSense
I create two 50Mbps limiters with the TailDrop AQM and RR scheduler assigned to the pipes.
I create two queues per limiter with CoDel AQM selected - one queue with a weight of (2) and one with a weight of (8).
I use floating WAN rules to match two clients used to test the limiters.

1.) Create "Out" limiter

Tick Enable
Name: WAN_OUT_RR
Bandwidth: 50 Mbit/s
Queue Management Algorithm: Tail Drop
Scheduler: Round Robin
Save/Apply Changes

2.) Add first "Out" queue

Tick "Enable"
Name: wan_out_cq2
Queue Management Algorithm: CoDel
Weight: 2
Save/Apply Changes

3.) Add second "Out" queue

Tick "Enable"
Name: wan_out_cq8
Queue Management Algorithm: CoDel
Weight: 8
Save/Apply Changes

4.) Create "In" limiter

Tick "Enable"
Name: WAN_IN_RR
Bandwidth: 50 Mbit/s
Queue Management Algorithm: Tail Drop
Scheduler: Round Robin
Save/Apply Changes

5.) Add first "In" queue

Tick "Enable"
Name: wan_in_cq2
Queue Management Algorithm: CoDel
Save/Apply Changes

6.) Add second "In" queue

Tick "Enable"
Name: wan_in_cq8
Queue Management Algorithm: CoDel
Save/Apply Changes

Two Ubuntu 16.04 clients are used to test the weighted queues using Flent. Client names are netperf2 [192.168.2.9] and netperf3 [192.168.2.8]. One Netperf server is used on the WAN side. Here is how I create the floating rules:

1.) Add "Out" limiter in floating firewall rule for netperf2

Action: Match
Interface: WAN
Direction: out
Address Family: IPv4
Protocol: Any
Source: 192.168.2.9
Destination: any
Description: netperf2 out limiter
Gateway: WANGW
In / Out pipe: wan_out_cq2 / wan_in_cq2

2.) Add "Out" limiter in floating firewall rule for netperf3

Action: Match
Interface: WAN
Direction: out
Address Family: IPv4
Protocol: Any
Source: 192.168.2.8
Destination: any
Description: netperf3 out limiter
Gateway: WANGW
In / Out pipe: wan_out_cq8 / wan_in_cq8

3.) Add "In" limiter in floating firewall rule for netperf2

Action: Match
Interface: WAN
Direction: in
Address Family: IPv4
Protocol: Any
Source: any
Destination: 192.168.2.9
Description: netperf2 in limiter
Gateway: Default
In / Out pipe: wan_in_cq2 / wan_out_cq2

4.) Add "In" limiter in floating firewall rule for netperf3

Action: Match
Interface: WAN
Direction: in
Address Family: IPv4
Protocol: Any
Source: any
Destination: 192.168.2.8
Description: netperf3 in limiter
Gateway: Default
In / Out pipe: wan_in_cq8 / wan_out_cq8

As you will see in the graph, the weights are being applied but the latency is much higher than it should be under load. I would expect latency to be around 5ms.

Here is a graph where CoDel is still enabled on the queues, but I have changed the queue size to (4) for the weight=2 queues and (16) for the queues with weight=8:

(Below is all of the various configuration output when queue size is not configured)

Contents of /tmp/rules.limiter:

pipe 1 config  bw 50Mb droptail
sched 1 config pipe 1 type rr
queue 1 config pipe 1 weight 2 codel target 5ms interval 100ms noecn
queue 2 config pipe 1 weight 8 codel target 5ms interval 100ms noecn
 

pipe 2 config  bw 50Mb droptail
sched 2 config pipe 2 type rr
queue 3 config pipe 2 weight 2 codel target 5ms interval 100ms noecn
queue 4 config pipe 2 weight 8 codel target 5ms interval 100ms noecn

ipfw limiter output:

[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw pipe show
00001:  50.000 Mbit/s    0 ms burst 0
q131073  50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
 sched 65537 type FIFO flags 0x0 0 buckets 0 active
00002:  50.000 Mbit/s    0 ms burst 0
q131074  50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail
 sched 65538 type FIFO flags 0x0 0 buckets 0 active
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw sched show
00001:  50.000 Mbit/s    0 ms burst 0
 sched 1 type RR flags 0x0 0 buckets 0 active
   Children flowsets: 2 1
00002:  50.000 Mbit/s    0 ms burst 0
 sched 2 type RR flags 0x0 0 buckets 0 active
   Children flowsets: 4 3
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw queue show
q00001  50 sl. 0 flows (1 buckets) sched 1 weight 2 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q00002  50 sl. 0 flows (1 buckets) sched 1 weight 8 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q00003  50 sl. 0 flows (1 buckets) sched 2 weight 2 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
q00004  50 sl. 0 flows (1 buckets) sched 2 weight 8 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:

Applicable floating firewall rules:

[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: pfctl -vvsr | grep "netperf"
@84(1542770685) match out on igb0 inet from 192.168.2.9 to any label "USER_RULE: netperf2 out limiter" dnqueue(1, 3)
@85(1542770664) match out on igb0 inet from 192.168.2.8 to any label "USER_RULE: netperf3 out limiter" dnqueue(2, 4)
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:

Limiter dnshaper configuration found in xml:
0_1542830252362_dnshaper_RR_CoDel_weighted.xml