CoDel does NOT work on limiter queues in 2.4.4?
-
It appears that limiter queues are using the default queue size (50) instead of CoDel managing the queue size. Functionally, based on observed latency, it appears that TailDrop is managing the queue and queue size is being used.
Example configuration:
- Outgoing NAT is not enabled in pfSense
- I create two 50Mbps limiters with the TailDrop AQM and RR scheduler assigned to the pipes.
- I create two queues per limiter with CoDel AQM selected - one queue with a weight of (2) and one with a weight of (8).
- I use floating WAN rules to match two clients used to test the limiters.
1.) Create "Out" limiter
- Tick Enable
- Name: WAN_OUT_RR
- Bandwidth: 50 Mbit/s
- Queue Management Algorithm: Tail Drop
- Scheduler: Round Robin
- Save/Apply Changes
2.) Add first "Out" queue
- Tick "Enable"
- Name: wan_out_cq2
- Queue Management Algorithm: CoDel
- Weight: 2
- Save/Apply Changes
3.) Add second "Out" queue
- Tick "Enable"
- Name: wan_out_cq8
- Queue Management Algorithm: CoDel
- Weight: 8
- Save/Apply Changes
4.) Create "In" limiter
- Tick "Enable"
- Name: WAN_IN_RR
- Bandwidth: 50 Mbit/s
- Queue Management Algorithm: Tail Drop
- Scheduler: Round Robin
- Save/Apply Changes
5.) Add first "In" queue
- Tick "Enable"
- Name: wan_in_cq2
- Queue Management Algorithm: CoDel
- Save/Apply Changes
6.) Add second "In" queue
- Tick "Enable"
- Name: wan_in_cq8
- Queue Management Algorithm: CoDel
- Save/Apply Changes
Two Ubuntu 16.04 clients are used to test the weighted queues using Flent. Client names are netperf2 [192.168.2.9] and netperf3 [192.168.2.8]. One Netperf server is used on the WAN side. Here is how I create the floating rules:
1.) Add "Out" limiter in floating firewall rule for netperf2
- Action: Match
- Interface: WAN
- Direction: out
- Address Family: IPv4
- Protocol: Any
- Source: 192.168.2.9
- Destination: any
- Description: netperf2 out limiter
- Gateway: WANGW
- In / Out pipe: wan_out_cq2 / wan_in_cq2
2.) Add "Out" limiter in floating firewall rule for netperf3
- Action: Match
- Interface: WAN
- Direction: out
- Address Family: IPv4
- Protocol: Any
- Source: 192.168.2.8
- Destination: any
- Description: netperf3 out limiter
- Gateway: WANGW
- In / Out pipe: wan_out_cq8 / wan_in_cq8
3.) Add "In" limiter in floating firewall rule for netperf2
- Action: Match
- Interface: WAN
- Direction: in
- Address Family: IPv4
- Protocol: Any
- Source: any
- Destination: 192.168.2.9
- Description: netperf2 in limiter
- Gateway: Default
- In / Out pipe: wan_in_cq2 / wan_out_cq2
4.) Add "In" limiter in floating firewall rule for netperf3
- Action: Match
- Interface: WAN
- Direction: in
- Address Family: IPv4
- Protocol: Any
- Source: any
- Destination: 192.168.2.8
- Description: netperf3 in limiter
- Gateway: Default
- In / Out pipe: wan_in_cq8 / wan_out_cq8
As you will see in the graph, the weights are being applied but the latency is much higher than it should be under load. I would expect latency to be around 5ms.
Here is a graph where CoDel is still enabled on the queues, but I have changed the queue size to (4) for the weight=2 queues and (16) for the queues with weight=8:
(Below is all of the various configuration output when queue size is not configured)
Contents of /tmp/rules.limiter:
pipe 1 config bw 50Mb droptail sched 1 config pipe 1 type rr queue 1 config pipe 1 weight 2 codel target 5ms interval 100ms noecn queue 2 config pipe 1 weight 8 codel target 5ms interval 100ms noecn pipe 2 config bw 50Mb droptail sched 2 config pipe 2 type rr queue 3 config pipe 2 weight 2 codel target 5ms interval 100ms noecn queue 4 config pipe 2 weight 8 codel target 5ms interval 100ms noecn
ipfw limiter output:
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw pipe show 00001: 50.000 Mbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail sched 65537 type FIFO flags 0x0 0 buckets 0 active 00002: 50.000 Mbit/s 0 ms burst 0 q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail sched 65538 type FIFO flags 0x0 0 buckets 0 active [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw sched show 00001: 50.000 Mbit/s 0 ms burst 0 sched 1 type RR flags 0x0 0 buckets 0 active Children flowsets: 2 1 00002: 50.000 Mbit/s 0 ms burst 0 sched 2 type RR flags 0x0 0 buckets 0 active Children flowsets: 4 3 [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw queue show q00001 50 sl. 0 flows (1 buckets) sched 1 weight 2 lmax 1500 pri 0 AQM CoDel target 5ms interval 100ms NoECN q00002 50 sl. 0 flows (1 buckets) sched 1 weight 8 lmax 1500 pri 0 AQM CoDel target 5ms interval 100ms NoECN q00003 50 sl. 0 flows (1 buckets) sched 2 weight 2 lmax 1500 pri 0 AQM CoDel target 5ms interval 100ms NoECN q00004 50 sl. 0 flows (1 buckets) sched 2 weight 8 lmax 1500 pri 0 AQM CoDel target 5ms interval 100ms NoECN [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
Applicable floating firewall rules:
[2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: pfctl -vvsr | grep "netperf" @84(1542770685) match out on igb0 inet from 192.168.2.9 to any label "USER_RULE: netperf2 out limiter" dnqueue(1, 3) @85(1542770664) match out on igb0 inet from 192.168.2.8 to any label "USER_RULE: netperf3 out limiter" dnqueue(2, 4) [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
Limiter dnshaper configuration found in xml:
0_1542830252362_dnshaper_RR_CoDel_weighted.xml