CoDel does NOT work on limiter queues in 2.4.4?



  • It appears that limiter queues are using the default queue size (50) instead of CoDel managing the queue size. Functionally, based on observed latency, it appears that TailDrop is managing the queue and queue size is being used.

    Example configuration:

    • Outgoing NAT is not enabled in pfSense
    • I create two 50Mbps limiters with the TailDrop AQM and RR scheduler assigned to the pipes.
    • I create two queues per limiter with CoDel AQM selected - one queue with a weight of (2) and one with a weight of (8).
    • I use floating WAN rules to match two clients used to test the limiters.

    1.) Create "Out" limiter

    • Tick Enable
    • Name: WAN_OUT_RR
    • Bandwidth: 50 Mbit/s
    • Queue Management Algorithm: Tail Drop
    • Scheduler: Round Robin
    • Save/Apply Changes

    2.) Add first "Out" queue

    • Tick "Enable"
    • Name: wan_out_cq2
    • Queue Management Algorithm: CoDel
    • Weight: 2
    • Save/Apply Changes

    3.) Add second "Out" queue

    • Tick "Enable"
    • Name: wan_out_cq8
    • Queue Management Algorithm: CoDel
    • Weight: 8
    • Save/Apply Changes

    4.) Create "In" limiter

    • Tick "Enable"
    • Name: WAN_IN_RR
    • Bandwidth: 50 Mbit/s
    • Queue Management Algorithm: Tail Drop
    • Scheduler: Round Robin
    • Save/Apply Changes

    5.) Add first "In" queue

    • Tick "Enable"
    • Name: wan_in_cq2
    • Queue Management Algorithm: CoDel
    • Save/Apply Changes

    6.) Add second "In" queue

    • Tick "Enable"
    • Name: wan_in_cq8
    • Queue Management Algorithm: CoDel
    • Save/Apply Changes

    Two Ubuntu 16.04 clients are used to test the weighted queues using Flent. Client names are netperf2 [192.168.2.9] and netperf3 [192.168.2.8]. One Netperf server is used on the WAN side. Here is how I create the floating rules:

    1.) Add "Out" limiter in floating firewall rule for netperf2

    • Action: Match
    • Interface: WAN
    • Direction: out
    • Address Family: IPv4
    • Protocol: Any
    • Source: 192.168.2.9
    • Destination: any
    • Description: netperf2 out limiter
    • Gateway: WANGW
    • In / Out pipe: wan_out_cq2 / wan_in_cq2

    2.) Add "Out" limiter in floating firewall rule for netperf3

    • Action: Match
    • Interface: WAN
    • Direction: out
    • Address Family: IPv4
    • Protocol: Any
    • Source: 192.168.2.8
    • Destination: any
    • Description: netperf3 out limiter
    • Gateway: WANGW
    • In / Out pipe: wan_out_cq8 / wan_in_cq8

    3.) Add "In" limiter in floating firewall rule for netperf2

    • Action: Match
    • Interface: WAN
    • Direction: in
    • Address Family: IPv4
    • Protocol: Any
    • Source: any
    • Destination: 192.168.2.9
    • Description: netperf2 in limiter
    • Gateway: Default
    • In / Out pipe: wan_in_cq2 / wan_out_cq2

    4.) Add "In" limiter in floating firewall rule for netperf3

    • Action: Match
    • Interface: WAN
    • Direction: in
    • Address Family: IPv4
    • Protocol: Any
    • Source: any
    • Destination: 192.168.2.8
    • Description: netperf3 in limiter
    • Gateway: Default
    • In / Out pipe: wan_in_cq8 / wan_out_cq8

    As you will see in the graph, the weights are being applied but the latency is much higher than it should be under load. I would expect latency to be around 5ms.
    0_1542837923584_RR_CoDel_weight_2_8_qsize50.jpg

    Here is a graph where CoDel is still enabled on the queues, but I have changed the queue size to (4) for the weight=2 queues and (16) for the queues with weight=8:
    0_1542837952943_RR_CoDel_weight_2_8_qsize4_16.jpg

    (Below is all of the various configuration output when queue size is not configured)

    Contents of /tmp/rules.limiter:

    pipe 1 config  bw 50Mb droptail
    sched 1 config pipe 1 type rr
    queue 1 config pipe 1 weight 2 codel target 5ms interval 100ms noecn
    queue 2 config pipe 1 weight 8 codel target 5ms interval 100ms noecn
     
    
    pipe 2 config  bw 50Mb droptail
    sched 2 config pipe 2 type rr
    queue 3 config pipe 2 weight 2 codel target 5ms interval 100ms noecn
    queue 4 config pipe 2 weight 8 codel target 5ms interval 100ms noecn
    

    ipfw limiter output:

    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw pipe show
    00001:  50.000 Mbit/s    0 ms burst 0
    q131073  50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 droptail
     sched 65537 type FIFO flags 0x0 0 buckets 0 active
    00002:  50.000 Mbit/s    0 ms burst 0
    q131074  50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail
     sched 65538 type FIFO flags 0x0 0 buckets 0 active
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw sched show
    00001:  50.000 Mbit/s    0 ms burst 0
     sched 1 type RR flags 0x0 0 buckets 0 active
       Children flowsets: 2 1
    00002:  50.000 Mbit/s    0 ms burst 0
     sched 2 type RR flags 0x0 0 buckets 0 active
       Children flowsets: 4 3
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: ipfw queue show
    q00001  50 sl. 0 flows (1 buckets) sched 1 weight 2 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
    q00002  50 sl. 0 flows (1 buckets) sched 1 weight 8 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
    q00003  50 sl. 0 flows (1 buckets) sched 2 weight 2 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
    q00004  50 sl. 0 flows (1 buckets) sched 2 weight 8 lmax 1500 pri 0  AQM CoDel target 5ms interval 100ms NoECN
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
    

    Applicable floating firewall rules:

    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root: pfctl -vvsr | grep "netperf"
    @84(1542770685) match out on igb0 inet from 192.168.2.9 to any label "USER_RULE: netperf2 out limiter" dnqueue(1, 3)
    @85(1542770664) match out on igb0 inet from 192.168.2.8 to any label "USER_RULE: netperf3 out limiter" dnqueue(2, 4)
    [2.4.4-RELEASE][admin@pfSense-dev.localdomain]/root:
    
    

    Limiter dnshaper configuration found in xml:
    0_1542830252362_dnshaper_RR_CoDel_weighted.xml