Wrong limiter speed

[bobbenheim]

@Jimbohello what are your limiter settings?

[Jimbohello]

@bobbenheim
<dnshaper>
<queue>
<name>in</name>
<number>1</number>
<qlimit></qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>wf2q+</sched>
<aqm>droptail</aqm>
<ecn></ecn>
<queue>
<name>in-queue</name>
<number>1</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>dstaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>droptail</aqm>
<ecn></ecn>
</queue>
</queue>
<queue>
<name>out</name>
<number>2</number>
<qlimit></qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>wf2q+</sched>
<aqm>droptail</aqm>
<ecn></ecn>
<queue>
<name>out-queue</name>
<number>2</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>srcaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>droptail</aqm>
<ecn></ecn>
</queue>
</queue>
</dnshaper>

[Stewart]

@noplan Certainly we can make it work. It's just that the numbers don't really line up. Oh, and fast.com goes crazy when put through one of the limiters. Through the ones we set up last week fast.com couldn't get more than 120Mbps no matter what the limiter was set to. Without the limiter it would get the full 600+Mbps. I thought that was odd. Speedtest.net is the one that would increase, just at a different rate to what the limiter was being set at.

[bobbenheim]

@Jimbohello Can you try enabling ECN in the limiter (not the queues) and see if that makes a difference, else you might also need to increase queue length from the default 50 to something larger.
Have you made any tweaks in loader.conf.local?

[Jimbohello]

@bobbenheim

what tweak ?

in loader.conf.local

[Jimbohello]

@bobbenheim
Explicit Congestion Notification is selected, but neither Tail Drop nor Worst-case Weighted fair Queueing (default) support it.

[Jimbohello]

@bobbenheim

this give a better result on lowering de error in the accuracy
from a limit of 799
i can reach 735

before it was close to 500 !
we are on the right way ! :)

<dnshaper>
<queue>
<name>in</name>
<number>1</number>
<qlimit>100</qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>fq_codel</sched>
<aqm>codel</aqm>
<ecn>on</ecn>
<queue>
<name>in-queue</name>
<number>1</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>dstaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>codel</aqm>
<ecn></ecn>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<param_fq_codel_target>5</param_fq_codel_target>
<param_fq_codel_interval>100</param_fq_codel_interval>
<param_fq_codel_quantum>1514</param_fq_codel_quantum>
<param_fq_codel_limit>10240</param_fq_codel_limit>
<param_fq_codel_flows>1024</param_fq_codel_flows>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<queue>
<name>out</name>
<number>2</number>
<qlimit>100</qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>fq_codel</sched>
<aqm>codel</aqm>
<ecn>on</ecn>
<queue>
<name>out-queue</name>
<number>2</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>srcaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>codel</aqm>
<ecn></ecn>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<param_fq_codel_target>5</param_fq_codel_target>
<param_fq_codel_interval>100</param_fq_codel_interval>
<param_fq_codel_quantum>1514</param_fq_codel_quantum>
<param_fq_codel_limit>10240</param_fq_codel_limit>
<param_fq_codel_flows>1024</param_fq_codel_flows>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
</dnshaper>

[bobbenheim]

@Jimbohello can you run "top -aSH" in Diagnostics/Command Prompt and paste what it output under load?
There might also be some tweaks you can do in loader.conf.local
can you do a sysctl (e.g. "sysctl hw.em.enable_aim") in the command prompt for the following:
hw.em.enable_aim
hw.em.flow_control
hw.em.num_queues
hw.em.rx_process_limit
hw.em.tx_process_limit
hw.em.rxd
hw.em.txd
hw.em.max_interrupt_rate
net.link.ifqmaxlen

[Jimbohello]

@bobbenheim

il get in touch as soon as i can do it ! but honestly
this is getting far deeper for something that is created on a simple click. i means creating a limiter suppose to be easy as 123.
IMO this is getthing ridiculous ! if so many thing need to be modify on a single limiter limitation, why when your press APPLY they don’t run a script to set the right adjustement ! imagine you need to forward a port, and then you have to modify this and that TO MAKE IT WORK ! That will be irrelevent or if you prefer useless function !

anyway ill get back to you
youre help is please !

[noplan]

@Jimbohello said in Wrong limiter speed:

but honestly
this is getting far deeper for something that is created on a simple click

yes true and 4 WHAT

cuz the limiter does what it is supposed to do !
here and on some other boxes

to be honest i can not see the use case for gettin as deep as shown here
an any set up

maybe u can help me out here explainin whats the urgent need

[Jimbohello]

@noplan

nothing urgent bro !
simple !

let say your 50 on a network, 5 start a download !
the gateway goes down and packetloss occur on high level !

with the limiter nothing of that appends because gateway still have room to breathed and nothing goes down on the gateway.

that the big advantage of the limiter.

[Stewart]

For us, we were setting it up in a church. They stream several services at the same time. To accommodate this we brought in a second WAN and route specific devices out that interface. To prevent each machine from taking up too much of the bandwidth and affecting the other streams we limit their max usage. In this case it is a 35Mbps upload split between 4 devices, each with a 10Mbps upload limit but only needing 5Mbps for the streams. For example, there are 4 streams going and someone decides to (unwisely and against policy) upload a previous stream to the platform from one of the streaming boxes. Normally that would use up all possible bandwidth but with a limiter in place that particular device is unable to affect the other streams. If the full 10Mbps is used then there is still 25Mbps remaining for the other 3 devices. If each of those 3 devices is using 5Mbps then they are only using 15Mbps out of the 25Mbps, leaving a cushion of 10Mbps. Technically, in this case, 3 users could be offending and the 4th stream should still run fine. Without the limiter just one bad actor could ruin it for everyone else.

[Stewart]

@Jimbohello said in Wrong limiter speed:

on

Could you post this as a screenshot, cropped to just the config section? I'm having a hard time following and duplicating. Thanks.

[bobbenheim]

@Jimbohello The problem is that traffic shaping isn't a one size fits all kinda setup. Default settings is more likely to be something that works with bandwidths likely to be 100 Mb/s or below because that is what general use cases were, when the settings were chosen. These settings doesn't take other factors, like different types of hardware setup used or what type of internet connection is used (COAX, xDSL, PTP fiber, GPON fiber), into consideration.
This also means that the above 900 Mb/s, like in your case, doesn't show the expected performance because the default settings just doesn't work well for that kinda of bandwidth. So this means that tuning is necessary to either obtain the maximum performance which is possible or what is expected as set by the limiter. Once done tuning you can leave it alone and don't bother with it again.

[Jimbohello]

@bobbenheim

thank’s for the explaination!
verry appreciate

[Jimbohello]

@bobbenheim said in Wrong limiter speed:

top -aSH - speddtest.net

last pid: 33530; load averages: 0.07, 0.06, 0.01 up 19+18:28:36 12:05:35
181 processes: 5 running, 149 sleeping, 27 waiting

Mem: 11M Active, 135M Inact, 189M Wired, 19M Buf, 7474M Free
Swap: 3979M Total, 3979M Free

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 64K CPU1 1 474.2H 100.00% [idle{idle: cpu1}]
11 root 155 ki31 0K 64K CPU3 3 474.2H 100.00% [idle{idle: cpu3}]
11 root 155 ki31 0K 64K CPU2 2 474.2H 100.00% [idle{idle: cpu2}]
11 root 155 ki31 0K 64K RUN 0 468.6H 100.00% [idle{idle: cpu0}]
12 root -92 - 0K 432K WAIT 2 1:48 4.59% [intr{irq269: em1:rx0}]
12 root -92 - 0K 432K WAIT 0 2:44 3.96% [intr{irq264: em0:rx0}]
346 root 26 0 99704K 39460K piperd 3 0:06 1.17% php-fpm: pool nginx (php-fpm){php-fpm}
12 root -92 - 0K 432K WAIT 1 0:11 0.20% [intr{irq265: em0:tx0}]
347 root 52 0 95092K 36704K accept 3 0:05 0.20% php-fpm: pool nginx (php-fpm)
0 root -92 - 0K 624K - 2 334:57 0.00% [kernel{dummynet}]
12 root -60 - 0K 432K WAIT 1 10:59 0.00% [intr{swi4: clock (0)}]
23 root -16 - 0K 16K - 2 1:02 0.00% [rand_harvestq]
84645 root 52 20 6976K 2752K wait 2 1:00 0.00% /bin/sh /var/db/rrd/updaterrd.sh
20 root -16 - 0K 16K pftm 2 0:41 0.00% [pf purge]
46951 unbound 20 0 60368K 37608K kqread 3 0:28 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
28775 root 20 0 6904K 2340K nanslp 2 0:23 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_PPPOE -B 65.94.26.33 -p /var/run/dpinger_WAN_PPPOE~65.94.26.33~10.11.16.17.pid -u /var/ru
0 root -16 - 0K 624K swapin 1 0:20 0.00% [kernel{swapper}]
70389 root 20 0 12464K 5760K select 3 0:16 0.00% /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid{ntpd}

sysctl: unknown oid 'hw.em.enable_aim'
sysctl: unknown oid hw.em.flow_control
sysctl: unknown oid hw.em.tx_process_limit
hw.em.rx_process_limit: 100
hw.em.rx_process_limit: 100
hw.em.rxd: 1024
hw.em.txd: 1024
sysctl: unknown oid 'hw.em.max_interrupt_rate'
net.link.ifqmaxlen: 128

[Jimbohello]

@Stewart in.zip in-queue.zip

same for the out queue except sources/adress

[bobbenheim]

@Jimbohello can you please run "sysctl hw.em" and "sysctl dev.em" in the command prompt and paste the result.

[Jimbohello]

@bobbenheim said in Wrong limiter speed:

sysctl hw.em

hw.em.eee_setting: 1
hw.em.rx_process_limit: 100
hw.em.enable_msix: 1
hw.em.sbp: 0
hw.em.smart_pwr_down: 0
hw.em.txd: 1024
hw.em.rxd: 1024
hw.em.rx_abs_int_delay: 66
hw.em.tx_abs_int_delay: 66
hw.em.rx_int_delay: 0
hw.em.tx_int_delay: 66
hw.em.disable_crc_stripping: 0

[noplan]

OK
I m Readin and learning

A Lot
Thank you!!!