Wrong limiter speed

Jimbohello

what tweak ?

in loader.conf.local

Jimbohello

@bobbenheim
Explicit Congestion Notification is selected, but neither Tail Drop nor Worst-case Weighted fair Queueing (default) support it.

Jimbohello

@bobbenheim

this give a better result on lowering de error in the accuracy
from a limit of 799
i can reach 735

before it was close to 500 !
we are on the right way ! :)

<dnshaper>
<queue>
<name>in</name>
<number>1</number>
<qlimit>100</qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>fq_codel</sched>
<aqm>codel</aqm>
<ecn>on</ecn>
<queue>
<name>in-queue</name>
<number>1</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>dstaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>codel</aqm>
<ecn></ecn>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<param_fq_codel_target>5</param_fq_codel_target>
<param_fq_codel_interval>100</param_fq_codel_interval>
<param_fq_codel_quantum>1514</param_fq_codel_quantum>
<param_fq_codel_limit>10240</param_fq_codel_limit>
<param_fq_codel_flows>1024</param_fq_codel_flows>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<queue>
<name>out</name>
<number>2</number>
<qlimit>100</qlimit>
<plr></plr>
<description></description>
<bandwidth>
<item>
<bw>799</bw>
<burst></burst>
<bwscale>Mb</bwscale>
<bwsched>none</bwsched>
</item>
</bandwidth>
<enabled>on</enabled>
<buckets></buckets>
<mask>none</mask>
<maskbits></maskbits>
<maskbitsv6></maskbitsv6>
<delay>0</delay>
<sched>fq_codel</sched>
<aqm>codel</aqm>
<ecn>on</ecn>
<queue>
<name>out-queue</name>
<number>2</number>
<qlimit></qlimit>
<description></description>
<weight></weight>
<enabled>on</enabled>
<buckets></buckets>
<mask>srcaddress</mask>
<maskbits>32</maskbits>
<maskbitsv6>128</maskbitsv6>
<aqm>codel</aqm>
<ecn></ecn>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
<param_fq_codel_target>5</param_fq_codel_target>
<param_fq_codel_interval>100</param_fq_codel_interval>
<param_fq_codel_quantum>1514</param_fq_codel_quantum>
<param_fq_codel_limit>10240</param_fq_codel_limit>
<param_fq_codel_flows>1024</param_fq_codel_flows>
<param_codel_target>5</param_codel_target>
<param_codel_interval>100</param_codel_interval>
</queue>
</dnshaper>

bobbenheim

@Jimbohello can you run "top -aSH" in Diagnostics/Command Prompt and paste what it output under load?
There might also be some tweaks you can do in loader.conf.local
can you do a sysctl (e.g. "sysctl hw.em.enable_aim") in the command prompt for the following:
hw.em.enable_aim
hw.em.flow_control
hw.em.num_queues
hw.em.rx_process_limit
hw.em.tx_process_limit
hw.em.rxd
hw.em.txd
hw.em.max_interrupt_rate
net.link.ifqmaxlen

Jimbohello

@bobbenheim

il get in touch as soon as i can do it ! but honestly
this is getting far deeper for something that is created on a simple click. i means creating a limiter suppose to be easy as 123.
IMO this is getthing ridiculous ! if so many thing need to be modify on a single limiter limitation, why when your press APPLY they don’t run a script to set the right adjustement ! imagine you need to forward a port, and then you have to modify this and that TO MAKE IT WORK ! That will be irrelevent or if you prefer useless function !

anyway ill get back to you
youre help is please !

noplan

@Jimbohello said in Wrong limiter speed:

but honestly
this is getting far deeper for something that is created on a simple click

yes true and 4 WHAT

cuz the limiter does what it is supposed to do !
here and on some other boxes

to be honest i can not see the use case for gettin as deep as shown here
an any set up

maybe u can help me out here explainin whats the urgent need

Jimbohello

@noplan

nothing urgent bro !
simple !

let say your 50 on a network, 5 start a download !
the gateway goes down and packetloss occur on high level !

with the limiter nothing of that appends because gateway still have room to breathed and nothing goes down on the gateway.

that the big advantage of the limiter.

Stewart

For us, we were setting it up in a church. They stream several services at the same time. To accommodate this we brought in a second WAN and route specific devices out that interface. To prevent each machine from taking up too much of the bandwidth and affecting the other streams we limit their max usage. In this case it is a 35Mbps upload split between 4 devices, each with a 10Mbps upload limit but only needing 5Mbps for the streams. For example, there are 4 streams going and someone decides to (unwisely and against policy) upload a previous stream to the platform from one of the streaming boxes. Normally that would use up all possible bandwidth but with a limiter in place that particular device is unable to affect the other streams. If the full 10Mbps is used then there is still 25Mbps remaining for the other 3 devices. If each of those 3 devices is using 5Mbps then they are only using 15Mbps out of the 25Mbps, leaving a cushion of 10Mbps. Technically, in this case, 3 users could be offending and the 4th stream should still run fine. Without the limiter just one bad actor could ruin it for everyone else.

Stewart

@Jimbohello said in Wrong limiter speed:

on

Could you post this as a screenshot, cropped to just the config section? I'm having a hard time following and duplicating. Thanks.

bobbenheim

@Jimbohello The problem is that traffic shaping isn't a one size fits all kinda setup. Default settings is more likely to be something that works with bandwidths likely to be 100 Mb/s or below because that is what general use cases were, when the settings were chosen. These settings doesn't take other factors, like different types of hardware setup used or what type of internet connection is used (COAX, xDSL, PTP fiber, GPON fiber), into consideration.
This also means that the above 900 Mb/s, like in your case, doesn't show the expected performance because the default settings just doesn't work well for that kinda of bandwidth. So this means that tuning is necessary to either obtain the maximum performance which is possible or what is expected as set by the limiter. Once done tuning you can leave it alone and don't bother with it again.

Jimbohello

@bobbenheim

thank’s for the explaination!
verry appreciate

Jimbohello

@bobbenheim said in Wrong limiter speed:

top -aSH - speddtest.net

last pid: 33530; load averages: 0.07, 0.06, 0.01 up 19+18:28:36 12:05:35
181 processes: 5 running, 149 sleeping, 27 waiting

Mem: 11M Active, 135M Inact, 189M Wired, 19M Buf, 7474M Free
Swap: 3979M Total, 3979M Free

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 64K CPU1 1 474.2H 100.00% [idle{idle: cpu1}]
11 root 155 ki31 0K 64K CPU3 3 474.2H 100.00% [idle{idle: cpu3}]
11 root 155 ki31 0K 64K CPU2 2 474.2H 100.00% [idle{idle: cpu2}]
11 root 155 ki31 0K 64K RUN 0 468.6H 100.00% [idle{idle: cpu0}]
12 root -92 - 0K 432K WAIT 2 1:48 4.59% [intr{irq269: em1:rx0}]
12 root -92 - 0K 432K WAIT 0 2:44 3.96% [intr{irq264: em0:rx0}]
346 root 26 0 99704K 39460K piperd 3 0:06 1.17% php-fpm: pool nginx (php-fpm){php-fpm}
12 root -92 - 0K 432K WAIT 1 0:11 0.20% [intr{irq265: em0:tx0}]
347 root 52 0 95092K 36704K accept 3 0:05 0.20% php-fpm: pool nginx (php-fpm)
0 root -92 - 0K 624K - 2 334:57 0.00% [kernel{dummynet}]
12 root -60 - 0K 432K WAIT 1 10:59 0.00% [intr{swi4: clock (0)}]
23 root -16 - 0K 16K - 2 1:02 0.00% [rand_harvestq]
84645 root 52 20 6976K 2752K wait 2 1:00 0.00% /bin/sh /var/db/rrd/updaterrd.sh
20 root -16 - 0K 16K pftm 2 0:41 0.00% [pf purge]
46951 unbound 20 0 60368K 37608K kqread 3 0:28 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
28775 root 20 0 6904K 2340K nanslp 2 0:23 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_PPPOE -B 65.94.26.33 -p /var/run/dpinger_WAN_PPPOE~65.94.26.33~10.11.16.17.pid -u /var/ru
0 root -16 - 0K 624K swapin 1 0:20 0.00% [kernel{swapper}]
70389 root 20 0 12464K 5760K select 3 0:16 0.00% /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid{ntpd}

sysctl: unknown oid 'hw.em.enable_aim'
sysctl: unknown oid hw.em.flow_control
sysctl: unknown oid hw.em.tx_process_limit
hw.em.rx_process_limit: 100
hw.em.rx_process_limit: 100
hw.em.rxd: 1024
hw.em.txd: 1024
sysctl: unknown oid 'hw.em.max_interrupt_rate'
net.link.ifqmaxlen: 128

Jimbohello

@Stewart in.zip in-queue.zip

same for the out queue except sources/adress

bobbenheim

@Jimbohello can you please run "sysctl hw.em" and "sysctl dev.em" in the command prompt and paste the result.

Jimbohello

@bobbenheim said in Wrong limiter speed:

sysctl hw.em

hw.em.eee_setting: 1
hw.em.rx_process_limit: 100
hw.em.enable_msix: 1
hw.em.sbp: 0
hw.em.smart_pwr_down: 0
hw.em.txd: 1024
hw.em.rxd: 1024
hw.em.rx_abs_int_delay: 66
hw.em.tx_abs_int_delay: 66
hw.em.rx_int_delay: 0
hw.em.tx_int_delay: 66
hw.em.disable_crc_stripping: 0

noplan

OK
I m Readin and learning

A Lot
Thank you!!!

bobbenheim

@Jimbohello I just assume you haven't tried this before so i will list the following out in simple steps:

go to Diagnostics / Edit File
press browse and find boot/loader.conf
add .local so you get the string "/boot/loader.conf.local" and press save
press browse then find and click on loader.conf.local
delete the commands within loader.conf.local and add the following:
hw.em.eee_setting="0"
hw.em.rx_process_limit="-1"
hw.em.txd="2048"
hw.em.rxd="2048"
net.link.ifqmaxlen="4096"
hit save and reboot

Jimbohello

@bobbenheim

WOW thank's for your precious help
on a limiter of 799 i can same as yesterday so close to 735
so no change !
up to date the queue lenght 100 was the best tunning
can you please provide fast explanation on these Tuning you provide !
i know that *.local usually will load the *.conf and override.
But what are these tunning ??
THANK'S

Jimbohello

@bobbenheim
1 came close to 760 !
i can easy live with that.
that's the best setting so far ! it's almost the limiter setting 799 ! the accuracy bettewn 760/799 is damw better !
BRAVO !
explaination on your tunning steel be please ! GRACIAS

bobbenheim

Try and keep a download running for some minutes and check monitoring you might have the correct limitation but the speedtest are not taking overhead into consideration.
Need to get back to you on those settings, it's time for some sleep :)