Dynamic Limiter script for pfsense

debremarkos · Nov 6, 2013, 1:55 PM

please rename to .pl and put in the /root dir more instruction to follow

see later post for file

debremarkos · Nov 6, 2013, 1:48 PM

This Script is a very basic script that is written in perl, there is no GUI.

The script looks at the maximum bandwidth (a user configurable variable ) and then uses the information passed to it by the limiter to determine how many actual connection there are ( ie active connections actually passing data). It then devides the bandwidth by this number to get a limit. As it would not be good to change the limit all the time the value is rounded down to the nearest 64 kb/s.

This has worked for me where we have 8000 users connecting over a 50 mb/s connection in Ethiopia.

The Installation institutions are as follows:

configure your limiter though pfsense (see pfsense wiki)
install the cron package.
upload the bandwidth.txt file to /root via ssh and rename in the shell (option 8) to bandwidth.pl
chmod 777 /root/bandwidth.pl
open bandwidth.pl in your favourite text editor and change the top variables to your environment. the dedaults are 50 mb/s as the connection speed the upper limit is 1mb/s the lower limit is 256 kb/s and the steps the limter takes are in 64 kb .
add a cron job to run the script every minutes in my case. You choose
in the root dir a daily log is created the log shows the date time the bandiwth used on your em1 (change to your wan interface name) interface.
it then shows the number of active buckets on the download limiter and then the actual number of those buckets that are transferring data. The next number is the actual limit and the last number is the caulculated limit before the rounding down.

I hope this help some one who is having difficulty fairly managing their bandwith

debremarkos · Nov 6, 2013, 1:54 PM

sorry made a mistake use this file, initialized the rate variable now the rate will show up in the logs

bandwidth.txt

Tillebeck · Nov 17, 2013, 10:44 PM

This sounds like the most wanted feature "share evenly on LAN" that can be found i nthe m0nowall :-) Very nice.

Just a tip on a similar feature.
I have used a router that can work with two limiters for each user (each IP).

Even bandwidth if more active users than bandwith (like your script)
Normal limiter when plenty of bandwidth (could be max. 20Mbit pr. user/IP)
If a user has more than 600 flows (probably p2p) then lower limit for that user to eg. 8Mbit

Those three rules gives in total a pretty nice setup without much need of QoS.

I have added some floating rules where traffic to/from the most used news servers end up in the p2p queue. Alias lists have to be updated now and then. But I really et to lower the priority of a lot of traffic that way.

Thanks for sharing.

debremarkos · Nov 18, 2013, 5:27 AM

Thanks for the comments.

I have done a little more work and will try and put this stuff into the script.

In the latest version I have fixed a couple of things.

things will now work when limit is over 1024 kb/s

It can send info to a rrd server if needed.(to make pretty graphs) I will also attach my rrd server that I run on a separate ubuntu computer.
The script looks at the units kb/s or mb/s so the log makes more sense.

bandwidthv2.txt

debremarkos · Nov 18, 2013, 5:34 AM

Here is my rrd server script that takes the data from the bandwidth script and makes nice graphs. I run it on a seperate computer runnung ubuntu 13.10

you need the rrdtool (apt-get install rrdtool) and you need to apt-get install librrdtool-oo-perl

variables are at the top of the script.

rrdservice.txt

debremarkos · Nov 18, 2013, 5:36 AM

again rename to a .pl extension. I have not tried it on a pfsense box but think there is a freebsd package for perl rrdtool::oo

debremarkos · Dec 3, 2013, 8:07 AM

same instructions as before, this scripts has many improvements, it checks users and the rate for a certain number of times (set to 15 here at 1 second intervals). Then it akes an average and uses this to calculate the limit. Also there are simple switches at the start of the script to turn on and off logging and sending things to the rrd server. Also there is now a log rotate feature tha will keep a certain number of logs and delete older logs.

I use this script in my environment fine and the new way of detecting who is an active user has led to a much better calculation of the limit.

Also the logs look prettier.

bandwidthv3.txt

myke · Dec 9, 2013, 10:33 AM

Hi,

hmm that's a great idea this script!!!

I will test it and tell you if it's okay.

Myke.

Tillebeck · Dec 10, 2013, 2:20 PM

Sorry, I have not tested it yet. I will test too :-)

debremarkos · Dec 20, 2013, 6:51 AM

Thanks, any feedback would be greatly appreciated :)

debremarkos · Jan 17, 2014, 8:01 AM

Just to let you know I am currently re writng this in php and getting it more integrated with the existing shaper php functions in pfsense.

debremarkos · May 21, 2014, 5:51 AM

Hi just thought I would let all know of the progress, I noticed alot of people asking for a limiting script so this might be helpful

I am working on v4 now which will have a better limiting algorithm and post the stats (users, limit, rate) as an xml file for other monitoring programs to read

cheonne · Nov 11, 2014, 6:14 AM

any update for this?
at least having a gui

thanks

HCJ · Nov 13, 2014, 12:47 PM

how are people getting on with this script ?

EricE · Jan 16, 2016, 11:28 PM

@debremarkos:

I am working on v4 now which will have a better limiting algorithm and post the stats (users, limit, rate) as an xml file for other monitoring programs to read

Hey debremarkos - any update on this?

GuHwk97 · Feb 2, 2020, 6:17 AM

Hey , This script is unavailable
Can anyone repost it ?
@EricE @debremarkos @HCJ
Thanks

divi · Nov 15, 2021, 2:18 PM

Does someone still have this available or a link to an equivalent approach?
@GuHwk97 did you find something ?