Per IP traffic shaping–share bandwith evenly between IP addresses??

[daq]

I'm not sure what I'm missing, but foxale08's suggestions seem to only restrict total available bandwidth. We have 100mbps up and down. If I run two speed tests side by side on separate machines, one gets around 85/85 while the other gets the rest.

If I set the limiters to 50mbps then total available bandwidth does go down to 50, but it is still not shared evenly so limiters appear to work, just not the way I expect them to.

[Derelict]

Start two long downloads and see what happens.

[daq]

@Derelict:

Start two long downloads and see what happens.

Downloads may not use the entire bandwidth. Few servers have the capacity to provide 100mbps speed for every client. Plus, speed test does the same thing.

[Derelict]

And the speedtest sites do a lot of weird buffering to set things up and it's really hard to get two or more going at the same time.

To see it work on a 100Mbit set the parent limiter to a lower value (Say 2Mbits) and start two or more downloads.

[Ashfaq]

Has anybody been successful in equally dividing the bandwidth b/w a pc with torrent download and one with simple browsing?
or a mobile device against with simple data usage and a torrent downloading pc ?

with me the torrent download PC always gets almost all of the bandwidth all the time !

Ashfaq

[Ashfaq]

Limits apply correctly when test with speedtest.com, but when torrent download starts on the same machine the limiters have no effect on it, the torrent downloads goes full throttle.

any suggestions ?

[Derelict]

Configured properly they will go full throttle until there's congestion.  That's the point.  If you want to limit that specific IP to a hard-set top-end, set up your limiter that way.

[Ashfaq]

Thanks for the reply Derelict,

hard-set-top-end is what i did to test if limiters are even working ! , since my other configuration (discussed in this thread) to share bandwidth equally was not working at all for torrents.

i made two limiters download and upload, limit them with 512Kbps and 128Kbps, applied destination and source masking. and then applied in the firewall rule.

the PC is now limited to 512Kbps on speed-test but the same PC when running torrent get the entire 2Mbps data.

[Derelict]

Is this a long-running torrent connection?  Did you clear states after making changes to the limiter?

[Ashfaq]

Oh yes I do reset the states after every change and also sometimes do a restart of PfSense to make sure of it.
its the same case for both long-running and recently-started torrent sessions.

i also noticed something unusual yesterday when for some reason my limiters just stopped working after a simple reboot, had to recreate them to start working properly again and they did, problem resolved.  but the strange thing i noticed in the exercise is that during this problem if a limiter is attached in the firewall rule i could not browse, ping www.google.com site or do any other network stuff like email etc, if i remove the limiter from the rule everything was okay. but the torrents were working full speed all that time without any break, with or without the limiter.

the recreation of limiters (like they were before) resolved the issue, but thats not the point to discuss, the torrents were like they never belonged to the rule or limiter (from the same machine), everything else seems to be effected though.  i looked closely to the diagnostics->limiter-info and i could see the no of packets/bytes from PC that clearly belonged to torrent, which means it was infect passing the queues.

i tried different things with rules and disable, enable, block, pass all the LAN rules but nothing effected the torrent streams(made sure to reset the states after each change).  the firewall logs was also showing the traffic on port torrent was using.

all in all I was clueless as to how to debug this trend.

Thanks for all the help
Regards

[Derelict]

Are you sure you have all your in/out source/dest mask rules right?  You should probably post screenshots of your limiters and the associated firewall rules.

The limiters are application agnostic.  They operate on IP addresses and interfaces.  Given the correct application in rules, they just work.

[Ashfaq]

Screen shots are attached.

its important to mention that i initially tried to use the queue based setup discussed earlier in this thread and u can see my earlier posts with screen-shots for that setup as well.  but since i failed to achieve the desired outcome i am now trying to apply a simple limiter to get this torrent thing resolved first.  and the screen-shots with this message are for this simple setup.

if u like i can post the speed-test and torrent screenshots separately as well with traffic graph n all.

[Derelict]

Hmm.  Are there any queues defined?  What you have should work.

One thing I see is the pass any to !LAN net rule with the limiters on it is unnecessary.  I'd just put the limiters on the pass any any rule.  If it's being processed by the LAN router interface, it's by default !LAN Net.

And the traffic out WAN is mirroring the traffic into LAN right (2.0Mbps when bittorrent is cranking away)?

[Derelict]

Actually, the way you have it, each destination IP will get 512 down and each source IP 128 up, so two downloaders can eat 1Mbit, 3 1.5Mbit, etc.

What are those other queues?

Are you sure there's not some other rules somewhere setting limiters, queues, etc?

Here's a sample that will evenly share 10Mbit down 3Mbit up among IPs


[Derelict]

Hmm… This isn't working for me either.  I currently have a limiter at 1Mbit / 1Mbit with rules on LAN for my desktop and phone.  Speed tests reflect this but I'm downloading torrents at about 10Mbits.  I applied the limiter, killed transmission, cleared all states, and restarted transmission.

Do you have an inbound rule on WAN forwarding your bittorrent listener port to your host?

I did.

I disabled that rule and am now getting better limiter results.  Seems like a limiter on the WAN rule should do the trick but I did try that and it didn't for me.

Now my speed test on the phone that's put in the same limiter gets about 512Kbit down and almost the full 1Mbit up, which is exactly what I would expect.

Anyone know the trick for throttling bittorrent using the limiter?

[Ashfaq]

wow so many messages :-) …. Thanks Derelict for the efforts, i really appreciate that.
now to answer ur queries one by one:

*) no other queues are defined, only limiters and no other rules are defined in firewall. 
*) the LAN to !LAN is there to copy the setup defined in this thread earlier... so just to make sure i am not doing what other are not doing to simplify the debugging of this issue. u r right we dont need that and i have tried the setup without it as well... no diff in result :-( .... one question though... what if i am transferring data from my LAN to PUBLIC (another LAN) ? putting the limiter in default rule might limit that to 2mbps as well.. just a thought.
*)those other queues are related to the equal sharing of 2mbps that i tried earlier based on this thread messages and did not succeed.  they are there and not actually used as of now in any rule.
*)YES ! i have UPNP turned on in pfsense and using a pre-defined-port in torrent application.  although i have not made any specific rule in firewall for this to work.  i even tried turning the UPNP off but didnt help either.

now let me tell u something new i tried....
*)yesterday i reverted back to the OTHER RULES u see on the screen-shots that are there to sharing the 2mbps connection using child queues, i attached those queues to the firewall rule of LAN to !LAN and
*)also created shapper-queues with help of wizard and it made a few floating rules with http(s), smtp, pop as HighPriority and rest as lowpriority.  and
*)then i manually attached the limiter-queues in those torrent related floating-rules and
*)tested again using two windows PC on torrents and an android with regular video streaming and browsing etc.

to my surprise it DID started to share bandwidth b/w the three devices :-)
if i disable those floating-rules, i end up like before... torrents take away ALL the bandwidth and no browsing is possible on android, as soon as i enable those rules i get the sharing working.
i cannot make out why its working but may be u can, so please try that.

Thanks again for all the help.
Regards

[zang3tsu]

@Ashfaq Could you post your final settings again as I 'm hoping to replicate your setup. Thanks!

[Ashfaq]

Check my previous posts in this thread for limiter setup screen-shots.
attached are the screen-shots for floating rules added by wizard and the 2 amendment i made in the p2p rule ONLY:

1. added in/out limiters values
2. changed port range to include ports used by my torrent client.

another observation with this setup is that sometimes my torrents dont get the FULL speed even if they are the ONLY thing running the network.  disabling the floating rules gives it the complete bandwidth.

Ashfaq

[ssp]

Dear all,

I'm a first time poster here - these forums have been tremendously useful just from reading them. Sorry for my phraseology, the terms are still difficult to grasp. We have a college in a developing country - using desktop grade hardware to share 10Mbps synchronous connection among all users. It's slow but it works. We see 100 simultaneous network users regularly, and 300 at peak times. Firewall is pfsense 2.1.5.

Obviously traffic shaping is very important for us. The details given in this thread have been very useful in getting things started. I have also noticed, as some users above, that a one-click rule for this important feature (dynamic per-IP traffic shaping) would be really helpful.

The shaping using limiters described here is a great starting point. It does 'break' occasionally in ways that amateurs will probably never be able to figure out, least of all me. Also, it doesn't 'share' or 'throttle' the bandwidth from hogs (downloaders, with sustained bandwidth usage) to 'new comers' (i.e. someone just trying to load a single webpage) fast enough. Any ideas on this?

In the above scenarios, I'm wondering if this 'burst' feature that's there with limiters might work. However, you have to set that at the parent level. Does that somehow also get shared among dynamic child pipes? Anyone figure out what the units for that are (bytes, kb, mb), or if it always matches the units of the bandwidth specified. I know it's supposed to be an amount/size, not a rate, but I haven't figured out the units of it, or like I said, if it gets divided equally into the child pipes.

[Basi]

Are there any news this will be an official feature? Maybe in 2.2?