Traffic shaper changes

wcoolnet · Nov 7, 2006, 2:12 AM

I changed my mind. I'll donate $200

sullrich · Nov 7, 2006, 2:20 AM

This is a great idea. Hope some others can find this feature useful and donate. I know the author of the original shaper is sorta burned out atm so this may get him interested again :)

pfSense will donate 50$ to the project.

billm · Nov 7, 2006, 2:28 AM

BTW, I'm the original author. I have the test gear for this (if I can figure it out!) and have spent some time thinking about the shaper code recently and have worked on some code that I'm hoping will help us perform layer 7 shaping (a seperate project altogether). As Scott said, getting the shaper code to the point we're at was kind of time consuming and draining. If there's serious interest in this, I'm willing to dedicate some time on this.

–Bill

sai · Nov 7, 2006, 1:17 PM

I've read up on the ALTQ theory and its really mindbendingly difficult!

Would the new code handle more than one interface? If yes then $100 from me to be added to the bounty.

Numbski · Nov 7, 2006, 1:43 PM

I'll check around to see if I can rustle up some funding here too.

Christian · Nov 7, 2006, 1:54 PM

I'll donate $75, if the new traffic shapper support the following:

shapping an all interfaces
shaping traffic inside individual IPSEC tunnels, that is tunnels terminating at the local pfsense box.

I might be able to donate more if it will be somehow possible to route traffic depending on the amount of traffic queueing. What I mean, is something like this:
Route everything through IPSEC tunnel X on Interface 1, prioritise VoIP. If there isn't enough bandwidth available, route everythoing except VoIP through Interface 2 instead of Interface 1.

Christian

wcoolnet · Nov 7, 2006, 2:20 PM

We're at $425 :o

wcoolnet · Nov 7, 2006, 2:35 PM

If interest grows anymore, I think we should have a section on the wiki to keep track of specific features that we want implemented into traffic shaping.

billm · Nov 7, 2006, 4:45 PM

I wanted to comment on this one specifically as there are certain limitations.

@Christian:

I'll donate $75, if the new traffic shapper support the following:

shapping an all interfaces

Consider this on the list. I'm struggling trying to figure out how to implement this in HEAD with the new bridge infrastructure (you can bridge more than two interfaces…essentially making a firewalling switch)

@Christian:

shaping traffic inside individual IPSEC tunnels, that is tunnels terminating at the local pfsense box.

This may not be possible. ALTQ works outbound on the interface. While we could in theory shape the traffic coming in to your network via the tunnel (assuming you aren't doing tunnel to tunnel routing), the best we could do for outbound is shape the entire tunnel. The traffic is already encrypted by the time it hits the WAN interface and can't be inspected to see what ports it's on.

@Christian:

I might be able to donate more if it will be somehow possible to route traffic depending on the amount of traffic queueing. What I mean, is something like this:
Route everything through IPSEC tunnel X on Interface 1, prioritise VoIP. If there isn't enough bandwidth available, route everythoing except VoIP through Interface 2 instead of Interface 1.

Hmmm, that digs into kernel space that I don't think I can modify in a satisfactory way.

–Bill

sullrich · Nov 7, 2006, 4:55 PM

Please, lets keep this thread on track. I know everyone is excited for new features but this bounty is for a filtering altq bridge only. We can address the new kitchen sink down the road.

wcoolnet · Nov 7, 2006, 8:52 PM

To give you an idea of what I need:
T1 line multiple IPs
|
pfsense (bridge)
|
router–-------------------------------------------------------------
/ \ \
workstation lan ip mail1 webserv2 server3(voip)

I need pfsense to manage my bandwidth. Give VOIP the highest priority, mail the lowest, have a fast web browsing experience when the bandwidth is available, etc. So I would need port and ip based shaping.

mrt_ok · Nov 7, 2006, 9:30 PM

hi guys,

I would donate $1000 after my first successful deal with the transparent shaper box.

my requirements are low:
just shaping in bridged mode (two interfaces) to give certain services the QoS they need (e.g. web / citrix / shh etc…)

kind regards,
mrt_ok

wcoolnet · Nov 7, 2006, 9:41 PM

Would it be possible to make this work on a larger scale?
For example: OC12 connected to a few hundred servers in a datacenter. PfSense would limit the maximum amount of bandwidth each ip could use…

If this can realistically be done, then the funding options for pfsense would dramatically increase.
This could open the door to companies with lots of money, as they could use pfsense on their own infrastructure.

sullrich · Nov 7, 2006, 9:48 PM

@mrt_ok:

hi guys,

I would donate $1000 after my first successful deal with the transparent shaper box.

my requirements are low:
just shaping in bridged mode (two interfaces) to give certain services the QoS they need (e.g. web / citrix / shh etc…)

kind regards,
mrt_ok

Sorry but we need the cash up front. We have already been fooled into believing this from others and the policy now is half is due up front and half on completion. With this many people pooling their funds together we will need to gather the money up before starting the project. Sorry!

sullrich · Nov 7, 2006, 9:49 PM

@wcoolnet:

Would it be possible to make this work on a larger scale?
For example: OC12 connected to a few hundred servers in a datacenter. PfSense would limit the maximum amount of bandwidth each ip could use…

If this can realistically be done, then the funding options for pfsense would dramatically increase.
This could open the door to companies with lots of money, as they could use pfsense on their own infrastructure.

Per user bandwidth is a little more difficult. ALTQ does not have a per user distribution classifier so it would require a rule and queue for every user which is not doable in your case (it sounds like).

wcoolnet · Nov 7, 2006, 9:58 PM

Would it be possible to limit any connection to any server to something like 10Mbps?

sullrich · Nov 7, 2006, 10:00 PM

@wcoolnet:

Would it be possible to limit any connection to any server to something like 10Mbps?

Yes. 1 rule + queue.

wcoolnet · Nov 7, 2006, 10:11 PM

Question:
100 servers connected to oc12. all are web servers. 1 server is experiencing a traffic spike and receiving thousands of connections.

Is there no way to prevent a single server from hogging the entire oc12, using altq?

sullrich · Nov 7, 2006, 10:58 PM

Again, 1 queue and 1 rule per server or ip.

billm · Nov 7, 2006, 11:18 PM

@wcoolnet:

Question:
100 servers connected to oc12. all are web servers. 1 server is experiencing a traffic spike and receiving thousands of connections.

Is there no way to prevent a single server from hogging the entire oc12, using altq?

You'll need a queue per server (I expect that we'll need a wizard for that…or a way to branch the existing wizard code...not terribly difficult) to make this work. Then all you do is specify realtime guarantee's on each queue such that each web server is guarantee'd a certain amount of bandwidth, but could burst to whatever limit you set (or don't set).

--Bill