Need Help Configuring Limiters with Squid Proxy

tman222

Hi all,

I have got bandwidth limiters configured on a guest network, but since this network also has Squid configured on it (in transparent mode), the limiters aren't always respected (sometimes they work, sometimes they don't). It seems that I'm having the same issue as was described here:

https://forum.pfsense.org/index.php?topic=132960.0

I'm curious if someone could help with the right steps to get limiters to work on the guest network. It seems to me that I somehow need to limit the traffic from the proxy (through e.g. firewall rules) vs. just limiting traffic originating on the guest network (before it hits the proxy).

Thanks in advance for your help, I really appreciate it.

Harvy66

I don't understand why you want to limit the speed between the proxy and the client. That's all free internal fast bandwidth. What you really want is to limit between the proxy and the internet so your proxy doesn't consume all of your internet bandwidth.

tman222

@Harvy66:

I don't understand why you want to limit the speed between the proxy and the client. That's all free internal fast bandwidth. What you really want is to limit between the proxy and the internet so your proxy doesn't consume all of your internet bandwidth.

Thanks Harvy66, you are exactly right. I realized I have the limiter setup incorrectly by limiting bandwidth to the proxy vs. limiting the proxy bandwidth to the internet. Could you (or anyone else) provide me with some help/instructions on how to limit the proxy bandwidth to the internet? I'm not quite sure what set of rules I need to create to accomplish this.

Thanks again for all your help.

tman222

Hi all - I'm curious if anyone had any idea how to configure this? Is there a way to create a firewall rule that uses as its source the Squid Proxy and then apply the limiters to that rule? I know Squid allows you to set a maximum download bandwidth in the configuration options, but how about upload?

Thanks again for all your help.

tman222

Hi all,

I wanted to go ahead resurrect this topic since I still have not been able to figure this out unfortunately. Can anyone give me some advice on the type of rule I would have to setup so limit Squid connection speed to the internet? As I understand traffic from the LAN is intercepted by Squid and then Squid makes the actual connection outside the network (i.e. to the internet). I have Squid setup as a transparent proxy. Do I need to create a floating rule for this? If yes, would the following rule work?

Floating Rule:
1) Action: Match
2) Interface: WAN
3) Direction: Any
4) Protocol: Any
5) Source: LAN segment that has Squid running on it, let's call it LAN1 net
6) Destination: Any
7) Then under advanced set in/out pipe to the appropriate limiters/queues created under traffic shaping

Thanks in advance.

Harvy66

Assuming limiters with Squid works, I would make a limiter that is fq_Codel and force all traffic into it.

tman222

@Harvy66:

Assuming limiters with Squid works, I would make a limiter that is fq_Codel and force all traffic into it.

Thanks Harvy66 - I actually already created a limiter and enabled fq_codel on it for the network interface that has Squid running on it. The problem is if I enable the limiter/queues on the LAN firewall rule for that interface that allows outbound traffic, I just end up limiting the speed from the client to the proxy, but not from the proxy to the internet. If I wanted to limit to limit the speed of the Squid proxy to the internet, what kind of firewall rule would I have to setup, and where would I setup it up? Am I on the right track with what I posted above yesterday, or do I need to approach it differently?

Thanks again for all your help, I really appreciate it.

Derelict

Floating Rule:
1) Action: Match
2) Interface: WAN
3) Direction: Any
4) Protocol: Any
5) Source: LAN segment that has Squid running on it, let's call it LAN1 net
6) Destination: Any
7) Then under advanced set in/out pipe to the appropriate limiters/queues created under traffic shaping

This won't work for two reasons:

1. Connections to the internet come from squid itself and not anything on Source: LAN Net

2. Even if the connections were sourced from LAN Net that would not match because NAT has almost certainly already translated the source address of the connection in the outbound direction at the stage that rule is evaluated.

You might be able to mark specific squid traffic with a QoS marker then match that in the floating rule for putting in the correct pipe/queue:

https://wiki.squid-cache.org/Features/QualityOfService

A limiter should be configurable to work for uploads and downloads. Other than that, squid itself would have to limit the download speeds, if that is even possible. Looks like delay pools might do it if they're available.

tman222

@Derelict:

Floating Rule:
1) Action: Match
2) Interface: WAN
3) Direction: Any
4) Protocol: Any
5) Source: LAN segment that has Squid running on it, let's call it LAN1 net
6) Destination: Any
7) Then under advanced set in/out pipe to the appropriate limiters/queues created under traffic shaping

This won't work for two reasons:

1. Connections to the internet come from squid itself and not anything on Source: LAN Net

2. Even if the connections were sourced from LAN Net that would not match because NAT has almost certainly already translated the source address of the connection in the outbound direction at the stage that rule is evaluated.

You might be able to mark specific squid traffic with a QoS marker then match that in the floating rule for putting in the correct pipe/queue:

https://wiki.squid-cache.org/Features/QualityOfService

A limiter should be configurable to work for uploads and downloads. Other than that, squid itself would have to limit the download speeds, if that is even possible. Looks like delay pools might do it if they're available.

Thanks for the response - I really appreciate it.

I did a bit more research on the issue - would setting up Delay Pools in Squid also accomplish what a limiter would do (i.e. limiting total bandwidth of the proxy):

https://wiki.squid-cache.org/Features/DelayPools
https://forum.pfsense.org/index.php?topic=74595.0

Thanks again.

Derelict

Sorry. Don't know. You'll need to ask in the squid forum.

tman222

@Derelict:

Sorry. Don't know. You'll need to ask in the squid forum.

I'll go ahead and do that - thanks again for your help. I did try out delay pools with Squid, but unfortunately I was only able to limit download bandwidth. Back to the drawing board….

Derelict

Yeah. upload bandwidth should be able to be limited by marking and matching as explained above.

periko

Until now, Traffic Shape/Limiters doesn't work with squid in the same box right?

matt_

I think if you use limiters with floating rules matching on the WAN state creation (out direction) it will limit Squid. Someone can correct me if I'm totally wrong…

magokbas

manual proxy redirect
maybe it might work
https://forum.pfsense.org/index.php?topic=147247.0

periko

Need to test again… :)

tman222

Thanks guys for this additional info. I"d be curious to see if the manual redirect method might work.

Hi Matt - if I was to use a floating rule on the WAN interface, what would I use for the source of the traffic?

Thanks again.