How-To: 2.0 Load-Balance + Transparent Squid (3 easy steps)
-
nassman for dyndns you need to override the floating rule that balances the http traffic ….
so basically you need to create a rule with destination=your_dyndns_provider to use the correct gateway
-
heper thank you, but can u give me a pic for example.
also after i installed squid, when open Dashboard, in system information some times appear this
Version 2.0-RC3 (i386)
built on Tue Jul 19 02:18:00 EDT 2011Unable to check for updates.
and some times work.
what the solution?
thanks -
heper thank you, but can u give me a pic for example.
also after i installed squid, when open Dashboard, in system information some times appear this
Version 2.0-RC3 (i386)
built on Tue Jul 19 02:18:00 EDT 2011Unable to check for updates.
and some times work.
what the solution?
Yes, it will not be able to check updates. My quick and dirty fix for this is to temporarily disable the floating rule when I want to check for updates.
thanks -
stramato,
i added floating rule and make pfsense website same as DMZ, not load balance, it work and check for update.
thanks. -
can you elaborate more?
on how the rule will look like? thanks!
-
does this mean that i dont need to create a rule on lan tab? can you show any screenies of your lan tab rule?
-
Yeah !
Thanks, it works !! -
Thak you for this how-to! It works wonderful with squid.
But it's not apply to HAVP.
We have SQUID with SQUIDguard as transparent and HAVP as it's parent.
If we set firewall rules as you show, pages load by half, styles and images miss or even some site become unaccessible (timeout).
How can we make HAVP to load balance? -
Hello,
The following aspects are not clear:
1. You're selecting only WAN1 in the "interface packets must arrive to match rule" in the floating rule. Now the questions are
(a) are you assuming that squid will always through packets on WAN1 only?
(b) does which WAN should be selected in the interfaces box depend on the default gateway setting of the pf box?
if no default gateway is selected in the pf general settings, then which interface(s) will squid output packets to? Is that random? Like in the case of 3 WANs, squid may output packets to any of the 3 WANs?
(d) if any of the 3 WANs may be used by squid, in that case do we have to multi-select all WANs in that interfaces box?
2. For loadbalancing in particular, @heper's instructions included an additional "matching rule" where he was marking packets and later on in another rule routing those marked packets - to achieve loadbalancing. But in your steps that rule is not there. So is it the case that loadbalancing may be achieved without going for such packet-marking-routing as done previously by @heper?
Dear @heper, you can also clarify please...
-
-
1a: squid will Allways TRY to go out the default gateway … assuming that is WAN1 , you only need floating rule on that one
1b: see 1a
1c: not sure but i guess the default "WAN" interface, try if you wanna be sure
1d: see 1a2. the matching rule is useless, it appeared packets were getting looped twice around the packet filter but
emal pointed out the following:It hits it twice but really it does not execute the policy routing the second time.
Only the nat rules are executed. -
Thanks for your answers, @heper - really appreciate this.
We're in the process of extensively testing our triple-WAN, load-balancing with transparent squid and will report back if there's any case where it doesn't function as expected.
-
@heper will any of the above change when the squid is not transparent?
-
I read this thread and was able to get loadbalancing squid working using the post of the OP.
But in my multi-LAN multi-WAN environment I don't want to run squid as a transparent proxy on ALL interfaces.I'm now afraid some packets that are going out to port 80 and not coming from squid are loadbalanced as well…
I don't want this as I have issues with round-robin on certain websites.....I took a look at the approach heper did and am now doing the following....
mark all TCP packets going to port 80 coming from 127.0.0.0/8
on the interface of the default gateway send the marked packages to loadbalance.
I have not switched to AON (manual NAT) nor did I select localhost as one of the proxy's interface...Could someone please comment?
BTW... it's still not actually working here...
EDIT:
I did some more testing....
If I route all TCP out packets to port 80 to the round-robin interface, it is working... But because I'm logging it I also know that packets not coming from squid are using round-robin as well.
I don't want this....If I also check the marking of the packets, it doesn't detect any of them and nothing goes through the round-robin interface.
It seems that marking of the packets coming from 127.0.0.1 doesn't work....In combination with the absence of the lo0 interface in the pfsense webif it makes me think there's no firewall between the local interface and the rest....
HELP?
-
I may think I know why things are not working as I expect….
According to the manual the "Floating rules" are executed before the interface rules, but it seems the "pass out on lo0 all" is an exception.
This rule is made by the system...I'm unfamiliar with pf still.
Does this mean my floating rule will never be executed?
If I read it as "iptables" it wouldn't...How can I control traffic coming from the lo0-interface?
Please enlighten me!pfctl -sr | grep 'pass out' . . . pass out on lo0 all flags S/SA keep state label "pass loopback" pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself" pass out route-to (dc0_vlan10 89.250.179.1) inet from 89.250.179.117 to ! 89.250.179.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself" pass out route-to (pppoe0 217.16.40.239) inet from 82.172.129.149 to ! 82.172.129.149 flags S/SA keep state allow-opts label "let out anything from firewall host itself" pass out route-to (dc0_vlan13 89.250.180.1) inet from 89.250.180.164 to ! 89.250.180.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself" pass out log inet proto tcp from 127.0.0.1 to any port = http flags S/SA keep state label "USER_RULE: Mark packets coming from Squid" tag from_squid
-
i honestly have trouble seeing the meaning in what you are trying to do …
-transparent squid will only accepts http traffic ... so no https is going over squid.
-you can set different gateway groups per interface and you can apply different gw groups to different rules on the same interface
-in theory if you use loadbalancing on port 80 and transparent squid is enabled on the interface, all http traffic will go over squid to loadbalancer.Could you clearify why u'd want to "identify" squid traffic and mark them ?
-
I want to use squid for only a few LAN interfaces.
Now all traffic to port 80 will go to the loadbalancing gateway.
Even if I set it to a certain gateway before.
I guess that traffic gets intercepted by this reroute rule.
I don't want this.I just want the traffic coming from squid go through the loadbalancer..
I can't use the source-address (127.0.0.1) anymore as the source-address is already the WAN-IP.I just mentioned https because they more often have problems with loadbalancing.
But do you know enough about pf to confirm/deny that the tagging rule will never be executed because it is later in the rule-list?
-
https does not go over port 80, so that rule will only affect http
did you set the "tcp_outgoing_address 127.0.0.1" in squid ? I'd think you should be able to use it as a source address in your floating rule
-
https does not go over port 80, so that rule will only affect http
I know it doesn't (at least normally).
Please forget I ever mentioned it…. As I said... I just mentioned https because these sites have even more problems with round-robin.
This issue with squid has nothing to do with https nor did I think it had.did you set the "tcp_outgoing_address 127.0.0.1" in squid ? I'd think you should be able to use it as a source address in your floating rule
Yes, I did.
It IS working (the loadbalancing), but it's loadbalancing all traffic to port 80.
I think most people either use squid on all LAN-interfaces or they don't use it.This traffic is indeed coming from 127.0.0.1, but not anymore when that rule is applied.
Turn on logging and check it.
The moment that floating rule is executed, the source address is the WAN-IP (as you can see, when you log it).
The filter is apparently between the WAN-IP and the WAN-gateway, which makes sense.
So I have no way of distinguishing between the normal traffic going to port 80 and the traffic to port 80 coming from squid.I think it even would work if I am able to get it like this:
pass out log on lo0 all flags S/SA route-to { (pppoe0 217.16.40.239), (dc0_vlan13 89.250.180.1), (dc0_vlan10 89.250.179.1) } round-robin inet proto tcp from any to any port = http flags S/SA keep state label "loadbalance for Squid" pass out on lo0 all flags S/SA keep state label "pass loopback"
The webif doesn't let me control the lo0 interface and that second line is being put there by the system.
-
This tutorial work in case of gateway fail-over? Wan1 -> tier1 and Wan2 -> tier2