Multi wan switiching issue with proxy enabled

deepakthai

heper,

Thanks for your suggestions. I have already followed the steps mentioned in DimitriS Post at http://securite-ti.com/pfSense_Web_Proxy_with_multi-WAN_links.pdf

I will try to brief the issue once again here.
We have two ISP's and using Pfsense we are able to connect to WAN on our local network pc's. We have a script running on our local network which helps us to select the desired ISPs from the two we have (ISP1 and ISP2). Now the issue is when we enable proxy in pfsense we are not able to switch the ISP's via the script on machines in local network. With proxy enabled if we try to switch the ISP say from ISP1 to ISP2, the switching will take place, but still the web browser will show up ISP1, in the meantime traceroute and mtr shows ISP2. This only happens with the proxy enabled. If we disable the proxy all will work fine as desired.

Hope it clearly explains the scenario.

jikjik101

squid uses the default wan (in you case, ISP1) for http 80.
the rest can be switched as per your testing in mtr.

the link you posted sometimes work, sometimes not.
it's the reason why i don't want to use proxy because of it's inability to failover/loadbalance without tweaking or adding numerous changes to pfsense.
and you need a lot of LUCK to run the proxy in failover. ;D

CDuv

@jikjik101:

squid uses the default wan (in you case, ISP1) for http 80.
the rest can be switched as per your testing in mtr.

the link you posted sometimes work, sometimes not.
it's the reason why i don't want to use proxy because of it's inability to failover/loadbalance without tweaking or adding numerous changes to pfsense.
and you need a lot of LUCK to run the proxy in failover. ;D

You're saying there is no simple way to make Squid works with a multi-WAN load-balanced/fail-overed pfSense setup?
What about using a separate server running Squid and specified in the pfSense's Squid configuration as an upstream proxy server (of course, this server won't be configured to use pfSense's Squid)?

jikjik101

@CDuv:

You're saying there is no simple way to make Squid works with a multi-WAN load-balanced/fail-overed pfSense setup?
What about using a separate server running Squid and specified in the pfSense's Squid configuration as an upstream proxy server (of course, this server won't be configured to use pfSense's Squid)?

in my experience, YES. no simple steps, no shortcut methods and no standard way to do it. Standard in a sense that all the HOWTO posted here MIGHT work in your system. But most of the time, they don't. For example, there is one howto that i followed in 2.0.1 that works but in 2.0.3 it doesn't.

I heard some users are using a separate proxy server. that is also my plan, but i am a window baby, so i have a hard time of installing and configuring squid in unix or different OS.
This time, i am testing the Freeproxy. If i can just eliminate squid in my pfsense box, i think my multiwan problem will be solved. There are some posts on the web about window squid, but transparent is not supported. if i cant make the external window proxy server, then i will be forced to learn the unix or redhat :o

I really love to have my network to have a failover and loadbalance WITH content filter. But i can't do the multiwan if there is squid (squid is required for content filtering).
It's either you implement multiwan OR proxy, but you cannot mix both.

CDuv

@jikjik101:

in my experience, YES. no simple steps, no shortcut methods and no standard way to do it. Standard in a sense that all the HOWTO posted here MIGHT work in your system. But most of the time, they don't. For example, there is one howto that i followed in 2.0.1 that works but in 2.0.3 it doesn't.

Would, by any luck, still have it somewhere?

@jikjik101:

I heard some users are using a separate proxy server. that is also my plan, but i am a window baby, so i have a hard time of installing and configuring squid in unix or different OS.

I'll give it a try and check if pfSense allows that (either via pfSense's Squid + "upsteam proxy" setting or without pfSense's Squid and two simples Firewall/NAT rules to allow direct TCP 80 access from OtherSquid and translate any TCP 80 traffic from LAN to OtherSquid:3128)

jikjik101

@CDuv:

Would, by any luck, still have it somewhere?

http://forum.pfsense.org/index.php/topic,39851.msg206226.html#msg206226 <- this was my setup in 2.0RC3 and 2.0.1 but I cannot do it in 2.0.2 and 2.0.3 prerelease

http://forum.pfsense.org/index.php/topic,37083.msg198593.html#msg198593 <- some claim that this one works but not on me.

@CDuv:

I'll give it a try and check if pfSense allows that (either via pfSense's Squid + "upsteam proxy" setting or without pfSense's Squid and two simples Firewall/NAT rules to allow direct TCP 80 access from OtherSquid and translate any TCP 80 traffic from LAN to OtherSquid:3128)

IMHO, i would prefer NO squid in my pfsense box. Squid is the culprit why multiwan is not working due to the limitation of the loop interface. All other traffic can be load balanced except for the http which will go out to the default WAN.

jimp

@jikjik101:

I heard some users are using a separate proxy server. that is also my plan, but i am a window baby, so i have a hard time of installing and configuring squid in unix or different OS.

That's when you put up a second pfSense install on a DMZ in appliance mode (one interface) running the squid package as just a proxy box. :-)

jikjik101

@jimp:

That's when you put up a second pfSense install on a DMZ in appliance mode (one interface) running the squid package as just a proxy box. :-)

If I can't run a multiwan with squid inside the pfsense box, then setting up a proxy box will be my last option. (edit: I can run multiwan, but not the way i wanted)

In 2.0.3 prerelease, I can use multiwan by just adding floating rule. But I can only use one gateway group. Example: GW1 is load balance, GW2 is failover1, and GW3 is failover2.
In my LAN rules, for Alias1 I want to use GW1, Alias2 for GW2 and Alias3 for GW3. But in reality, Alias1-3 will use the last floating rule. If in my floating rule, i set DNS and http for GW1, then all aliases will use GW1 which is loadbalance even if I set Alias2 to use GW2 and Alias3 to use GW3.

In my attached pictures, LoadBalance GW is at the bottom of the floating rules, so all my LAN clients will use the LoadBalance GW even if I set them to use different GWs.
If I change the bottom rule to FailOver1, then all my LAN clients will also use FailOver1 GW.

GW.jpg_thumb

Float.jpg_thumb

LAN.jpg_thumb

niebla

with 2.0.3 try disabling the floating rule, add the suggested DNS floating rule and make sure you have rules in in of your lans to your multil wan gateway.

Working with squid so far. We'll see tomorrow.

amavin

Having the same issue getting traffic from localhost to work with multiple gateways, doesn't really matter if its failover or load balanced. The traffic from localhost will always follow the default gateway.
Sure you can force the traffic to leave another interface trough a floating rule, but trouble is that the floating rule will kick in after SNAT(outbound NAT) has already happened, leaving you with a packet that will always have a source address of the default gateway interface. So effectively in a dual-wan load balanced setup, the 1st request will leave wan1(default) and return on wan1, request 2 will leave wan2, and return on wan1, and of course pfSense kills request 2.

<slightly ot="">I believe this is the same issue that I'm experiencing with OpenVPN on udp listening on "any" interface, the request can enter any interface, but OpenVPN will always respond through the default GW. Though it seems to work fine with TCP.</slightly>

What we need is some kind of logic that can apply the rules before SNAT, possibly something with routing the traffic trough a dummy-interface and reflecting it back, making it look like regular traffic entering the interface, for (re-)processing. Isn't this how NAT-reflection works? wouldn't it be possible to make something like this?