Multi wan + squid(transparent) + squidguard on 2.2.4



  • hi folks,
    I know this is a recurring question and I did read tons of posts about the above configuration .. but I'm dead in the water on a 2.2.4.
    I'm not trying to make anything complicated, just two WAN in balancing (working without squid) and transparent proxy.

    I did follow all the suggestion regarding tcp_output_address to 127.0.0.1 and floating rule but I think the info I have are not correct,
    or at least they don;t work on my 2.2.4

    any gentlemen out there that can post screenshot of his config if you have transparent on lan for a multi wan?

    thanks in advance.



  • Could you please clarify what your exact problem is?
    I suspect, given what you wrote, that "it doesn't work" but it would be helpful to understand better if there is any error message, unexpected behaviour (like no balancing) or something else.

    I doubt Squid will really benefit from WAN load balancing.
    WAN fail-over definitely works (I'm relying on it, although not with local Squid server) but load-balancing will definitely generate some issues with some sites in case client IP (here WAN) changes.



  • correct … I can't get it to work :-(

    I have two eth on wan, in load balancing on a gateway group.
    if I use pfsense without enabling squid+squidguard (but squidguard is not part of the issue...) I can see the balancing to work correctly, no problem there.

    problem is: I need to filter traffic in a transparent mode (local office setup) so I need squid+squidguard(+clamav ideally)
    once I turn that on .. as we know squid will try to use always the default WAN gateway.

    I did read a lot on the forum and many howto about the floating rule to apply and the tcp_output_address to 127.0.0.1 ..
    did that .. but still does not work.

    ===
    out of curiosity ... I'm happy with failover too if I can use squid+squidguard on your same setup,
    could you post your configuration / setup rules?

    thanks



  • I don't face this problem, although I'm using both pfSense in multi-WAN and Squid because Squid, like any other "non-FW" service (*****) is not running on server that is hosting pfSense.

    Therefore, Squid is one server (almost) inside my network, in fact deployed on "infrastructure services" server on DMZ. Its default gateway is pfSense interface (as for any other server on this DMZ) and everything work smoothly  ;)

    (*****)  my pfSense server is running FW obviously but also VPN because this is easier and DNS Resolver. Anything else is move elsewhere, meaning proxy, anti-virus, file sharing, mail etc.

    That quite easy and efficient  8) but it will not work for you  :-
    Reason is that you define "transparent proxy" as a prerequisite. Why not  ::) I'm definitely not fan of transparent proxy. To me, it only exhibits drawbacks and no added value. If reason behind such design choice is to ensure that proxy is always used without need to configure it on each device, the answer is WPAD  ;)

    This said, you may have other reason to make transparent proxy mandatory. Why not. This doesn't mean proxy has to run on pfSense.
    I'm pretty sire there is a couple of different designs  ;) able to fit your needs without merging proxy end FW on same hardware.

    • depending on protocols used by your internal devices, if everything relies on proxy, you are not obliged to define your FW as your default gateway. That's a bit tricky but you could define your internal proxy as default gateway for internal device. configure it as a transparent proxy. It will catch all requests and forward it to internet through pfSense. Pay attention to your DNS design because when using explicit proxy, URL is resolved at proxy level while when using transparent proxy, URL is resolved at "browser" (that's a short-cut) level.

    • keeping pfSense as your default gateway, you could also tweak the redirection generated when enabling transparent proxy so that requests are not redirected to internal proxy but an external one. This may require to pay attention to some NAT and routing aspects depending on where this proxy is deployed.



  • I have the same problem.

    Generally speaking, the problem is not only squid, but all the local services that use default gateway (squid, BIND, available packages for PF).

    Just for example:
    i have 2 WAN, configured as WIKI suggests (1 gateway group for load balancing, one for failover on WAN1 and one for failover on WAN2)
    Everything work perfectly (LAN clients and local services)

    If i unplug WAN1 (the default gateway), LAN clients continue to work without problems (using WAN2), but i can't view Package list to install (System -> Packages -> Available packages)

    As superslot says, i tried using Floating Rule in Firewall, with every combination, but without success.

    To prevent problem with DNS, i tried also via command line this command:
    telnet <ipaddress>80
    on pfsense, but it goes on timeout

    If I try the same command on a client machine, it works.

    Those are the list of posts i read this morning (last three days post list will be huge):

    https://forum.pfsense.org/index.php?topic=52171.15

    https://forum.pfsense.org/index.php?topic=45458.0

    https://forum.pfsense.org/index.php/topic,60977.0.html

    https://forum.pfsense.org/index.php?topic=57606.0

    I think something changed in 2.2.4 that need different solution.

    Thanks to all!</ipaddress>



  • Same problem here - this has been a problem FOR YEARS and NEVER gets adequately answered.

    Superslot hit the nail on the head.

    Apparently, load balancing routes are completely skipped by a number of services on the pfsense machine as those service ONLY use the PC default routes.  My problem goes slightly further in that I have two routes to load balance one is a traditional WAN, the other is a private link to our parent office and does NOT NAT.  The Squid Random ACL to loadbalance does not work because if I specify the Private WAN address, that address does not NAT, even on the internal network.

    The further problem is that I cannot NAT to company link, or connections from other company locations cannot be made into our PC network for management.

    Anyway, I did read a small blurb about trying to get squid to use traffic shaping policies.  Apparently, Squid.inc is hard coded to one particular bucket, thus the load balancing and other traffic levels are ignored.  I wonder if a similar mechanism is at work here?

    What I really do not understand is the order that packets are processed by the various modules, or even if that order can be changed.  Since we use both transparent proxy and normal proxy, squid is normally the first recipient of any traffic (I believe).  So, it needs to cache or filter but then pass traffic to the normal handling.  I'm pretty sure that squid just dumps the traffic post firewall locations, because any attempts to to route internal traffic fail unless squid is instructed to bypass the proxy for local traffic.

    The ONLY way that I have found to fix this is to setup TWO firewalls.  One runs squid and any filtering, the second is the actual boarder firewall and it does policy routing, etc, but does NOT run squid.

    In the end, I find it very confusing.  (Similar to the very annoying Garbage collection process that runs at 2 minutes after EVERY hour and causes a 30+ second outage while squid + squidguard + dansguardian or Shallalist reload.  The more filtering in place, the longer the hang.  Under heavy traffic, there is an interface overflow somewhere and the whole thing crashes… and yes, it is a full PC implementation with Core2 CPU and 8GB or RAM, not an appliance.)

    I have posted SEVERAL times or several years on the second issue and never gotten more than another user question and zero responses for a fix.



  • I believe There will not be a fix , unless some of the ones that want it fixed - fix it themselfs or make a different effort.

    Squid and squidguard packages havent been actively maintained in years.
    They keep running by community & coredev bugfixes, but honestly they are a mess.

    If it is at all possible to do what you want it to do: start digging https://www.github.com/pfsense



  • The main reason why it doesn't work as you expect is that fail-over & load balancing is done using group gateway but also rule at FW level which allows to specify this group gateway within these rules.

    With external proxy, it obviously benefits from such rule while with internal proxy that is not using any interface but 127.0.0.1, it doesn't work.


Log in to reply