SQUID and Load Balancing doesn't work!!!! I tried for a week but nothing :'(
-
Well bad news :'( :'( :'(.. I woke up one morning realizing that I had everything set up but nothing I prove everything with all possibles rules in floating and nothing,
rules like for exampleFloating Rules
pass out quick route-to { ( vr0 192.168.1.1 ) , ( xl1 192.168.20.1 ) } round-robin from 127.0.0.1/24 to any keep state label "USER_RULE"nothing ???.. other rules, the rules below doesn't work >:(:
User-defined rules follow
pass out quick route-to { ( vr0 192.168.1.1 ) , ( xl1 192.168.20.1 ) } round-robin proto tcp from 127.0.0.1 to any port = 80 keep state label "USER_RULE"
pass in quick on $LAN route-to { ( vr0 192.168.1.1 ) , ( xl1 192.168.20.1 ) } round-robin from 192.168.55.0/24 to any keep state label "USER_RULE: Default allow LAN to any rule"let me tell you that i prove thousands of rules but nothing :-, I will prove again this weekend but I almost sure that IS NOT POSSIBLE USE SQUID AND LOAD BALANCING IN PFSENSE 2.0 :-[.
My friend Ermal tell me that this is possible :o, but I wonder if there is person that work with SQUID and BALANCE ( I think that none)..
Ermal if you send me one configuration that I can use in my network :-[ I will send a beautiful gift from Peru.. jajaj well your help is appreciable
I beg you send me a setup that really works.. my whole country will thank you ;) -
CONCLUSION:
SQUID and LOAD BALANCING doesn't work together.. why??? well is a little hard to explain, you can see more details in picture below, Squid only work with DEFAULT GATEWAY not with LOAD BALANCE GATEWAY, Squid choose one gateway, and this gateway is "DEFAULT GATEWAY"
See the pictures please
PS: All configuration is OK including the floating, Load Balancing works fine and Squid also.. but when Squid and Balance works, Squid only works with one Gateway or one WAN INTERFACE.. it can be WAN or OPT1.. I thought that Squid was working with Virtual Interface created by Load Balancing i.e that two WANs provide cache
-
Is it not possible to make the gateway-group default? I don't have Multi-WAN, so its only a thought… Maybe wrong.
-
Can you send your config.xml this is not going to take you/me anywhere like this.
-
Here I send config.xml I hope all is OK.. if you have other thinks please only tell me..
<pfsense><version>6.0</version> <lastchange></lastchange> <theme>pfsense_ng</theme> <sysctl><desc>Set the ephemeral port range to be lower.</desc> <tunable>net.inet.ip.portrange.first</tunable> <value>1024</value> <desc>Drop packets to closed TCP ports without returning a RST</desc> <tunable>net.inet.tcp.blackhole</tunable> <value>2</value> <desc>Do not send ICMP port unreachable messages for closed UDP ports</desc> <tunable>net.inet.udp.blackhole</tunable> <value>1</value> <desc>Randomize the ID field in IP packets (default is 0: sequential IP IDs)</desc> <tunable>net.inet.ip.random_id</tunable> <value>1</value> <desc>Drop SYN-FIN packets (breaks RFC1379, but nobody uses it anyway)</desc> <tunable>net.inet.tcp.drop_synfin</tunable> <value>1</value> <desc>Enable sending IPv4 redirects</desc> <tunable>net.inet.ip.redirect</tunable> <value>1</value> <desc>Enable sending IPv6 redirects</desc> <tunable>net.inet6.ip6.redirect</tunable> <value>1</value> <desc>Generate SYN cookies for outbound SYN-ACK packets</desc> <tunable>net.inet.tcp.syncookies</tunable> <value>1</value> <desc>Maximum incoming/outgoing TCP datagram size (receive)</desc> <tunable>net.inet.tcp.recvspace</tunable> <value>65228</value> <desc>Maximum incoming/outgoing TCP datagram size (send)</desc> <tunable>net.inet.tcp.sendspace</tunable> <value>65228</value> <desc>IP Fastforwarding</desc> <tunable>net.inet.ip.fastforwarding</tunable> <value>1</value> <desc>Do not delay ACK to try and piggyback it onto a data packet</desc> <tunable>net.inet.tcp.delayed_ack</tunable> <value>0</value> <desc>Maximum outgoing UDP datagram size</desc> <tunable>net.inet.udp.maxdgram</tunable> <value>57344</value> <desc>Handling of non-IP packets which are not passed to pfil (see if_bridge(4))</desc> <tunable>net.link.bridge.pfil_onlyip</tunable> <value>0</value> <desc>Set to 0 to disable filtering on the incoming and outgoing member interfaces.</desc> <tunable>net.link.bridge.pfil_member</tunable> <value>1</value> <desc>Set to 1 to enable filtering on the bridge interface</desc> <tunable>net.link.bridge.pfil_bridge</tunable> <value>0</value> <desc>Allow unprivileged access to tap(4) device nodes</desc> <tunable>net.link.tap.user_open</tunable> <value>1</value> <desc>Verbosity of the rndtest driver (0: do not display results on console)</desc> <tunable>kern.rndtest.verbose</tunable> <value>0</value> <desc>Randomize PID's (see src/sys/kern/kern_fork.c: sysctl_kern_randompid())</desc> <tunable>kern.randompid</tunable> <value>347</value> <desc>Maximum size of the IP input queue</desc> <tunable>net.inet.ip.intr_queue_maxlen</tunable> <value>1000</value> <desc>Disable CTRL+ALT+Delete reboot from keyboard.</desc> <tunable>hw.syscons.kbd_reboot</tunable> <value>0</value> <desc>Enable TCP Inflight mode</desc> <tunable>net.inet.tcp.inflight.enable</tunable> <value>1</value> <desc>Enable TCP extended debugging</desc> <tunable>net.inet.tcp.log_debug</tunable> <value>0</value> <desc>Set ICMP Limits</desc> <tunable>net.inet.icmp.icmplim</tunable> <value>750</value> <desc>TCP Offload Engine</desc> <tunable>net.inet.tcp.tso</tunable> <value>0</value> <desc>TCP Offload Engine - BCE</desc> <tunable>hw.bce.tso_enable</tunable> <value>0</value></sysctl> <system><optimization>normal</optimization> <hostname>pfsense</hostname> <domain>local</domain> <group><name>all</name> <description>All Users</description> <scope>system</scope> <gid>1998</gid> <member>0</member></group> <group><name>admins</name> <description>System Administrators</description> <scope>system</scope> <gid>1999</gid> <member>0</member> <priv>page-all</priv></group> <user><name>admin</name> <fullname>System Administrator</fullname> <scope>system</scope> <groupname>admins</groupname> <password>$1$dSJImFph$GvZ7.1UbuWu.Yb8etC0re.</password> <uid>0</uid> <priv>user-shell-access</priv></user> <nextuid>2000</nextuid> <nextgid>2000</nextgid> <timezone>Etc/UTC</timezone> <time-update-interval></time-update-interval> <timeservers>0.pfsense.pool.ntp.org</timeservers> <webgui><protocol>http</protocol></webgui> <disablenatreflection>yes</disablenatreflection> <dns1gwint>wan</dns1gwint> <dns2gwint>none</dns2gwint> <dns3gwint>none</dns3gwint> <dns4gwint>none</dns4gwint> <dnsserver>200.48.225.130</dnsserver> <dnsserver>200.31.108.106</dnsserver> <dnsallowoverride></dnsallowoverride></system> <interfaces><lan><if>xl0</if> <ipaddr>192.168.55.1</ipaddr> <subnet>24</subnet> <media></media> <mediaopt></mediaopt> <bandwidth>100</bandwidth> <bandwidthtype>Mb</bandwidthtype> <descr>LAN</descr></lan> <opt1><descr>OPT1</descr> <if>xl1</if> <enable><ipaddr>192.168.20.34</ipaddr> <subnet>24</subnet> <gateway>americatel</gateway> <spoofmac></spoofmac></enable></opt1> <wan><if>vr0</if> <mtu><media></media> <mediaopt></mediaopt> <bandwidth>100</bandwidth> <bandwidthtype>Mb</bandwidthtype> <descr>WAN</descr> <enable><ipaddr>192.168.1.251</ipaddr> <subnet>24</subnet> <gateway>telefonica</gateway> <spoofmac></spoofmac></enable></mtu></wan></interfaces> <staticroutes></staticroutes> <pppoe><username></username> <password></password></pppoe> <pptp><username></username> <password></password> <local></local></pptp> <dhcpd><lan><enable><range><from>192.168.55.100</from> <to>192.168.55.200</to></range></enable></lan></dhcpd> <pptpd><localip></localip></pptpd> <dnsmasq><enable></enable></dnsmasq> <snmpd><syslocation></syslocation> <syscontact></syscontact> <rocommunity>public</rocommunity></snmpd> <diag><ipv6nat><ipaddr></ipaddr></ipv6nat></diag> <nat><ipsecpassthru><enable></enable></ipsecpassthru></nat> <filter><rule><id><type>pass</type> <tag><tagged><direction>out</direction> <quick>yes</quick> <floating>yes</floating> <max-src-nodes><max-src-states><statetimeout><statetype>keep state</statetype> <source> <address>127.0.0.1/24</address> <destination><any></any></destination> <descr><gateway>balanceo</gateway></descr></statetimeout></max-src-states></max-src-nodes></tagged></tag></id></rule> <rule><id><type>pass</type> <interface>lan</interface> <max-src-nodes><max-src-states><statetimeout><statetype>keep state</statetype> <source> <network>lan</network> <destination><any></any></destination> <descr>Default allow LAN to any rule</descr> <gateway>balanceo</gateway></statetimeout></max-src-states></max-src-nodes></id></rule></filter> <shaper></shaper> <ipsec><preferredoldsa></preferredoldsa></ipsec> <aliases></aliases> <proxyarp></proxyarp> <cron><minute>0</minute> <hour>*</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 newsyslog <minute>1,31</minute> <hour>0-5</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 adjkerntz -a <minute>1</minute> <hour>3</hour> <mday>1</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /etc/rc.update_bogons.sh <minute>*/60</minute> <hour>*</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /usr/local/sbin/expiretable -v -t 3600 sshlockout <minute>1</minute> <hour>1</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /etc/rc.dyndns.update <minute>*/60</minute> <hour>*</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /usr/local/sbin/expiretable -v -t 3600 virusprot <minute>*/5</minute> <hour>*</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /usr/local/bin/checkreload.sh <minute>*/5</minute> <hour>*</hour> <mday>*</mday> <month>*</month> <wday>*</wday> <who>root</who> <command></command>/usr/bin/nice -n20 /etc/ping_hosts.sh</cron> <rrd><enable></enable></rrd> <revision><time>1254853809</time></revision> <gateways><gateway_item><interface>wan</interface> <name>telefonica</name> <gateway>192.168.1.1</gateway> <descr><monitor>200.48.225.130</monitor> <defaultgw></defaultgw></descr></gateway_item> <gateway_item><interface>opt1</interface> <name>americatel</name> <gateway>192.168.20.1</gateway> <descr><monitor>200.48.225.146</monitor></descr></gateway_item> <gateway_group><name>balanceo</name> telefonica|1 americatel|1 <trigger>down</trigger></gateway_group></gateways> <installedpackages><package><name>squid</name> <descr>High performance web proxy cache.</descr> <website>http://www.squid-cache.org/</website> <category>Network</category> <version>2.6.21_10</version> <status>Stable</status> <required_version>1.2.1</required_version> <maintainer>fernando@netfilter.com.br seth.mos@xs4all.nl mfuchs77@googlemail.com</maintainer> <depends_on_package_base_url>http://files.pfsense.org/packages/7/All/</depends_on_package_base_url> <depends_on_package>squid-2.6.21.tbz</depends_on_package> <depends_on_package>squid_radius_auth-1.10.tbz</depends_on_package> <depends_on_package>openldap-client-2.4.10.tbz</depends_on_package> <config_file>http://www.pfsense.org/packages/config/squid/squid.xml</config_file> <configurationfile>squid.xml</configurationfile> <noembedded>true</noembedded> <filter_rule_function>squid_generate_rules</filter_rule_function></package> <menu> <name>Proxy server</name> <tooltiptext>Modify the proxy server's settings</tooltiptext> Services <url>/pkg_edit.php?xml=squid.xml&id=0</url> </menu> <tab><text>General settings</text> <url>/pkg_edit.php?xml=squid.xml&id=0</url> <active></active></tab> <service><name>squid</name> <rcfile>squid.sh</rcfile> <executable>squid</executable> <description>Proxy server Service</description></service> <squid><config><active_interface>lan</active_interface> <allow_interface>on</allow_interface> <transparent_proxy>on</transparent_proxy> <private_subnet_proxy_off><defined_ip_proxy_off><log_enabled><log_dir>/var/squid/log</log_dir> <log_rotate><proxy_port>3128</proxy_port> <icp_port><visible_hostname>localhost</visible_hostname> <admin_email>admin@localhost</admin_email> <error_language>English</error_language> <disable_xforward><disable_via><uri_whitespace>strip</uri_whitespace> <dns_nameservers><disable_squidversion></disable_squidversion></dns_nameservers></disable_via></disable_xforward></icp_port></log_rotate></log_enabled></defined_ip_proxy_off></private_subnet_proxy_off></config></squid></installedpackages></pfsense>Best regards
-
Is it not possible to make the gateway-group default? I don't have Multi-WAN, so its only a thought… Maybe wrong.
Well Squid only work with the local IP 127.0.0.1 then Squid works with default gateway, it doesn't work with gateway-group, you can see in the picture below an example, if I take OPT1 like defaul gateway then Squid works with OPT1 interface, if you take WAN like default gateway then Squid work with Wan interface.
-
The only problem i see in your config is that you might want to try
from:<address>127.0.0.1/24</address>
to:
<address>127.0.0.1/8</address>
afterwards i need access to the system.
-
All work fine, SQUID + LOAD BALANCING, but apparently, SQUID only works with default gateway, if you remove WAN or turn off ADSL telefonica, you will not have internet but will continue to operate Squid
PLEASE SEE THE PICTURE BELOW.
SQUID + LOAD BALANCING DOESN'T WORK
-
Do you by any chance have sticky-connections active?!
-
:-[ I don't have sticky-connection you can see below in the picture, now I prove again the squid and load balancing and nothing
-
anyone have this working?
-
All work fine, SQUID + LOAD BALANCING, but apparently, SQUID only works with default gateway, if you remove WAN or turn off ADSL telefonica, you will not have internet but will continue to operate Squid
PLEASE SEE THE PICTURE BELOW.
SQUID + LOAD BALANCING DOESN'T WORK
I know this is an old post but I'm seeing the same symptoms as the OP, squid works fine with load balancing but if the default gateway modem is turned off/unplugged, clients are unable to browse the internet.
Slam
-
How you are creating rules?
-
Hi Ermal
Action: PASS / Quick: TICKED / Interface: NONE / Direction: OUT / Protocol: TCP / Source: 127.0.0.1 / Dst: ANY / Dst Port Range: 80 / GW: Multiwan
I think that rule is correct, appologies if I have it wrong!
Thanks
Slam
EDIT
I have squid bound to LAN/CP1/CP2, all 3 interfaces have gateway also set to Multiwan in their respective rules, I dont know if thats relevant.
-
You need direction out and source any.
You can even try with direction out and source WAN ip to try to tighten it. -
@ermal:
You need direction out and source any.
You can even try with direction out and source WAN ip to try to tighten it.I've tried the above suggestions without any luck.
On my testbed, I have 2 wans, currently wan2 is set as default gw, the interface is hooked up to a hub so that the interface shows as up as active, although the hub is not connected to anything. (I needed its modem for a production box)
wan1 is connected to a modem and not set for default gw.
I have tried almost everything, from unticking "Block private/bogon networks" to setting static IP and DNS on my client machine.
I hope the following can helps you track down the problem, if you need anymore information Ill be happy to help.
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 78.xx.xx.xx UGS 2 70 fxp1 <–-wan2 gw set as default
10.0.0.0/24 link#10 U 0 0 re1_vl
10.0.0.1 link#10 UHS 0 0 lo0
10.0.1.0/24 link#11 U 0 0 re1_vl
10.0.1.1 link#11 UHS 0 0 lo0
10.0.2.0/24 link#12 U 0 0 re1_vl
10.0.2.1 link#12 UHS 0 0 lo0
10.0.20.0/24 link#13 U 0 0 re1_vl
10.0.20.1 link#13 UHS 0 0 lo0
78.xx.xx.xx/20 link#2 U 2 463 fxp1 <–-wan2 gw
78.xx.xx.xx link#2 UHS 0 0 lo0 <–-wan2 IP
94.xx.xx.xx/20 link#1 U 2 1051 fxp0 <–-wan1 gw
94.xx.xx.xx link#1 UHS 0 0 lo0 <–-wan1 IP
127.0.0.1 link#6 UH 0 19 lo0
127.0.0.2 127.0.0.1 UHS 0 0 lo0
192.168.5.0/24 link#3 U 1 25807 re0
192.168.5.1 link#3 UHS 0 0 lo0
192.168.50.0/24 link#9 U 0 0 re1_vl
192.168.50.1 link#9 UHS 0 0 lo0
208.67.220.220 94.xx.xx.xx UGHS 1 25 fxp0 <–-94.xx.xx.xx wan1 gw
208.67.222.222 78.xx.xx.xx UGHS 0 29 fxp1 <–-78.xx.xx.xx wan2 gwconfig.xml rule
<rule><id><type>pass</type>
<tag><tagged><direction>out</direction>
<quick>yes</quick>
<floating>yes</floating>
<max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype>
<os><nosync><protocol>tcp</protocol>
<source>
<any><destination><any><port>80</port></any></destination>
<descr>Load Balance Squid</descr>
<gateway>multiwan</gateway></any></nosync></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule>pfctl -sr
NOTE: 94.xx.xx.xx is wan1 gwpass out quick route-to (fxp0 94.xx.xx.xx) inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: Load Balance Squid"
pfctl -sa Filter rules:
NOTE: 94.xx.xx.xx is wan1 gwpass out quick route-to (fxp0 94.xx.xx.xx) inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: Load Balance Squid"
Counters:
USER_RULE: Load Balance Squid 6 96 5040 0 0 96 5040
pfctl -s rules -vv
NOTE: 94.xx.xx.xx is wan1 gw@119 pass out quick route-to (fxp0 94.xx.xx.xx) inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: Load Balance Squid"
[ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ]
[ Inserted: uid 0 pid 28028 ]rules.debug
pass out quick $multiwan proto tcp from any to any port 80 keep state label "USER_RULE: Load Balance Squid"
I dont know how much of the above is relevant, but I thought I'd post anyway ;D
Thanks in advance
Slam
-
Are you using Manual Outbound NAT?
pfctl -vvsr is the output i want to see actually.Do you have issues on traffic with this rule?
Please show me even a pfctl -vvsnWhile there can you try enable manual outbound NAT and add a rule with source any and external your wan1(94…../20) gw?
See if it works. -
Ive included output for the those commands
pfctl -vvsr - command1.txt
pfctl -vvsn - command2.txtI dont use outbound nat and dont even know how to use it, I got stuck when I tried to follow your comments, I've sent you a pm with details if you like access to the box yourself.
Thanks
Slam
-
Check out now it should be ok from what i see!?
If you want to tighten you can change the any in the outbound nat rules at the end to the respective interfaces subnet.
-
Ive just gave it another go, my browser just sits there trying to access the internet without any luck
is it ok to leave those other rules in outbound nat? will they interfere with the rules I have for the interfaces? shall i reboot and try?
Thanks
Slam