Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid 3.3.4 package for pfsense with ssl filtering

    Scheduled Pinned Locked Moved Cache/Proxy
    305 Posts 72 Posters 307.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      Fehler20
      last edited by

      What do you want to do exactly? Because if this Cache rules should work, you nearly block all dynamic content (included google etc.).

      I've selected cache dynamic content and use some custom pattern to cache exe, gif, png etc. (those content does normally not change often without changing URL). I use slightly different override options:

      override-expire ignore-must-revalidate ignore-no-cache ignore-no-store ignore-private

      (ignore-no-cache should not be noticed anymore but it doesn't create any error)

      I get hit rates about 20%

      Moreover there seems to be a problem, if Minimum Disk Cache Size is not set to 0. In that case no caching happens to me.

      Maybe you should change your cache-time, too: http://www.squid-cache.org/mail-archive/squid-users/201211/0279.html

      1 Reply Last reply Reply Quote 0
      • A
        athurdent
        last edited by

        @marcelloc:

        @athurdent:

        Edit: Reinstalled and used amd64 now, still crashes at the first request as soon as I turn on SSL intercept.

        What you get with squid -v on console?
        And with openssl version?

        [2.1-BETA1][root@pfsense-kvm.local-lan]/root(1): squid -v
        Squid Cache: Version 3.3.4
        configure options:  '--with-default-user=squid' '--bindir=/usr/pbi/squid-amd64/sbin' '--sbindir=/usr/pbi/squid-amd64/sbin' '--datadir=/usr/pbi/squid-amd64/etc/squid' '--libexecdir=/usr/pbi/squid-amd64/libexec/squid' '--localstatedir=/var/squid' '--sysconfdir=/usr/pbi/squid-amd64/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid/squid.pid' '--enable-auth' '--enable-build-info' '--enable-loadable-modules' '--enable-removal-policies=lru heap' '--disable-epoll' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-translation' '--enable-auth-basic=DB MSNT MSNT-multi-domain NCSA PAM POP3 RADIUS  fake getpwnam LDAP SASL NIS' '--enable-auth-digest=file' '--enable-external-acl-helpers=file_userip time_quota unix_group LDAP_group kerberos_ldap_group' '--enable-auth-negotiate=kerberos wrapper' '--enable-auth-ntlm=fake smb_lm' '--enable-storeio=diskd rock ufs aufs' '--enable-disk-io=AIO Blocking DiskDaemon IpcIo Mmapped DiskThreads' '--enable-log-daemon-helpers=file' '--enable-url-rewrite-helpers=fake' '--enable-delay-pools' '--enable-ssl' '--with-openssl=/usr' '--enable-ssl-crtd' '--enable-icmp' '--enable-htcp' '--disable-forw-via-db' '--enable-cache-digests' '--enable-wccp' '--enable-wccpv2' '--enable-eui' '--disable-ipfw-transparent' '--enable-pf-transparent' '--disable-ipf-transparent' '--enable-follow-x-forwarded-for' '--disable-ecap' '--enable-icap-client' '--disable-esi' '--enable-kqueue' '--with-large-files' '--prefix=/usr/pbi/squid-amd64' '--mandir=/usr/pbi/squid-amd64/man' '--infodir=/usr/pbi/squid-amd64/info/' '--build=amd64-portbld-freebsd8.3' 'build_alias=amd64-portbld-freebsd8.3' 'CC=cc' 'CFLAGS=-O2 -pipe -I/usr/pbi/squid-amd64/include -I/usr/pbi/squid-amd64/include -I/usr/include -DLDAP_DEPRECATED -fno-strict-aliasing' 'LDFLAGS= -L/usr/pbi/squid-amd64/lib -L/usr/pbi/squid-amd64/lib -pthread -Wl,-rpath=/usr/lib:/usr/pbi/squid-amd64/lib -L/usr/lib' 'CPPFLAGS=-I/usr/pbi/squid-amd64/include' 'CXX=c++' 'CXXFLAGS=-O2 -pipe -I/usr/pbi/squid-amd64/include -I/usr/pbi/squid-amd64/include -I/usr/include -DLDAP_DEPRECATED -fno-strict-aliasing' 'CPP=cpp' --enable-ltdl-convenience
        [2.1-BETA1][root@pfsense-kvm.local-lan]/root(2): openssl version
        OpenSSL 0.9.8y 5 Feb 2013
        
        1 Reply Last reply Reply Quote 0
        • marcellocM
          marcelloc
          last edited by

          [quote]
          OpenSSL 0.9.8y 5 Feb 2013
          [/quote]
          
          I'll try to push the fix I've applied to 2.0.x to freebsd ports.
          My snapshot is older then yours. On mine, squid does not crash with openssl version(OpenSSL 0.9.8q 2 Dec 2010).
          
          

          Treinamentos de Elite: http://sys-squad.com

          Help a community developer! ;D

          1 Reply Last reply Reply Quote 0
          • marcellocM
            marcelloc
            last edited by

            @marcelloc:

            I'll try to push the fix I've applied to 2.0.x to freebsd ports.

            Ports change request sent.

            Since it's merged, I'll ask core team for another compile run.  :)

            att,
            Marcello Coutinho

            Treinamentos de Elite: http://sys-squad.com

            Help a community developer! ;D

            1 Reply Last reply Reply Quote 0
            • Q
              quetzalcoatl
              last edited by

              Squid no caching and getting more data in from WAN than what i get out of LAN (in interface statistics)

              I download the missing files…
              Then i install squid 3.3.4 dev
              Then all in the GUI i just set resolve ipv4 first, enable transparent proxy, and set max object size in RAM 8192KB, Max RAM for squid 6000MB, max HDD 50GB max object size in disk 900MB, enable caching for dynamic stuff selecting youtube and windows updates and leave everything else as it is.
              I don't even add any custom option.

              and as soon as i start the squid service i start getting more data downloaded from WAN than what is served to LAN.

              in fact:

              interface statistics before starting squid service: WAN in: 2.12GB - LAN OUT 2.12GB
              interface statistics after starting squid service: WAN in: 2.93GB - LAN OUT 2.59GB

              I always got more LAN out than WAN in when Squid is caching properly.

              I remember those manual settings that made me get more WAN in than LAN out:
              quick_abort_min
              quick_abort_max
              quick_abort_pct

              but even if i don't use them i get more WAN in than LAN out.

              I even set them as:
              quick_abort_min 0 KB
              quick_abort_max 0 KB
              quick_abort_pct 0

              but nothing improves.

              I can roughly say that with squid 2.7(lusca) i was getting a 15% byte hit ratio, with squid 2.1 a 5% byte hit ratio, and with 3.3.4 0% hit ratio...or should i say -3% hit ratio? (note the negative value)

              it looks that the newer the squid is, the less caching it does.........
              i miss the good days when 100% of windows and office updates were downloaded from the squid cache at 100 megabit speeds!
              at that time it took more to install updates than to download 1Gb of updates from pfsense. even with internet speeds of just 1 megabit!!!!!
              i wonder where is going all this effort to update squid with these results..........or am i just doing massive configuration errors?

              1 Reply Last reply Reply Quote 0
              • marcellocM
                marcelloc
                last edited by

                Disabling dynamic content option on gui and check if on log files you get only TCP_MISS or you start seeing some TCP_HIT.

                Treinamentos de Elite: http://sys-squad.com

                Help a community developer! ;D

                1 Reply Last reply Reply Quote 0
                • Q
                  quetzalcoatl
                  last edited by

                  at last getting some hits.

                  I don't even know how to see squid logs, except than going to diagnostics, edit file, /var/squid/logs/cache.log but i don't see any miss/hit statistics there.

                  anyways as soon as i disabled dynamic caching the "LAN out" number is growing faster than "WAN in" in interface statistics.

                  So at last my squid cache is working.

                  Maybe i got confused because the previous squid 3.3.4 release was not caching for some reason even when dynamic content caching was off.

                  Thanks marcelloc

                  1 Reply Last reply Reply Quote 0
                  • marcellocM
                    marcelloc
                    last edited by

                    @quetzalcoatl:

                    I don't even know how to see squid logs

                    There is a realtime tab on gui.

                    You can also go via console/ssh and do a tail -f /var/squid/logs/access.log

                    Treinamentos de Elite: http://sys-squad.com

                    Help a community developer! ;D

                    1 Reply Last reply Reply Quote 0
                    • Q
                      quetzalcoatl
                      last edited by

                      The only file i have in the "/var/squid/logs" folder is "cache.log"

                      Also if i go to the "real time" tab in the GUI i see this stuff:


                      Max lines: Max. lines to be displayed.
                      String filter:  Enter a grep like string/pattern to filterlog.
                      eg. username, ip addr, url.
                      Use ! to invert the sense of matching, to select non-matching lines.

                      Squid Logs
                      Date IP Status Address User Destination

                      SquidGuard Logs
                      Date-Time ACL Address Host User


                      but everything is empty and i see no data or statistics, just the section titles i just pasted here.

                      1 Reply Last reply Reply Quote 0
                      • marcellocM
                        marcelloc
                        last edited by

                        Did you enabled logging on squid GUI configuration?

                        Treinamentos de Elite: http://sys-squad.com

                        Help a community developer! ;D

                        1 Reply Last reply Reply Quote 0
                        • F
                          Fehler20
                          last edited by

                          I have to report another bug and a problem:

                          1.) If you enable transparent proxy an disable SSL-interception, transparent mode does not work. It seems that there is a problem with the intercept command with transparent connections at the config file. If you change from "192.168.x.x:8080 intercept" to "192.168.x.x:8080 transparent" everything is ok.

                          2.) If you enable the proxy for more than one interface only the first gets access to the internet. For every other interface, access is denied, regardless wether you define an acl or check the option "Allow users on interface".

                          1 Reply Last reply Reply Quote 0
                          • P
                            packeteer
                            last edited by

                            I have a strange issue with this version of squid. Squid service is running but unable to connect on the designated port (3128).

                            Reverted back to the older version.

                            1 Reply Last reply Reply Quote 0
                            • marcellocM
                              marcelloc
                              last edited by

                              @packeteer:

                              I have a strange issue with this version of squid. Squid service is running but unable to connect on the designated port (3128).

                              If you are on 2.0.x, you need to enable ivp6 or update package via pkg_delete and pkg_add from my repo.

                              Treinamentos de Elite: http://sys-squad.com

                              Help a community developer! ;D

                              1 Reply Last reply Reply Quote 0
                              • marcellocM
                                marcelloc
                                last edited by

                                Util freebsd ports are updated, I've pushed squid-3.3.5 pbi files to my repo.

                                This way you can test/use squid3 package on latest 2.1-rc0

                                pfsense 2.1 amd64
                                pbi_delete squid-3.3.4-amd64
                                fetch  http://e-sac.siteseguro.ws/packages/amd64/8/All/squid-3.3.5-amd64.pbi
                                fetch  http://e-sac.siteseguro.ws/packages/amd64/8/All/squid-3.3.5-amd64.pbi.sha256
                                pbi_add –no-checksig squid-3.3.5-amd64.pbi
                                rehash

                                pfsense 2.1 i386
                                pbi_delete squid-3.3.4-i386
                                fetch  http://e-sac.siteseguro.ws/packages/8/All/squid-3.3.5-i386.pbi
                                fetch  http://e-sac.siteseguro.ws/packages/8/All/squid-3.3.5-i386.pbi.sha256
                                pbi_add –no-checksig squid-3.3.5-i386.pbi
                                rehash

                                Treinamentos de Elite: http://sys-squad.com

                                Help a community developer! ;D

                                1 Reply Last reply Reply Quote 0
                                • Q
                                  quetzalcoatl
                                  last edited by

                                  Fehler20 says that if he keeps SSL-interception disabled the transparent proxy doesn't work.

                                  That is the same issue that i believe is happening to me.

                                  I will try to turn on SSL-interception, but i don't know if i have to configure something to make it work properly with the transparent proxy.

                                  And yes, i did forget to turn squid logging on!
                                  Sorry!

                                  1 Reply Last reply Reply Quote 0
                                  • W
                                    wheelz
                                    last edited by

                                    I set up my 2.0.3 version to try the SSL filtering.  I got the HTTP traffic going through squid so that works.  I generated a Test CA cert in Cert Manager and installed in on my test workstation.  However when I go to https://www.google.com it just spins at connecting…  I can telnet to port 3129 and something answers so I think the traffic is getting through the firewall.  Squid service appears to be running but I don't see anything in the logs about the request.  Any help?

                                    1 Reply Last reply Reply Quote 0
                                    • marcellocM
                                      marcelloc
                                      last edited by

                                      @wheelz:

                                      Any help?

                                      Did you replaced squid to 3.3.5 from my repo?

                                      Treinamentos de Elite: http://sys-squad.com

                                      Help a community developer! ;D

                                      1 Reply Last reply Reply Quote 0
                                      • W
                                        wheelz
                                        last edited by

                                        @marcelloc:

                                        Did you replaced squid to 3.3.5 from my repo?

                                        I have now but I'm not sure if I did it right.  I'm getting this when I'm try to run squid:

                                        /libexec/ld-elf.so.1: /usr/lib/librt.so.1: unsupported file layout

                                        I wasn't sure if I needed to do that pbi command either since I'm on 2.0.3.  I tried to see if it would execute but it wasn't there.

                                        1 Reply Last reply Reply Quote 0
                                        • marcellocM
                                          marcelloc
                                          last edited by

                                          @wheelz:

                                          /libexec/ld-elf.so.1: /usr/lib/librt.so.1: unsupported file layout

                                          on 2.0.3, use pkg_add and pkg_delete to get squid 3.3.5

                                          i386
                                          pkg_add -r http://e-sac.siteseguro.ws/packages/8/All/squid-3.3.5.tbz

                                          amd64
                                          pkg_add -r http://e-sac.siteseguro.ws/packages/amd64/8/All/squid-3.3.5.tbz

                                          Treinamentos de Elite: http://sys-squad.com

                                          Help a community developer! ;D

                                          1 Reply Last reply Reply Quote 0
                                          • W
                                            wheelz
                                            last edited by

                                            @marcelloc:

                                            @wheelz:

                                            /libexec/ld-elf.so.1: /usr/lib/librt.so.1: unsupported file layout

                                            on 2.0.3, use pkg_add and pkg_delete to get squid 3.3.5

                                            i386
                                            pkg_add -r http://e-sac.siteseguro.ws/packages/8/All/squid-3.3.5.tbz

                                            amd64
                                            pkg_add -r http://e-sac.siteseguro.ws/packages/amd64/8/All/squid-3.3.5.tbz

                                            Hmm… that's what I did.  It complained about perl and openssl (I think) so I deleted them and let the pkg_add -r install the versions it wanted.  After that it was successfull, however squid -v won't run with the error above.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.