Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid3 - New GUI with sync, normal and reverse proxy

    Cache/Proxy
    104
    428
    469.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Q
      quetzalcoatl
      last edited by

      Something else that i´m not sure of is maximum object size in RAM.
      Since i have a 32 Gigabyte RAM system, I dedicate at least 25 Gigs for Squid only and the rest of RAM for the PFSense system
      To make sure that RAM is filled with useful objects i set the maximum object size in ram as 4 Megabytes.

      I´m not sure about the limits because in squid documents they always talk about minimum (0) and default which is a ridiculous (32kb) if i'm not wrong.

      So i wonder if squid can properly work with a maximum object size in RAM of 4096kb.

      Also, when the system is restarted, are those smaller than 4096kb objects reloaded from the Disk? or no objects is ever saved in the disk if smaller than maximum RAM object size?

      I would like to have squid cache saving absolutely everything cacheable inside the disk and then after a system restart reload all cached objects smaller than maximum RAM object size back into the RAM at each restart.

      Maybe squid already does something like that but if not, i think that COSS (cyclic object storage system) already does it.

      Lusca can optionally support COSS but they say it's not very reliable even if sometimes it's even faster than squid's "aufs"

      1 Reply Last reply Reply Quote 0
      • N
        Nachtfalke
        last edited by

        @quetzalcoatl

        1.) To use squid in LoadBalancing /MultiWAN you just have to do additional configuration. There are many forum posts about that and how to configure that.

        RAM is more a kind of "cache" so that squid does not have to access the HDD too much. After reboot the RAM will be empty. The RAM will be filled again automatically with files which are most requested by users of the proxy.

        My setting an intention is as following:
        The minimum file size must be 128kb. I do not want to cache smaller files because getting them from cache will ber perhaps not much faster than downloading them (depending on your WAN connection).
        The maximum object size in RAM is 512kb. Caching many small files on HDD will cause many read/write accesses on HDD which cost time so I will cache small files in my RAM which is very fast.
        The maximum object size is 1GB because I want to cache windows updates and so on but I do not want to have a ServicePack in my RAM which wasts much space and will not be needed so often.

        As memory replacement policy I use:

        
        Heap GDSF: Greedy-Dual Size Frequency - The Heap GDSF policy optimizes object-hit rate by keeping smaller, popular objects in cache. It achieves a lower byte hit rate than LFUDA though, since it evicts larger (possibly popular) objects. 
        
        

        As disk cache replacement I use:

        
        Heap LFUDA: Least Frequently Used with Dynamic Aging - The Heap LFUDA policy keeps popular objects in cache regardless of their size and thus optimizes byte hit rate at the expense of hit rate since one large, popular object will prevent many smaller, slightly less popular objects from being cached. 
        
        
        1 Reply Last reply Reply Quote 0
        • Q
          quetzalcoatl
          last edited by

          Nachtfalke

          Thanks for your reply

          Those settings that you are using are very efficient and are very similar to what i use. But it think they can be improved a little:
          RAM is still faster than any WAN even for very small files, so i see no reason to limit the minimum object size in RAM setting it above 0KB.

          If you have a big hard drive you can have a maximum object size in Disk bigger than 1 GB.
          I actually keep the maximum object size to 9999 Megabytes so if a user downloads a double layer dvd, it will still be there for anyone to re-download it at huge LAN speeds.
          This is good when you are using a Terabyte or bigger hard drive dedicated to squid.

          I did notice that RAM gets filled slowly after the system is restarted, but it would be great to have it filled quickly with a synchronous cache backup done in the hard drive.
          But that may be too complicated. Some other caches actually do this.

          Also, i saw in other forum posts here about the load balancing with squid but many are still complaining about having only one WAN used when squid service is active.
          I found a post that had a way to make squid work with both WANS and it did work but it crashed squid several times a week.

          That is why i´m using two machines, one with pfsense with pppoe wan connections(to make wan connection faster than router), load balancing and failover but without squid,
          and a second machine with pfsense and just squid cache service.

          1 Reply Last reply Reply Quote 0
          • N
            Nachtfalke
            last edited by

            Hi,

            can someone help me with that:
            http://www.squid-cache.org/Doc/config/accept_filter/

            
            2012/10/30 17:27:44| Installing accept filter 'httpready' on FD 359
            2012/10/30 17:27:44| SO_ACCEPTFILTER 'httpready': (2) No such file or directory
            
            

            I enabled this on my squid.conf

            
            accept_filter httpready
            
            

            Is this possible with pfsense (kernel) and squid2 or squid3 ?

            Thank you!

            1 Reply Last reply Reply Quote 0
            • N
              Nachtfalke
              last edited by

              Another question to the squid users.

              The description for http://www.squid-cache.org/Doc/config/minimum_object_size/ is:

              
              Objects smaller than this size will NOT be saved on disk.  The
              value is specified in kilobytes, and the default is 0 KB, which
              means there is no minimum.
              
              

              Will the objects smaller than 128KB be cache in mem or not if I set the value to 128 ? I want to prevent that to many very small files will be cached on disk because this causes so much I/O on disk. These small files should be placed in RAM which is much faster.

              1 Reply Last reply Reply Quote 0
              • G
                griffin
                last edited by

                Is there a guide somewhere to get this Reverse Proxy working in pfsense.. ?
                I can't get the right combination of NAT, firewall, ports and interfaces.

                Maybe an example?
                Has Anyone really got it working?

                thanks!

                -Griffin

                1 Reply Last reply Reply Quote 0
                • T
                  Truster
                  last edited by

                  Hi,

                  i found a bug/typo in /usr/local/pkg/squid_reverse.inc:136

                  RPC-over-HTTPS doen't work correct.

                      136                         if($settngs['reverse_owa_rpchttp'])
                      137                                         array_push($owa_dirs,'rpc/rpcproxy.dll','rpcwithcert/rpcproxy.dll');
                  
                  

                  correct is:

                      136                         if($settings['reverse_owa_rpchttp'])
                      137                                         array_push($owa_dirs,'rpc/rpcproxy.dll','rpcwithcert/rpcproxy.dll');
                  
                  

                  tooks lot of time to get Outlook RPC-over-HTTPS working :P

                  1 Reply Last reply Reply Quote 0
                  • marcellocM
                    marcelloc
                    last edited by

                    @Truster:

                    i found a bug in /usr/local/pkg/squid_reverse.inc:136

                    I've pushed the fix for this typo, thank's for the feedback!  :)

                    Treinamentos de Elite: http://sys-squad.com

                    Help a community developer! ;D

                    1 Reply Last reply Reply Quote 0
                    • M
                      moh10ly
                      last edited by

                      Could you please post your configuration ? I would like to host my webserver through squid reverse proxy and it didn't work for me..

                      Power is Knowledge.

                      1 Reply Last reply Reply Quote 0
                      • T
                        Truster
                        last edited by

                        1st: You have to correct the typo as descripted in my post before…

                        I configured it like this:
                        reverse Proxy Interface: WAN
                        External FQDN webmail.domain.tld
                        Reset TCP connections if request is unauthorized: active
                        Enable HTTPS reverse proxy: Active
                        reverse HTTPS port: 443
                        reverse SSL certificate
                        Ignore internal Certificate validation: active (we use self signed certs)
                        OWA frontend IP address: IP Address four your Frontent OWA Server
                        Enable ActiveSync: yes
                        Enable Outlook Anywhere: yes
                        Enable Exchange WebServices: yes
                        Enable AutoDiscover:yes

                        and of course, you have to open that port in the firewall rules.

                        i'm using OWA/Exch2010...

                        hope, this helps.

                        Edit: if you correct the typo after enabling RPC-over-HTTPS you have to uncheck that box, save the configuration, recheck the box and save it again to apply the changes.

                        1 Reply Last reply Reply Quote 0
                        • marcellocM
                          marcelloc
                          last edited by

                          @Truster:

                          1st: You have to correct the typo as descripted in my post before…

                          Or just install latest package version (2.0.5_6)  ;)

                          Treinamentos de Elite: http://sys-squad.com

                          Help a community developer! ;D

                          1 Reply Last reply Reply Quote 0
                          • Q
                            quetzalcoatl
                            last edited by

                            Thank you marcellok!!!

                            At last, since today, squid works again with both http and https caching everything.

                            I have been waiting for this fix for months!!!

                            Thanks again. May Quetzalcoatl bless you!

                            I wonder who is porting squid to freeBSD.

                            Now squid 3.3 beta is outwith several nice improvements and fixes since 3.2 and 3.1
                            Hopefully some kind guy will port it soon to freeBSD to make it available for pfSense as well.

                            1 Reply Last reply Reply Quote 0
                            • M
                              mauricioniñoavella
                              last edited by

                              Hi marcelloc,

                              I wonder if it works  squid3 with SquidGuard
                              or I can suggest that I can work with squid for Access Control List (ACL)

                              Thanks for your understanding.

                              Regards

                              Mauricio Niño

                              1 Reply Last reply Reply Quote 0
                              • marcellocM
                                marcelloc
                                last edited by

                                @mauricioniñoavella:

                                I wonder if it works  squid3 with SquidGuard

                                Yes, just reinstall squid3 after squidguard install

                                @mauricioniñoavella:

                                or I can suggest that I can work with squid for Access Control List (ACL)

                                On current package version, if acl fields available does not fit your need, you can create manual acls on custom_options

                                Treinamentos de Elite: http://sys-squad.com

                                Help a community developer! ;D

                                1 Reply Last reply Reply Quote 0
                                • P
                                  phil.davis
                                  last edited by

                                  Notification/doco: The latest squid and squid3 have replaced the previous process called proxy_monitor with sqpmon.

                                  [2.1-BETA1][admin@test.mydomain]/root(6): ls -l /usr/local/pkg/sqpmon.sh
                                  -rwxr-xr-x  1 root  wheel  2906 Jan 21 15:28 /usr/local/pkg/sqpmon.sh
                                  [2.1-BETA1][admin@test.mydomain]/root(7): ls -l /usr/local/etc/rc.d/sq*
                                  -rwxr-xr-x  1 root  wheel  335 Jan 21 15:28 /usr/local/etc/rc.d/sqp_monitor.sh
                                  -rwxr-xr-x  1 root  wheel  449 Jan 21 15:46 /usr/local/etc/rc.d/squid.sh
                                  [2.1-BETA1][admin@test.mydomain]/root(8): ps ax | grep sqpmon
                                  83263   0- I      0:00.26 /bin/sh /usr/local/pkg/sqpmon.sh
                                  

                                  If you install the latest squid (2.7.9 pkg v.4.3.2) or squid3 (3.1.20 pkg 2.0.5_8), you will find an sqpmon process appears (squid process monitor) running /usr/local/pkg/sqpmon.sh. It does what proxy_monitor used to do - checks every minute or so to see if squid is still running. The sqp_monitor.sh script is called at startup and shutdown to start and stop it.
                                  This helps out a guy in Ethiopia who was having trouble fetching files with "proxy" in the filename. It also fixes some potential problems with proxy_monitor - it was possible to have a scenario where proxy_monitor.sh would get called at shutdown and actually hang the shutdown, preventing a reboot from ever completing. Now sqpmon works the way that other package component startup/shutdown scripts are designed.
                                  This all works on 2.0.n or 2.1-BETAn systems.

                                  As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                  If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    fragged
                                    last edited by

                                    I want to not cache Steam downloads. Shouldn't this work in "Custom settings" on the General-tab:

                                    
                                    acl steamtest url_regex -i ^http://.*/depot/.*/chunk/.*
                                    cache deny steamtest
                                    
                                    

                                    Squid is still caching Steam downloads (url http://81.171.70.221/depot/107202/chunk/52f888f7848537d725be13719ef9870f0b1da910). My regexp should match the url each time. The IP changes, but /depot/ and /chunk/ are always there.

                                    What am I doing wrong? :F

                                    Regards,
                                    Joona

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      Tikimotel
                                      last edited by

                                      Will we see an update of the squid3 package to the 3.2 branch?

                                      Stable v3.2.7 released on the 1st of Feb.
                                      http://www.squid-cache.org/Versions/v3/3.2/RELEASENOTES.html

                                      Some bugs regarding ipv6 have been fixed in this latest version.
                                      http://www.squid-cache.org/Versions/v3/3.2/changesets/SQUID_3_2_7.html

                                      Still no freshports.org update though, that is still at v3.2.6

                                      p.s. the package name is 3.1.20 and 3.1.22 is rolled out in PBI. (I'm running 2.1 beta)

                                      1 Reply Last reply Reply Quote 0
                                      • marcellocM
                                        marcelloc
                                        last edited by

                                        I'm testing 3.2.6 and SSL filtering this weekend. :)

                                        Treinamentos de Elite: http://sys-squad.com

                                        Help a community developer! ;D

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          Tikimotel
                                          last edited by

                                          I've checked just now, freshports.org has finished porting v3.2.7  ;)

                                          1 Reply Last reply Reply Quote 0
                                          • marcellocM
                                            marcelloc
                                            last edited by

                                            @Tikimotel:

                                            I've checked just now, freshports.org has finished porting v3.2.7  ;)

                                            Compiling and testing

                                            big changelog

                                            Fri 2013-02-01 03:54:21 -0700 Amos Jeffries +23 -6 3.2.7
                                            Thu 2013-01-31 21:58:59 -0700 Amos Jeffries +4 -0 Fix ipv6 enabled pinger.
                                            Thu 2013-01-31 21:57:13 -0700 Amos Jeffries +11 -8 Bug #3687: unhandled exception: c when using interception and peers
                                            Thu 2013-01-31 21:56:07 -0700 Alex Rousskov +2 -0 Bug #3111: Mid-term fix for the forward.cc "err" assertion.
                                            Thu 2013-01-31 21:54:23 -0700 Sebastien Wenske +5 -0 Support OpenSSL NO_Compression option
                                            Tue 2013-01-29 00:00:55 -0700 Amos Jeffries +2 -2 Fix WCCPv2 'comparison between signed and unsigned integer expressions'
                                            Mon 2013-01-28 04:49:40 -0700 Amos Jeffries +1 -1 Bug #3567: Memory leak handling malformed requests
                                            Mon 2013-01-28 04:48:06 -0700 Amos Jeffries +9 -0 Bug #3735: raw-IPv6 domain URLs crash if IPv6-disabled
                                            Mon 2013-01-28 04:45:07 -0700 Amos Jeffries +2 -3 Bug #3732: Fix ConnOpener IPv6 awareness
                                            Mon 2013-01-28 04:41:04 -0700 Amos Jeffries +6 -2 Initialize mem_node fully
                                            Mon 2013-01-28 04:40:12 -0700 Tianyin Xu +6 -0 Bug #3736: Floating point exception due to divide by zero
                                            Mon 2013-01-28 04:29:16 -0700 Alex Rousskov +5 -0 Fix "address.GetPort() != 0" assertion for helpers on FreeBSD (at least).
                                            Mon 2013-01-28 04:25:34 -0700 Amos Jeffries +25 -18 WCCP: Fix memory leak in mask assignment, improve debuggsing.
                                            Mon 2013-01-28 04:23:27 -0700 Amos Jeffries +32 -32 Polish quick_abort feature decision code
                                            Mon 2013-01-28 04:22:25 -0700 Amos Jeffries +1 -0 Fix memory leak in IP address unit test
                                            Mon 2013-01-28 02:59:18 -0700 Amos Jeffries +12 -2 Fix memory leaks in ICMP
                                            Mon 2013-01-28 02:58:16 -0700 Tianyin Xu +1 -1 Bug #3729: 32-bit overflow in parsing 64-bit configuration values
                                            Sun 2013-01-27 22:43:11 -0700 Tomas Hozza +36 -10 Fix various Disk I/O issues in all modules
                                            Sun 2013-01-27 22:40:06 -0700 Francesco Chemolli +2 -2 Fix error in config parser which would mis-assign the sslcrlfile directive.
                                            Sun 2013-01-27 22:39:19 -0700 Francesco Chemolli +54 -26 Plugged memory leaks in digest authentication module
                                            Sun 2013-01-27 21:33:00 -0700 Tianyin Xu +4 -0 Bug #3728: Improve debug for cache_dir
                                            Sun 2013-01-27 21:30:26 -0700 Tomas Hozza +3 -0 Make sure copied strings are properly terminated in snmplib and wccp2
                                            Sun 2013-01-27 21:28:17 -0700 Tomas Hozza +32 -10 Fix various issues in snmplib
                                            Sun 2013-01-27 21:27:32 -0700 Tomas Hozza +31 -11 Fix various issues in smblib
                                            Fri 2013-01-25 02:59:54 -0700 Timo Teras +7 -2 Bug #3678: external acl grace period causes acl lookup failures

                                            Treinamentos de Elite: http://sys-squad.com

                                            Help a community developer! ;D

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.