Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid Package Tuning

    pfSense Packages
    8
    29
    16.7k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sam0t
      last edited by

      Came across this this informative post while searching for troubleshooting guidance with squid and Windows Updates. The problem being that the Windows Updates are not caching, otherwise the Squid is working as intended.

      As suggested in this post, I have copy/pasted the following entry to Squid > Services > Proxy server > General settings > Custom Settings -box

      refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;

      However it does not seem to be working and when I was removing the cache folder and executing the command "squid -k" I get the following error:

      refreshAddToList: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp)': empty (sub)expression

      Any ideas about this?

      1 Reply Last reply Reply Quote 0
      • S
        serangku
        last edited by

        same here … squid -k, report error with that pattern

        some one must validate again with that refresh pattern ...
        or ... simple ignore since squid still running ?

        thanks ...

        1 Reply Last reply Reply Quote 0
        • N
          Nachtfalke
          last edited by

          Hi,

          I am using the following pattern and there is no error in syslog when rebooting or reloading squid:

          refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims;
refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims;
range_offset_limit -1;
          1 Reply Last reply Reply Quote 0
          • _
            _igor_
            last edited by

            I had this errors only with squid3, since some days. With squid 2.7 and squid3 also before never had such problems.

            1 Reply Last reply Reply Quote 0
            • N
              Nachtfalke
              last edited by

              I am using squid 2.7

              1 Reply Last reply Reply Quote 0
              • S
                serangku
                last edited by

                yes, on sys log gui there is no error with that pattern …
                please try on pfsense shell with command squid -k rotate or parse

                
[2.0-BETA5][root@server.localan.vvt]/root(11): squid -k parse
2011/02/08 09:15:39| pattern.conf line 1: refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 2: refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 3: refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 4: refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 5: refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 6: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 7: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 8: refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                or is simple to ignore it …
                if ... when it works it works beautifully!

                thanks

                1 Reply Last reply Reply Quote 0
                • N
                  Nachtfalke
                  last edited by

                  Hi,

                  you are right. If I do squid -k rotate than there is the following output

                  2011/02/08 07:50:54| squid.conf line 74: refresh_pattern ([^.]+.|)avg.com/.*\.(b                                   in) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 75: refresh_pattern ([^.]+.|)spywareblaster                                   .net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 76: refresh_pattern ([^.]+.|)symantecliveup                                   date.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 77: refresh_pattern ([^.]+.|)avast.com/.*\.                                   (vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 78: refresh_pattern ([^.]+.|)adobe.com/.*\.                                   (exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 79: refresh_pattern ([^.]+.|)(download|adcd                                   ownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 80: refresh_pattern ([^.]+.|)(download|(win                                   dows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-int                                   o-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub                                   )expression
2011/02/08 07:50:54| squid.conf line 81: refresh_pattern ([^.]+.|)ubuntu.com/.*\                                   .(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                  But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                  1 Reply Last reply Reply Quote 0
                  • S
                    sam0t
                    last edited by

                    I got bit older system at hands, Pfsense 1.2-RELEASE with the following squid packages installed:

                    squid Network 2.6.21_11
                    squidGuard Current: 1.2.0_1 pkg v.1.5 , Installed: 1.2.0_1-2

                    Maybe this is the reason for Windows Updates not caching.

                    1 Reply Last reply Reply Quote 0
                    • S
                      serangku
                      last edited by

                      @Nachtfalke:

                      Hi,

                      you are right. If I do squid -k rotate than there is the following output

                      But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                      also when checking on cachemanager …
                      that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                      then pattern will show on report
                      well … it seem need validate on wiki from squid experience ...

                      thanks

                      1 Reply Last reply Reply Quote 0
                      • K
                        khan
                        last edited by

                        Dear serangku
                        Look at this i m trying to solve this too

                        http://forum.pfsense.org/index.php/topic,33158.0.html

                        1 Reply Last reply Reply Quote 0
                        • S
                          serangku
                          last edited by

                          i'll try tomorrow …

                          thanks for sharing

                          1 Reply Last reply Reply Quote 0
                          • S
                            scooterdouglas
                            last edited by

                            @serangku:

                            also when checking on cachemanager …
                            that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                            then pattern will show on report
                            well … it seem need validate on wiki from squid experience ...

                            I'm sorry that everyone is having difficulty, I have only tried the regex on a 2-3 machines and everything appeared to be OK.  I running Squid 2.7 with pfSense 1.2.3 and I'm not getting any errors when I run squid -k rotate.  But that is not to say the regex is "faulty".  I had hoped that more people would have tested it before Jim had updated the wiki page, I think it might be time for myself to try a fresh install of version 1.2.3 or maybe try version 2 beta to see what is wrong.

                            Thanks.

                            1 Reply Last reply Reply Quote 0
                            • N
                              Nachtfalke
                              last edited by

                              Hi,

                              in the past we talked about caching Kaspersky updates:
                              I tried this:

                              
refresh_pattern -i .*kaspersky\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;
refresh_pattern -i .*kaspersky-labs\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;

                              I think this is working just fine for the "Kaspersky Offline Updater Tool"
                              http://support.kaspersky.com/updater?level=2

                              Need more testing if it works with the "normal" updater function of the client software.
                              Perhaps someone else will find some time for this.

                              Further I found out, that if we use

                              range_offset_limit -1

                              squid is still downloading files even if the user has left a website or canceld some download.
                              To avoid this, I am using this:

                              
quick_abort_min 0 KB;
quick_abort_max 0 KB;
quick_abort_pct 100;

                              For windows updates I am using this without any errors in syslog and with success:

                              
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;

                              To avoid, that a user is getting still a "Error 404" if a website wasn't available for a short time and then ist up again because of the high time for negative dns I am using this:

                              
negative_ttl 5 second;
negative_dns_ttl 5 second;

                              I am using this all in squid "Custom options".
                              It would be really nice if someone else could test and verify this or post some other usefull squid options.
                              For me it was hard to find these parameters and so I wnat to share them and get some feedback.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.