Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid Package Tuning

    Scheduled Pinned Locked Moved pfSense Packages
    29 Posts 8 Posters 16.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      Nachtfalke
      last edited by

      Hi,

      thanks for your great job.

      Perhaps somebody could help me with updates for Kaspersky for Windows Workstations Updates ?
      If I update on client, it connects to one of this sites:
      http://dnl-01.geo.kaspersky.com
      …
      http://dnl-19.geo.kaspersky.com

      And McAfee seems to connect to:
      update.nai.com

      But I don't know, which extensions will be downloaded :(

      Thanks for your help!

      1 Reply Last reply Reply Quote 0
      • _
        _igor_
        last edited by

        Best thing is looking at your squid-logs. There you have the full path of that update-files. Other way could be a manual download of the respective files via Webbrowser. The rest is copy/paste.

        1 Reply Last reply Reply Quote 0
        • D
          DigitalJer
          last edited by

          SWEET, this is one of my favourite features of pfsense + Squid.

          Nice work, thanks for sharing!

          –------------------------------------------------
          2.4.3-RELEASE (amd64)
          built on Mon Mar 26 18:02:04 CDT 2018
          FreeBSD 11.1-RELEASE-p7
          VM in ESXi 5.5
          1 x 1000baseTX (WAN)
          1 x 1000baseTX (LAN)

          1 Reply Last reply Reply Quote 0
          • S
            sam0t
            last edited by

            Came across this this informative post while searching for troubleshooting guidance with squid and Windows Updates. The problem being that the Windows Updates are not caching, otherwise the Squid is working as intended.

            As suggested in this post, I have copy/pasted the following entry to Squid > Services > Proxy server > General settings > Custom Settings -box

            refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;

            However it does not seem to be working and when I was removing the cache folder and executing the command "squid -k" I get the following error:

            refreshAddToList: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp)': empty (sub)expression

            Any ideas about this?

            1 Reply Last reply Reply Quote 0
            • S
              serangku
              last edited by

              same here … squid -k, report error with that pattern

              some one must validate again with that refresh pattern ...
              or ... simple ignore since squid still running ?

              thanks ...

              1 Reply Last reply Reply Quote 0
              • N
                Nachtfalke
                last edited by

                Hi,

                I am using the following pattern and there is no error in syslog when rebooting or reloading squid:

                refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims;
refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims;
range_offset_limit -1;
                1 Reply Last reply Reply Quote 0
                • _
                  _igor_
                  last edited by

                  I had this errors only with squid3, since some days. With squid 2.7 and squid3 also before never had such problems.

                  1 Reply Last reply Reply Quote 0
                  • N
                    Nachtfalke
                    last edited by

                    I am using squid 2.7

                    1 Reply Last reply Reply Quote 0
                    • S
                      serangku
                      last edited by

                      yes, on sys log gui there is no error with that pattern …
                      please try on pfsense shell with command squid -k rotate or parse

                      
[2.0-BETA5][root@server.localan.vvt]/root(11): squid -k parse
2011/02/08 09:15:39| pattern.conf line 1: refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 2: refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 3: refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 4: refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 5: refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 6: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 7: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 8: refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                      or is simple to ignore it …
                      if ... when it works it works beautifully!

                      thanks

                      1 Reply Last reply Reply Quote 0
                      • N
                        Nachtfalke
                        last edited by

                        Hi,

                        you are right. If I do squid -k rotate than there is the following output

                        2011/02/08 07:50:54| squid.conf line 74: refresh_pattern ([^.]+.|)avg.com/.*\.(b                                   in) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 75: refresh_pattern ([^.]+.|)spywareblaster                                   .net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 76: refresh_pattern ([^.]+.|)symantecliveup                                   date.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 77: refresh_pattern ([^.]+.|)avast.com/.*\.                                   (vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 78: refresh_pattern ([^.]+.|)adobe.com/.*\.                                   (exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 79: refresh_pattern ([^.]+.|)(download|adcd                                   ownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 80: refresh_pattern ([^.]+.|)(download|(win                                   dows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-int                                   o-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub                                   )expression
2011/02/08 07:50:54| squid.conf line 81: refresh_pattern ([^.]+.|)ubuntu.com/.*\                                   .(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                        But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                        1 Reply Last reply Reply Quote 0
                        • S
                          sam0t
                          last edited by

                          I got bit older system at hands, Pfsense 1.2-RELEASE with the following squid packages installed:

                          squid Network 2.6.21_11
                          squidGuard Current: 1.2.0_1 pkg v.1.5 , Installed: 1.2.0_1-2

                          Maybe this is the reason for Windows Updates not caching.

                          1 Reply Last reply Reply Quote 0
                          • S
                            serangku
                            last edited by

                            @Nachtfalke:

                            Hi,

                            you are right. If I do squid -k rotate than there is the following output

                            But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                            also when checking on cachemanager …
                            that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                            then pattern will show on report
                            well … it seem need validate on wiki from squid experience ...

                            thanks

                            1 Reply Last reply Reply Quote 0
                            • K
                              khan
                              last edited by

                              Dear serangku
                              Look at this i m trying to solve this too

                              http://forum.pfsense.org/index.php/topic,33158.0.html

                              1 Reply Last reply Reply Quote 0
                              • S
                                serangku
                                last edited by

                                i'll try tomorrow …

                                thanks for sharing

                                1 Reply Last reply Reply Quote 0
                                • S
                                  scooterdouglas
                                  last edited by

                                  @serangku:

                                  also when checking on cachemanager …
                                  that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                                  then pattern will show on report
                                  well … it seem need validate on wiki from squid experience ...

                                  I'm sorry that everyone is having difficulty, I have only tried the regex on a 2-3 machines and everything appeared to be OK.  I running Squid 2.7 with pfSense 1.2.3 and I'm not getting any errors when I run squid -k rotate.  But that is not to say the regex is "faulty".  I had hoped that more people would have tested it before Jim had updated the wiki page, I think it might be time for myself to try a fresh install of version 1.2.3 or maybe try version 2 beta to see what is wrong.

                                  Thanks.

                                  1 Reply Last reply Reply Quote 0
                                  • N
                                    Nachtfalke
                                    last edited by

                                    Hi,

                                    in the past we talked about caching Kaspersky updates:
                                    I tried this:

                                    
refresh_pattern -i .*kaspersky\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;
refresh_pattern -i .*kaspersky-labs\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;

                                    I think this is working just fine for the "Kaspersky Offline Updater Tool"
                                    http://support.kaspersky.com/updater?level=2

                                    Need more testing if it works with the "normal" updater function of the client software.
                                    Perhaps someone else will find some time for this.

                                    Further I found out, that if we use

                                    range_offset_limit -1

                                    squid is still downloading files even if the user has left a website or canceld some download.
                                    To avoid this, I am using this:

                                    
quick_abort_min 0 KB;
quick_abort_max 0 KB;
quick_abort_pct 100;

                                    For windows updates I am using this without any errors in syslog and with success:

                                    
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;

                                    To avoid, that a user is getting still a "Error 404" if a website wasn't available for a short time and then ist up again because of the high time for negative dns I am using this:

                                    
negative_ttl 5 second;
negative_dns_ttl 5 second;

                                    I am using this all in squid "Custom options".
                                    It would be really nice if someone else could test and verify this or post some other usefull squid options.
                                    For me it was hard to find these parameters and so I wnat to share them and get some feedback.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.