Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    No squid packages will start (user 'squid' not found) on 2.1-DEVELOPMENT

    Scheduled Pinned Locked Moved
    2.1 Snapshot Feedback and Problems - RETIRED
    5
    22
    28.7k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      eXtermia
      last edited by

      I have the exact same errors. I can chown or even squid -z what ever the next time it starts i lose permission to the logs and the cache. Then it dies.

      With Squid 2.x and 3.x as well as the Reverse squid package. This is using 2.1-DEVELOPMENT (amd64)
      built on Mon Dec 12 18:16:13 EST 2011

      /usr/local/sbin(8): squid -z           2012/02/12 03:39:22| Creating Swap Directories
      FATAL: Failed to make swap directory /var/squid/cache/01/00: (13) Permission denied
      Squid Cache (Version 2.7.STABLE9): Terminated abnormally.
      CPU Usage: 0.002 seconds = 0.000 user + 0.002 sys
      Maximum Resident Size: 0 KB
      Page faults with physical i/o: 0

      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(28): chown squid /var/squid/cache/0A
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(29): chown squid /var/squid/cache/0B
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(30): chown squid /var/squid/cache/0C
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(31): chown squid /var/squid/cache/0D
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(32): chown squid /var/quid/cache/0E
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(33): chown squid /var/quid/cache/0F
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(34): squid -z          2012/02/12 03:41:02| Creating Swap Directories
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(35): chown squid /var/squid/logs/
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(36): chown squid /var/squid/logs/access.log
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(37): pw useradd -g proxy -s /sbin/nologin -d /var/squid -n squid
      pw: login name `squid' already exists
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(38): squid -z          2012/02/12 03:47:18| Creating Swap Directories
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(39): squid -k shutdown
      squid: ERROR: No running copy
      [2.1-DEVELOPMENT][root@]/usr/local/sbin(40): squid -k rotate
      squid: ERROR: No running copy
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(42): chown squid /var/squid/logs/store.log
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(43):
      [2.1-DEVELOPMENT][root@pfsense.]/usr/local/sbin(43): 2012/02/12 03:40:24| Creating Swap Directories
      FATAL: Failed to make swap directory /var/squid/cache/09/00: (13) Permission denied

      this goes for the logs too ( i fixed those first then had problems with the cache)

      after I got everything running the service started but the first time I made a change all persmissions were removed again and I lost all

      1 Reply Last reply Reply Quote 0
      • marcellocM
        marcelloc
        last edited by

        chown -R squid /var/squid/cache/

        or

        rm -rf /var/squid/cache/*

        then try to start squid.

        Treinamentos de Elite: http://sys-squad.com

        Help a community developer! ;D

        1 Reply Last reply Reply Quote 0
        • E
          eXtermia
          last edited by

          no go, service starts once but any configuration changes at all results back to

          Feb 22 02:41:24 squid[28812]: Squid Parent: child process 26120 started
          Feb 22 02:41:24 squid[26120]: Cannot open '/var/squid/logs/access.log' for writing. The parent directory must be writeable by the user 'squid', which is the cache_effective_user set in squid.conf.
          Feb 22 02:41:24 squid[28812]: Squid Parent: child process 26120 exited due to signal 6
          Feb 22 02:41:24 kernel: pid 26120 (squid), uid 100: exited on signal 6
          Feb 22 02:41:27 squid[28812]: Squid Parent: child process 26494 started
          Feb 22 02:41:27 squid[26494]: Cannot open '/var/squid/logs/access.log' for writing. The parent directory must be writeable by the user 'squid', which is the cache_effective_user set in squid.conf.
          Feb 22 02:41:27 squid[28812]: Squid Parent: child process 26494 exited due to signal 6
          Feb 22 02:41:27 kernel: pid 26494 (squid), uid 100: exited on signal 6
          Feb 22 02:41:27 squid[28812]: Exiting due to repeated, frequent failures

          strangest thing is I changed the logs  to '/var/squid/logs123' but the error still shows up as   Cannot open '/var/squid/logs/access.log' for writing. even although the /usr/local/etc/squid/squid.conf
          says

          icon_directory /usr/local/etc/squid/icons
          visible_hostname localhost
          cache_mgr admin@localhost
          access_log /var/squid/log123/access.log
          cache_log /var/squid/log123/cache.log
          cache_store_log none
          I have repeatedly chown the log folder
          but as soon as the sevices starts, stops, or reconfigures it goes back to failing

          even disabling the logging makes no change

          1 Reply Last reply Reply Quote 0
          • E
            eXtermia
            last edited by

            even more strange news.

            Still same problems from the web interface but if I

            /usr/local/sbin(127): squid -s

            /usr/local/sbin(128): ps ax | grep squid
            3130  ??  Is    0:00.00 /usr/pbi/squid-amd64/sbin/squid -s
            3284  ??  S      0:00.02 (squid) -s (squid)
            55633  0  R+    0:00.00 grep squid

            /usr/local/sbin(129): squid -k reconfigure

            squid runs but as soon as I change ANYTHING in the web GUI I again have to
            chown -R squid /var/squid/

            and then
            squid -s to get it to run again

            restarting from the GUI it always fails and always lets the permissons on the access.log unaccessible.

            but running the command from shell it works again like above.. a wth moment or what?

            1 Reply Last reply Reply Quote 0
            • P
              phil.davis
              last edited by

              I have all the same issues on a 1G nanobsd system running 2.1-DEVELOPMENT

              After installing Squid from the package installer web interface I had to:

              pw useradd -g proxy -s /sbin/nologin -d /var/squid -n squid
              chown -R squid /var/squid
              mkdir /var/squid/cache
              squid -z

              It looks like it starts OK from the WebGUI, /var/squid/logs/cache.log has good looking stuff in it, the system log looks like it has started a process for the service. But "ps ax | grep squid" doesn't find a process any more! It disappears for some reason that I haven't worked out yet.

              squid -s

              starts it happily and it runs.

              So there are issues with the Squid installation scripts and startup mechanism on 2.1-DEVELOPMENT.

              Note: 2.1-DEVELOPMENT uses the PBI package system. It fetches the squid 2.7.9-1 pbi ffile and installs it.

              As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
              If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

              1 Reply Last reply Reply Quote 0
              • P
                phil.davis
                last edited by

                Since the /var filesystem is only transient on the nanobsd, the /var/squid stuff does not get recreated after a reboot. So, on 2.1-DEVELOPMENT, after startup, to get Squid going, the following is done from a command prompt:

                chown -R squid /var/squid
                mkdir /var/squid/cache
                chown -R squid /var/squid/cache
                squid -z
                squid -s

                The squid username is preserved - that lives in /etc/passwd on the CF card.
                The /var/squid dir got created by something, so there must be some script that is trying to setup things for squid, but doesn't get too far.

                As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                1 Reply Last reply Reply Quote 0
                • jimpJ
                  jimp Rebel Alliance Developer Netgate
                  last edited by

                  That should be all done by squid_resync() that should be run when squid starts at bootup.

                  Next time you reboot, try to do something like this in Diag > Command, PHP exec box:
                  include 'squid.inc';
                  squid_resync();

                  then see if it works.

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  1 Reply Last reply Reply Quote 0
                  • P
                    phil.davis
                    last edited by

                    As suggested, after rebooting, I did:
                    include 'squid.inc';
                    squid_resync();

                    No joy, the system log complains that there is no /var/squid/cache dir.

                    ls -ld /var/squid
                    drwxr-xr-x  5 proxy  proxy  512 Mar  8 11:19 /var/squid

                    The squid dir is owned by proxy, not by squid.

                    After manually resetting the owner, creating /var/squid/cache and doing "squid -z", "squid -s" it is fine. But then after a while (I think after doing other stuff in the web GUI) /var/squid goes back to being owned by proxy and squid does not work any more. So it seems that there is code in webGUI php scripts somewhere that doesn't set the squid owner correctly - if that is fixed then maybe all the downstream effects/problems will be resolved.

                    As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                    If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                    1 Reply Last reply Reply Quote 0
                    • P
                      phil.davis
                      last edited by

                      The difference between 2.0.1 and 2.1-DEVELOPMENT is that the package is installed using a PBI. The "squid" program in /usr/local/sbin is now just a link to:
                      /usr/pbi/squid-i386/.sbin/squid

                      There is a default squid.conf in:
                      /usr/pbi/squid-i386/.etc/squid/squid.conf

                      The system seems to be using this conf file, which specifies cache_effective_user squid - and from that point all the /var/squid file owner issues occur.

                      The conf file that is supposed to be used is /usr/local/etc/squid/squid.conf

                      I modified /usr/local/pkg/squid.inc - on the end of all places that run "/usr/local/sbin/squid -D" add " -f /usr/local/etc/squid/squid.conf"
                      That makes it use the pfSense-specific squid.conf file.

                      There are still places that do "squid -k" commands to get Squid to reread its conf file, and I get some messages about 'squid: ERROR: No running copy' - I think that adding the "-f" parameter means that other checks for the squid process might need to be modified.

                      An easier solution might be to put an actual copy of the squid program into /usr/local/sbin rather than a link, then it might find its conf file OK?

                      As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                      If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                      1 Reply Last reply Reply Quote 0
                      • P
                        phil.davis
                        last edited by

                        I tried putting a real copy of the squid program in /usr/local/sbin
                        That doesn't work, it still uses /usr/pbi/squid-i386/etc/squid/squid.conf
                        It seems that the default squid.conf location is an absolute path hard-coded into the program. I was hoping that it would be a relative path (relative to the location that the squid program was run from), but not so.

                        I have got Squid and SquidGuard running nicely on 2.0.1-DEVELOPMENT by editing /usr/local/pkg/squid.inc
                        (a) Change all the occurrences of "squid -D" to "squid -D -f /usr/local/etc/squid/squid.conf"
                        (b) Change all the occurrences of "squid -k *" to "squid -k * -f /usr/local/etc/squid/squid.conf"
                           (where * is reconfigure, rotate, shutdown, kill)

                        (a) makes it use the correct conf file at startup.
                        (b) makes it find the squid process to change its configuration, rotate log files or stop it.

                        These changes are also needed in:
                        squid_ng.xml
                        squidguard_configurator.inc
                        swapstate_check.php

                        Whoever maintains the squid package, can they make this change (or another better designed one) to squid.inc for 2.1?

                        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                        1 Reply Last reply Reply Quote 0
                        • P
                          phil.davis
                          last edited by

                          I suspect that Squid Traffic Management will not work (but I haven't tested it).
                          /var/squid/logs/cache.log reports unrecognized parameters on squid startup, lines like
                          parseConfigFile: squid.conf:58 unrecognized: 'delay_pools'
                          This happens for delay_pools, delay_class, delay_parameters, delay_initial and delay_access.
                          It looks like squid needs to be compiled with –enable_delay_pools - the supplied version in the pbi maybe does not have this compiler flag set?

                          None of these parseConfigFile messages are in the log on my 2.0.1 nanobsd system.

                          As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                          If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                          1 Reply Last reply Reply Quote 0
                          • P
                            phil.davis
                            last edited by

                            SquidGuard timed rules work on 2.1-DEVELOPMENT.
                            I tried a rule that turned on and off every 10 minutes for a hour or so.
                            /var/squidGuard/log/squidGuard.log contained regular "Info: recalculating alarm in nn seconds" messages.
                            The blocked website became available and blocked as the time changed.
                            (Note that you often have to be careful to clear the browser cache when doing this testing, otherwise you can just be looking at locally-cached data in the client.)
                            On my 2.0.1 nanobsd system, I get "Info: recalculating alarm in nn seconds" messages a couple of times, then they just stop appearing in the log file. It seems to just forget that there are timed rules to calculate.
                            So, it looks like this problem in 2.0.1 is fixed in 2.1

                            As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                            If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                            1 Reply Last reply Reply Quote 0
                            • jimpJ
                              jimp Rebel Alliance Developer Netgate
                              last edited by

                              @phil.davis:

                              I have got Squid and SquidGuard running nicely on 2.0.1-DEVELOPMENT by editing /usr/local/pkg/squid.inc
                              (a) Change all the occurrences of "squid -D" to "squid -D -f /usr/local/etc/squid/squid.conf"
                              (b) Change all the occurrences of "squid -k *" to "squid -k * -f /usr/local/etc/squid/squid.conf"
                                 (where * is reconfigure, rotate, shutdown, kill)

                              (a) makes it use the correct conf file at startup.
                              (b) makes it find the squid process to change its configuration, rotate log files or stop it.

                              These changes are also needed in:
                              squid_ng.xml
                              squidguard_configurator.inc
                              swapstate_check.php

                              Whoever maintains the squid package, can they make this change (or another better designed one) to squid.inc for 2.1?

                              I can do that but I won't have time to get to that today. That should be a safe change to make both on 2.0 and 2.1 though, but it would need to be tested. If someone wants to do that and make a merge request on github we can pull it in, otherwise it'll be sometime next week before I can get to it.

                              @phil.davis:

                              I suspect that Squid Traffic Management will not work (but I haven't tested it).
                              /var/squid/logs/cache.log reports unrecognized parameters on squid startup, lines like
                              parseConfigFile: squid.conf:58 unrecognized: 'delay_pools'
                              This happens for delay_pools, delay_class, delay_parameters, delay_initial and delay_access.
                              It looks like squid needs to be compiled with –enable_delay_pools - the supplied version in the pbi maybe does not have this compiler flag set?

                              None of these parseConfigFile messages are in the log on my 2.0.1 nanobsd system.

                              Yeah that would suggest it's not honoring the build flags in the file. I opened a ticket for that here: http://redmine.pfsense.org/issues/2274

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              1 Reply Last reply Reply Quote 0
                              • P
                                phil.davis
                                last edited by

                                I just put the latest 2G nanobsd image http://iserv.nl/files/pfsense/releng83/i386/pfSense-2.1-DEVELOPMENT-2g-i386-nanobsd-20120319-1526.img.gz onto a CF, ran the wizard and loaded Squid.

                                I get the following warnings in /tmp/PHP_errors.txt

                                [19-Mar-2012 16:57:23 UTC] PHP Warning:  unlink(/etc/squid/squid_radius_auth.conf): No such file or directory in /etc/inc/pkg-utils.inc on line 802
[19-Mar-2012 16:57:23 UTC] PHP Warning:  symlink(): No such file or directory in /etc/inc/pkg-utils.inc on line 803
[19-Mar-2012 16:57:23 UTC] PHP Warning:  unlink(/etc/squid/mime.conf): No such file or directory in /etc/inc/pkg-utils.inc on line 802
[19-Mar-2012 16:57:23 UTC] PHP Warning:  symlink(): No such file or directory in /etc/inc/pkg-utils.inc on line 803
[19-Mar-2012 16:57:23 UTC] PHP Warning:  unlink(/etc/squid/squid.conf): No such file or directory in /etc/inc/pkg-utils.inc on line 802
[19-Mar-2012 16:57:23 UTC] PHP Warning:  symlink(): No such file or directory in /etc/inc/pkg-utils.inc on line 803

                                This comes from the unlink and symlink calls in /etc/inc/pkg-utils.inc

                                exec("/usr/local/sbin/pbi_info | grep {$pkg} | xargs /usr/local/sbin/pbi_info | awk '/Prefix/ {print $2}'",$pbidir);
$pbidir = $pbidir[0];
exec("find /usr/local/etc/ -name *.conf | grep {$pkg}",$files);
foreach($files as $f) {
	$pbiconf = str_replace('/usr/local',$pbidir,$f);
	unlink($pbiconf);
	symlink($f,$pbiconf);
}

                                Perhaps this is part of the reason for the problems finding the squid.conf file?

                                The system log complains about not finding the user 'squid'. It should be using username 'proxy'. This is because the proper conf file is not being used. I will apply the edits in my post above to get squid starting again. But maybe getting the above pkg_utils.inc code fragment working successfully will put symlinks in from the pbi dirs to point at the conf files we want to use in /usr/local/etc/squid - then adding the "-f" parameter to all the squid commands in scripts would not be necessary.

                                As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                1 Reply Last reply Reply Quote 0
                                • P
                                  phil.davis
                                  last edited by

                                  On rebooting the squid now comes up OK (after adding the "-f" parameter to all the squid commands in scripts). The system log has the odd-looking message:

                                  php: : Not calling package sync code for dependency squid of squid because some include files are missing

                                  This seems like not a good thing. I looked in squid.xml but can't see a file there that is not in the dirs on disk. Squid has still come up.

                                  Also, there are 2 squid processes:

                                  59573  ??  INs    0:00.00 /usr/pbi/squid-i386/sbin/squid -D -f /usr/local/etc/s
60077  ??  SN     0:00.27 (squid) -D -f /usr/local/etc/squid/squid.conf (squid)

                                  But maybe getting symlinks to the conf file right in the installation will prevent the 2 processes?

                                  As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                  If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                  1 Reply Last reply Reply Quote 0
                                  • P
                                    phil.davis
                                    last edited by

                                    The main problem turned out to be that squid also includes squid_radius_auth (and libwww). When the code in /etc/inc/pkg-utils.inc uses pbi_info to find packages that are called squid* it finds 2 packages. The xargs pbi_info code doesn't work for 2 package names. And in any case we only want to deal with "squid" in that place.
                                    As a side-issue, the output of the exec goes to $pbidir. The PHP exec doc says that if the output array is non-empty, then the output will be appended to the array. This is a possible problem, because $pbidir is used in other places in pkg-utils.inc and might have text in it already left-over from elsewhere. So it would be safer to use different variable names. You could also do isset() and unset() code before using $pbidir, to make sure it is empty.

                                    Here is some code that worked for me:

                                    exec("/usr/local/sbin/pbi_info | grep {$pkg}- | xargs /usr/local/sbin/pbi_info | awk '/Prefix/ {print $2}'",$pbidirarray);
$pbidir0 = $pbidirarray[0];
exec("find /usr/local/etc/ -name *.conf | grep {$pkg}",$files);
foreach($files as $f) {
	$pbiconf = str_replace('/usr/local',$pbidir0,$f);
	unlink($pbiconf);
	symlink($f,$pbiconf);
}

                                    The changes to pkg-utils.inc are"

                                    a) "grep {$pkg}-" : add the "-" to the package name being looked for. This prevents "squid" matching "squid_radius_auth". In general, the PBI package name is always followed by a dash and then other version, platform etc text. So this will add safety for all PBI installs. This is the 1-character addition that really makes it work!

                                    b) Use unused variables $pbidirarray and $pbidir0 to prevent any possible side-effects of $pbidir that is used elsewhere.

                                    Now I get just 1 squid process started once the system has booted. There is no need to add "-f /usr/local/etc/squid/squid.conf" to a lot of squid scripts. The symlink to squid.conf now gets setup correctly and squid finds the proper pfSense-generated squid.conf. This means that it runs as proxy:proxy and can find its cache OK (or know not to use a cache in the nanobsd case).

                                    Note that there will still be issues for some packages who's names are substring of each other - e.g. if there is a package "auth" and "squid_radius_auth" then looking for "auth-" will all find "squid_radius_auth-". I suspect that this is a real pest all through this sort of code already! At least adding the "-" reduces these cross-package name issues. Someone who has lots of spare time can try and make sub-string selection bullet-proof through the whole package system.

                                    I will put something in RedMine and GitHub about this.

                                    As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                    If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                    1 Reply Last reply Reply Quote 0
                                    • jimpJ
                                      jimp Rebel Alliance Developer Netgate
                                      last edited by

                                      You can anchor the grep.

                                      "^foo-"

                                      Would match only if the line started with foo

                                      So it may work better with:

                                      grep '^{$pkg}-'

                                      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                      Need help fast? Netgate Global Support!

                                      Do not Chat/PM for help!

                                      1 Reply Last reply Reply Quote 0
                                      • P
                                        phil.davis
                                        last edited by

                                        I just added a pull request to add the "^" plus a few other extra checks adding/removing symlinks that tidies up the sequence: install squid, install squidGuard, remove squidGuard, remove squid. It resolves all the package install/remove interactions that I can see, particularly those caused by "squid" being a substring of "squidGuard" and "squidGuard" being a mixed-case package name. Hopefully the changes to /etc/inc/pkg-utils.inc will also fixup generic issues for these cases for other packages.
                                        I have tested with the new 22 March 2012 2G nanobsd FreeBSD 8.3-RC2 snapshot.

                                        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post