Squid 3 and squidguard problems



  • I'm using the squid 3 package now since a while in conjunction with squidguard without any problem.
    Due to a hardware-crash i had to reinstall all again (2.0 beta5, Jan 31 install-date, updated to Feb 03.)

    First what happened was the fact that squid doesn't work any more in transparent mode. :( If set to transparent, no more internet-surfing. Looking with pkg_info, i saw that the squid 2.7.9 was installed by squidguard. This packages reinstall before the squid-package, so maybe here is one problem.

    I'm not shure if i should open another thread about the squidguard-auto-install-squid-thing…

    Other thing are this log-entries:

    Feb 3 10:44:03	squid[11575]: Squid Parent: child process 12030 started
    Feb 3 10:44:03	php: : Starting Squid
    Feb 3 10:43:48	php: : The command '/usr/local/sbin/squid -k kill' returned exit code '1', the output was '2011/02/03 10:43:48| aclParseAclLine: WARNING: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl" 2011/02/03 10:43:48| squid.conf line 77: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:48| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression 2011/02/03 10:43:48| squid.conf line 78: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:48| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression squid: ERROR: No running copy'
    Feb 3 10:43:43	php: : The command '/usr/local/sbin/squid -k shutdown' returned exit code '1', the output was '2011/02/03 10:43:43| aclParseAclLine: WARNING: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl" 2011/02/03 10:43:43| squid.conf line 77: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:43| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression 2011/02/03 10:43:43| squid.conf line 78: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:43| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression squid: ERROR: No running copy'
    Feb 3 10:43:43	php: : Creating squid cache subdirs in /var/squid/cache
    

    The same expressions worked before without any glitch! Nasty

    A manual deinstall of the "old" squid:

    equired by these other packages
    and may not be deinstalled:
    squidGuard-1.4_2
    #: pkg_delete  -f squid-2.7.9
    pkg_delete: package 'squid-2.7.9' is required by these other packages
    and may not be deinstalled (but I'll delete it anyway):
    squidGuard-1.4_2
    pkg_delete: unable to completely remove directory '/usr/local/libexec/squid'
    pkg_delete: unable to completely remove directory '/usr/local/etc/squid/errors'
    pkg_delete: unable to completely remove directory '/usr/local/etc/squid'
    pkg_delete: file '/usr/local/etc/rc.d/squid' doesn't exist
    pkg_delete: couldn't entirely delete package (perhaps the packing list is
    incorrectly specified?)
    ===> post-deinstallation information for squid-2.7.9:
    
         Note:
         Squid related user accounts and groups were not removed.
    
         To remove the 'squid' user and the 'squid' group which were
         created by a default installation of this package, run
    
         pw userdel -n squid -u 100
    
         In order to ease updates the cache and log directories
         and all configuration files modified by you were preserved.
    
         Please remove them manually if you do not want to use
         Squid any longer.
    
    

    After that i removed squid3, installed it again and all works again like before.
    So the big work i have after every update i have to do the same things again.

    Who has to review his/her code now? The squidguard-maintainer? Or is that a package-manager-problem? Then this thread would be better in the 2.0 forum. Not shure…

    Oh, a maybe silly question: Do i have to enable the loopback-device too, or only the interfaces which are used by my LAN? (LAN; WIFI; OPTx)

    edit: I was too early: squid alone starts, squidguard not. If i try to start squidguard too, both services die. Last log-entry from squidguard was "servicing requests". No message about its death.

    squid logs this message now:

    Feb 3 14:05:42	php: : SQUID is installed but not started. Not installing "filter" rules.
    Feb 3 14:05:41	php: : SQUID is installed but not started. Not installing "nat" rules.
    Feb 3 14:05:40	php: /pkg_edit.php: The command '/usr/local/sbin/squid -D' returned exit code '1', the output was '2011/02/03 14:05:40| WARNING: -D command-line option is obsolete. 2011/02/03 14:05:40| WARNING: Netmasks are deprecated. Please use CIDR masks instead. 2011/02/03 14:05:40| WARNING: IPv4 netmasks are particularly nasty when used to compare IPv6 to IPv4 ranges. 2011/02/03 14:05:40| WARNING: For now we will assume you meant to write /27 2011/02/03 14:05:40| ERROR: '0.0.0.0/0.0.0.0' needs to be replaced by the term 'all'. 2011/02/03 14:05:40| SECURITY NOTICE: Overriding config setting. Using 'all' instead. 2011/02/03 14:05:40| WARNING: (B) '::/0' is a subnetwork of (A) '::/0' 2011/02/03 14:05:40| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable 2011/02/03 14:05:40| WARNING: You should probably remove '::/0' from the ACL named 'all' 2011/02/03 14:05:40| WARNING: Netmasks are deprecated. Please use CIDR masks instead. 2011/02/03 14:05:40| WARNING: IPv4 netmasks a
    Feb 3 14:05:40	squid: Bungled squid.conf line 62: reply_body_max_size 0 allow all
    Feb 3 14:05:40	php: /pkg_edit.php: Starting Squid
    


  • You use HDD full installed version of the pfsense?



  • Its a full install. It doesn't work even after deinstalling/reinstalling.



  • Deinstall (1)squidGuard/(2)squid from GUI, and type pkg_info from console. Result pkg_info pls post here.



  • What happens on deinstallation of squidguard:

    Backing up libraries... 
    Removing package...
    Starting package deletion for squidGuard-1.4_2...done.
    Starting package deletion for cyrus-sasl-2.1...done.
    Starting package deletion for openldap-client-2.4...done.
    Starting package deletion for openssl-1.0...done.
    Starting package deletion for squid-2.7...done.
    Starting package deletion for db3-3.3...done.
    Starting package deletion for db41-4.1.25_4...done.
    Starting package deletion for db3-3.3.11_3,1...done.
    Starting package deletion for cyrus-sasl-2.1.23...done.
    Removing squidGuard components...
    Tabs items... done.
    Menu items... done.
    Services... done.
    Loading package instructions...
    

    The squid 2.7 here… i had installed squid3!

    output of pkg_info after deinstallation:

    aspell-0.60.6_3    Spelling checker with better suggestion logic than ispell
    bsdinstaller-2.0.2011.0131 BSD Installer mega-package
    expat-2.0.1_1      XML 1.0 parser written in C
    gettext-0.18.1.1    GNU gettext package
    grub-0.97_4        GRand Unified Bootloader
    joe-3.7,1          Joe's Own Editor
    jpeg-8_3            IJG's jpeg compression utilities
    libevent-1.4.14b_1  Provides an API to execute callback functions on certain ev
    libiconv-1.13.1_1  A character set conversion library
    lightsquid-1.8_2    A light and fast web based squid proxy traffic analyser
    p7zip-9.13          File archiver with high compression ratio
    perl-5.10.1_2      Practical Extraction and Report Language
    perl-5.10.1_3      Practical Extraction and Report Language
    unbound-1.4.8      A validating, recursive, and caching DNS resolver
    zip-3.0            Create/update ZIP files compatible with pkzip



  • The squid 2.7 here... i had installed squid3!
    

    Yes - it's a SG depences. Need fix.
    You can try install SG before and squid 3 then.



  • @dvserg:

    The squid 2.7 here... i had installed squid3!
    

    Yes - it's a SG depences. Need fix.
    You can try install SG before and squid 3 then.

    There is a missing units in the line that creates the .conf file for Squid3. in the squid.inc file

    The line

            $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " deny all\n";
    
    

    Should read

            $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " all\n";
    
    

    removing the "deny".  In Squid version 3 the use of allow or deny are no longer valid for this directive.

    the squid.inc file can be found in the /usr/local/pkg directory

    Strangely though I was also having problems when the limit was set to 0, which according to the Squid documentation should be valid, however I would always receive a "request to large" error.  I added a qualifier ( != 0) so that the line is only added if required.

    if ($down_limit != 0) $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " all \n";
    


  • kewl!

    That change should go into the package. Thats that nasty error which made me scratch my head in a way my balls never like…
    I took out the whole line in my config to get rid of the error at last. What i can tell is that i didn't see any failures in the function of squid3 without that line.

    Thanks a lot for your finding!!!!


  • Rebel Alliance Developer Netgate



  • @jimp:

    Here ya go:
    https://github.com/bsdperimeter/pfsense-packages/commit/54c49bf2b5358b35602cae3cf6a9fead0ba886e5

    thank you very much, just installed a new router, install the Squid3 package and viola, the Change was made.  :)

    Woot, I don't have to make the change manually any more.

    If I find any other bugs, I will post them here.



  • Newb question here..
    I've been searching and can't seem to find an answer on version 3 vs 2.  I can see posts from over a year ago on squid-cache about the comparison. v3 was in development, and 2 was stable.
      Since then, I see v3 has stable versions (3.1).
    So does this not mean instead of having both 2.x and 3.0.8 in pfsense, the path forward should actually be to remove the 2.x package and move into a stable 3.x release?
      Not pushing and developers (although I'd contribute to a bounty), just trying to make sense of all the versions.
      Am I understanding this right?


  • Rebel Alliance Developer Netgate

    You are somewhat confusing the stability of squid with the stability of the pfSense package.

    We still have a squid 2.7.x package for pfSense because it works, and it works well, and there isn't a large compelling reason to rush into 3.

    The squid 3 pfSense package is largely untested and most likely still needs work. Until the squid 3 package for pfSense is proven to be stable, and work well with squidGuard/HAVP/whatever, then it will likely still remain in limbo.

    There are FreeBSD ports still for Squid 2.x, 3.0.x, and 3.1.x, and 2.x is still the default as far as I can see there.



  • Thanks for the clarification.  I'm running 2.X right now and it's running with no issues.  I've always been confused as to the point of the 3.x packages.


Locked