Squid 3 and squidguard problems

_igor_

I'm using the squid 3 package now since a while in conjunction with squidguard without any problem.
Due to a hardware-crash i had to reinstall all again (2.0 beta5, Jan 31 install-date, updated to Feb 03.)

First what happened was the fact that squid doesn't work any more in transparent mode. :( If set to transparent, no more internet-surfing. Looking with pkg_info, i saw that the squid 2.7.9 was installed by squidguard. This packages reinstall before the squid-package, so maybe here is one problem.

I'm not shure if i should open another thread about the squidguard-auto-install-squid-thing…

Other thing are this log-entries:

Feb 3 10:44:03	squid[11575]: Squid Parent: child process 12030 started
Feb 3 10:44:03	php: : Starting Squid
Feb 3 10:43:48	php: : The command '/usr/local/sbin/squid -k kill' returned exit code '1', the output was '2011/02/03 10:43:48| aclParseAclLine: WARNING: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl" 2011/02/03 10:43:48| squid.conf line 77: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:48| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression 2011/02/03 10:43:48| squid.conf line 78: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:48| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression squid: ERROR: No running copy'
Feb 3 10:43:43	php: : The command '/usr/local/sbin/squid -k shutdown' returned exit code '1', the output was '2011/02/03 10:43:43| aclParseAclLine: WARNING: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl" 2011/02/03 10:43:43| squid.conf line 77: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:43| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression 2011/02/03 10:43:43| squid.conf line 78: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims 2011/02/03 10:43:43| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression squid: ERROR: No running copy'
Feb 3 10:43:43	php: : Creating squid cache subdirs in /var/squid/cache

The same expressions worked before without any glitch! Nasty

A manual deinstall of the "old" squid:

equired by these other packages
and may not be deinstalled:
squidGuard-1.4_2
#: pkg_delete  -f squid-2.7.9
pkg_delete: package 'squid-2.7.9' is required by these other packages
and may not be deinstalled (but I'll delete it anyway):
squidGuard-1.4_2
pkg_delete: unable to completely remove directory '/usr/local/libexec/squid'
pkg_delete: unable to completely remove directory '/usr/local/etc/squid/errors'
pkg_delete: unable to completely remove directory '/usr/local/etc/squid'
pkg_delete: file '/usr/local/etc/rc.d/squid' doesn't exist
pkg_delete: couldn't entirely delete package (perhaps the packing list is
incorrectly specified?)
===> post-deinstallation information for squid-2.7.9:

     Note:
     Squid related user accounts and groups were not removed.

     To remove the 'squid' user and the 'squid' group which were
     created by a default installation of this package, run

     pw userdel -n squid -u 100

     In order to ease updates the cache and log directories
     and all configuration files modified by you were preserved.

     Please remove them manually if you do not want to use
     Squid any longer.

After that i removed squid3, installed it again and all works again like before.
So the big work i have after every update i have to do the same things again.

Who has to review his/her code now? The squidguard-maintainer? Or is that a package-manager-problem? Then this thread would be better in the 2.0 forum. Not shure…

Oh, a maybe silly question: Do i have to enable the loopback-device too, or only the interfaces which are used by my LAN? (LAN; WIFI; OPTx)

edit: I was too early: squid alone starts, squidguard not. If i try to start squidguard too, both services die. Last log-entry from squidguard was "servicing requests". No message about its death.

squid logs this message now:

Feb 3 14:05:42	php: : SQUID is installed but not started. Not installing "filter" rules.
Feb 3 14:05:41	php: : SQUID is installed but not started. Not installing "nat" rules.
Feb 3 14:05:40	php: /pkg_edit.php: The command '/usr/local/sbin/squid -D' returned exit code '1', the output was '2011/02/03 14:05:40| WARNING: -D command-line option is obsolete. 2011/02/03 14:05:40| WARNING: Netmasks are deprecated. Please use CIDR masks instead. 2011/02/03 14:05:40| WARNING: IPv4 netmasks are particularly nasty when used to compare IPv6 to IPv4 ranges. 2011/02/03 14:05:40| WARNING: For now we will assume you meant to write /27 2011/02/03 14:05:40| ERROR: '0.0.0.0/0.0.0.0' needs to be replaced by the term 'all'. 2011/02/03 14:05:40| SECURITY NOTICE: Overriding config setting. Using 'all' instead. 2011/02/03 14:05:40| WARNING: (B) '::/0' is a subnetwork of (A) '::/0' 2011/02/03 14:05:40| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable 2011/02/03 14:05:40| WARNING: You should probably remove '::/0' from the ACL named 'all' 2011/02/03 14:05:40| WARNING: Netmasks are deprecated. Please use CIDR masks instead. 2011/02/03 14:05:40| WARNING: IPv4 netmasks a
Feb 3 14:05:40	squid: Bungled squid.conf line 62: reply_body_max_size 0 allow all
Feb 3 14:05:40	php: /pkg_edit.php: Starting Squid

dvserg

You use HDD full installed version of the pfsense?

_igor_

Its a full install. It doesn't work even after deinstalling/reinstalling.

dvserg

Deinstall (1)squidGuard/(2)squid from GUI, and type pkg_info from console. Result pkg_info pls post here.

_igor_

What happens on deinstallation of squidguard:

Backing up libraries... 
Removing package...
Starting package deletion for squidGuard-1.4_2...done.
Starting package deletion for cyrus-sasl-2.1...done.
Starting package deletion for openldap-client-2.4...done.
Starting package deletion for openssl-1.0...done.
Starting package deletion for squid-2.7...done.
Starting package deletion for db3-3.3...done.
Starting package deletion for db41-4.1.25_4...done.
Starting package deletion for db3-3.3.11_3,1...done.
Starting package deletion for cyrus-sasl-2.1.23...done.
Removing squidGuard components...
Tabs items... done.
Menu items... done.
Services... done.
Loading package instructions...

The squid 2.7 here… i had installed squid3!

output of pkg_info after deinstallation:

aspell-0.60.6_3 Spelling checker with better suggestion logic than ispell
bsdinstaller-2.0.2011.0131 BSD Installer mega-package
expat-2.0.1_1 XML 1.0 parser written in C
gettext-0.18.1.1 GNU gettext package
grub-0.97_4 GRand Unified Bootloader
joe-3.7,1 Joe's Own Editor
jpeg-8_3 IJG's jpeg compression utilities
libevent-1.4.14b_1 Provides an API to execute callback functions on certain ev
libiconv-1.13.1_1 A character set conversion library
lightsquid-1.8_2 A light and fast web based squid proxy traffic analyser
p7zip-9.13 File archiver with high compression ratio
perl-5.10.1_2 Practical Extraction and Report Language
perl-5.10.1_3 Practical Extraction and Report Language
unbound-1.4.8 A validating, recursive, and caching DNS resolver
zip-3.0 Create/update ZIP files compatible with pkzip

dvserg

The squid 2.7 here... i had installed squid3!

Yes - it's a SG depences. Need fix.
You can try install SG before and squid 3 then.

mnemonics.ca

@dvserg:

The squid 2.7 here... i had installed squid3!
Yes - it's a SG depences. Need fix.
You can try install SG before and squid 3 then.

There is a missing units in the line that creates the .conf file for Squid3. in the squid.inc file

The line

        $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " deny all\n";

Should read

        $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " all\n";

removing the "deny". In Squid version 3 the use of allow or deny are no longer valid for this directive.

the squid.inc file can be found in the /usr/local/pkg directory

Strangely though I was also having problems when the limit was set to 0, which according to the Squid documentation should be valid, however I would always receive a "request to large" error. I added a qualifier ( != 0) so that the line is only added if required.

if ($down_limit != 0) $conf .= 'reply_body_max_size ' . ($down_limit * 1024) . " all \n";

_igor_

kewl!

That change should go into the package. Thats that nasty error which made me scratch my head in a way my balls never like…
I took out the whole line in my config to get rid of the error at last. What i can tell is that i didn't see any failures in the function of squid3 without that line.

Thanks a lot for your finding!!!!

jimp

Here ya go:
https://github.com/bsdperimeter/pfsense-packages/commit/54c49bf2b5358b35602cae3cf6a9fead0ba886e5

mnemonics.ca

@jimp:

Here ya go:
https://github.com/bsdperimeter/pfsense-packages/commit/54c49bf2b5358b35602cae3cf6a9fead0ba886e5

thank you very much, just installed a new router, install the Squid3 package and viola, the Change was made. :)

Woot, I don't have to make the change manually any more.

If I find any other bugs, I will post them here.

tester_02

Newb question here..
I've been searching and can't seem to find an answer on version 3 vs 2. I can see posts from over a year ago on squid-cache about the comparison. v3 was in development, and 2 was stable.
Since then, I see v3 has stable versions (3.1).
So does this not mean instead of having both 2.x and 3.0.8 in pfsense, the path forward should actually be to remove the 2.x package and move into a stable 3.x release?
Not pushing and developers (although I'd contribute to a bounty), just trying to make sense of all the versions.
Am I understanding this right?

jimp

You are somewhat confusing the stability of squid with the stability of the pfSense package.

We still have a squid 2.7.x package for pfSense because it works, and it works well, and there isn't a large compelling reason to rush into 3.

The squid 3 pfSense package is largely untested and most likely still needs work. Until the squid 3 package for pfSense is proven to be stable, and work well with squidGuard/HAVP/whatever, then it will likely still remain in limbo.

There are FreeBSD ports still for Squid 2.x, 3.0.x, and 3.1.x, and 2.x is still the default as far as I can see there.

tester_02

Thanks for the clarification. I'm running 2.X right now and it's running with no issues. I've always been confused as to the point of the 3.x packages.