Squid 3.3.4 package for pfsense with ssl filtering

Nachtfalke

Hi Marcello,

this are really amazing features. Transparent SSL filtering and an up to date anti-virus feature ist great!

Thank you very much for the very very hard work on that! I hope I will find some time to test all the festures in the near future ;D

quetzalcoatl

On pfsense2.1 snapshot from first week of may 2013 I uninstalled squid 3.1 and installed squid 3.3.4

I noticed that the squid service was not starting anymore and there were 2 new services not starting as well.

the other 2 services are:
Icap inteface for squid and clamav integration
Clamav Antivirus

Also i noticed that after installing squid 3.3.4 my old config was still there…...handy but not sure if was a very clean installation.

so i installed in a new machine the latest 2.1 snapshot from today may 13th and installed the latest squid 3.3.4
and the very same issue is still there, squid service is still not starting even in an absolutely clean install.

what should i do to make it work?

wheelz

quetzalcoatl,

@marcelloc:

know bugs

Clamav integration via i-cap is not working(loops until crash)

It is a known bug that I'm sure he is working on.

quetzalcoatl

thanks, and i supposed that those new 2 services still need to be fixed.
I don't even need the antivirus and stuff anyways.
I just need the squid3.3 service to start but it does not.

marcelloc

@quetzalcoatl:

I just need the squid3.3 service to start but it does not.

What you get on squid logs? I've did a clean install and service is up with antivirus disabled.

squid -NsXY is a good way to find what is not working.

Did you saved config after package installing 3.3?

Cino

it installs now but its missing a shared object.. I haven't had a chance to to see if its on the box and just needs to be linked.


php: : The command '/usr/pbi/squid-i386/sbin/squid -f /usr/pbi/squid-i386/etc/squid/squid.conf' returned exit code '1', the output was '/libexec/ld-elf.so.1: Shared object "libgssapi.so.10" not found, required by "squid"'

quetzalcoatl

@marcelloc:

@quetzalcoatl:

I just need the squid3.3 service to start but it does not.

What you get on squid logs? I've did a clean install and service is up with antivirus disabled.

squid -NsXY is a good way to find what is not working.

Did you saved config after package installing 3.3?

i don't even know where to look for squid logs.
also as soon as i install squid 3.3 the icap and clamd services get installed by themselves and i don't know how to remove them (if i have to).

if i go to the pfsense console and hit 8 to get into the shell, then i write "squid -NsXY"
it says "/libexec/lds-elf.so.1: Shared object "libgssapi.so.10" not found, required by "squid".

I did not save my squid config before installing 3.3 but the old config was still there.

Right now i have 2 virtual machines, 1 with pfsense 2.1 with reinstalled squid 3.3 from 3.1, and another one with the today's pfsense snapshot and clean install of squid 3.3.

I see "/libexec/lds-elf.so.1: Shared object "libgssapi.so.10" not found, required by "squid" in both virtual machines.

quetzalcoatl

I hope the file libgssapi.so.10 will be added to the next pfsense snapshots or within the next release of squid3.3

I have been trying all evening to get this file into that /usr/local/lib folder
but i did not succeed.

How do i copy or move a file from the internet to a certain folder inside pfsense?
Using the GUI i can upload a file but i don't know how to place it in /usr/local/lib

marcelloc

@quetzalcoatl:

How do i copy or move a file from the internet to a certain folder?

using console/ssh

cd /usr/local/lib
fetch url_for_libs

Download all libs from my ldd folder.

quetzalcoatl

Thanks to you now I'm a little less ignorant about pfsense and squid! :P

I did copy several files from http://e-sac.siteseguro.ws/pfsense/8/amd64/All/ldd/ to /usr/local/lib

until when i do squid -NsXY it doesn't ask me any more for missing files but now squid still service doesn't start and when i do squid -NsXY the shell shows me a bunch of:
"Acl.cc(353) ~ACL: ACL::~ACL: ´"

and i don't know how to see above these lines because there are too many.

I don't know how to scroll text lines inside the shell.
(tons of ignorance here! sorry)

Legion

Hi marcelloc,

I tried it with my normal configuration (pfSense -> DG -> Squid3 -> Internet) and couldn't get it to work.

DG on pfSense:8080 with Squid as parent
Squid3 on 127.0.0.1:3128

marcelloc

@Legion:

DG on pfSense:8080 with Squid as parent
Squid3 on 127.0.0.1:3128

Check if squid is running, on log you sent I can see only warnings.

squid -NsXY on console can show you squid startup error or /var/squid/logs/cache.log

dbannack

Hi, I am having problems using squid 3.3
I can not start the service ..
Returns the following error ..

May 14 14:49:28	php: /pkg_edit.php: Starting Squid
May 14 14:49:28	squid: Bungled squid.conf line 45: offline_mode offcache_swap_low 90
May 14 14:49:28	php: /pkg_edit.php: The command '/usr/local/sbin/squid -f /usr/local/etc/squid/squid.conf' returned exit code '1', the output was '2013/05/14 14:49:28| Warning: empty ACL: acl localnet src FATAL: Bungled squid.conf line 45: offline_mode offcache_swap_low 90 Squid Cache (Version 3.3.4): Terminated abnormally. CPU Usage: 0.021 seconds = 0.021 user + 0.000 sys Maximum Resident Size: 26512 KB Page faults with physical i/o: 0'
May 14 14:49:38	check_reload_status: Reloading filter

Tikimotel

I've been trying to get my "Squid3-dev"+"Squidguard" config running (no ssl & no antivirus active, not yet anyway)

Copied the libs and chmod 755, used WinSCP.

First warning:

php: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '1', the output was '2013/05/14 19:27:02| Warning: empty ACL: acl localnet src FATAL: Bungled squid.conf line 33: offline_mode offcache_swap_low 90 Squid Cache (Version 3.3.4): Terminated abnormally. CPU Usage: 0.038 seconds = 0.030 user + 0.008 sys Maximum Resident Size: 39824 KB Page faults with physical i/o: 0

Squid complains "localnet" was not set correctly. I changed the code to work.
I hardcoded my localnet into "squid.inc" so the squid.conf line would read "acl localnet src 192.168.0.1/24"
So that was a quick and very dirty temporary fix. (but hey it works…)


	if ($settings['allow_interface'] == 'on') {
		$src = '';
		foreach ($real_ifaces as $iface) {
			list($ip, $mask) = $iface;
			$ip = long2ip(ip2long($ip) & ip2long($mask));
			$mask = 32-log((ip2long($mask) ^ ip2long('255.255.255.255'))+1,2);
			$src .= " $ip/$mask";
		}
		$conf .= "# Allow local network(s) on interface(s)\n";
		$conf .= "acl localnet src 192.168.0.0/24\n";
		$valid_acls[] = 'localnet';

Second warning:

php: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '1', the output was '2013/05/14 19:29:35| WARNING: Netmasks are deprecated. Please use CIDR masks instead. 2013/05/14 19:29:35| WARNING: IPv4 netmasks are particularly nasty when used to compare IPv6 to IPv4 ranges. 2013/05/14 19:29:35| WARNING: For now we will assume you meant to write /24 FATAL: Bungled squid.conf line 33: offline_mode offcache_swap_low 90 Squid Cache (Version 3.3.4): Terminated abnormally. CPU Usage: 0.038 seconds = 0.031 user + 0.008 sys Maximum Resident Size: 38832 KB Page faults with physical i/o: 0

Hmmm.. "offline_mode offcache_swap_low 90" not good…
So checked "squid.inc" again, added extra linefeed between "offline_mode" and "EOD".

cache_mem $memory_cache_size MB
maximum_object_size_in_memory {$max_objsize_in_mem} KB
memory_replacement_policy {$memory_policy}
cache_replacement_policy {$cache_policy}
$disk_cache_opts
minimum_object_size {$min_objsize} KB
maximum_object_size {$max_objsize}
offline_mode {$offline_mode}

EOD;

Changes for "squidguard_configurator.inc" (replaced redirector commands, for squid 3.3 compatibility)

# ------------------------------------------------------------------------------
# squid config options
# ------------------------------------------------------------------------------
define('REDIRECTOR_OPTIONS_REM',   '# squidGuard options');
define('REDIRECTOR_PROGRAM_OPT',   'url_rewrite_program');
define('REDIRECT_BYPASS_OPT',      'url_rewrite_bypass');
define('REDIRECT_CHILDREN_OPT',    'url_rewrite_children');
define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started

Clear old settings from the "Proxy server" page, then save on the "proxy filter" page again so the new redirector commands are used.

marcelloc

I've pushed a fix for some of these issues

on 2.0.x, if you got squid running but no port listening, you may try squid 3.3.4 from my repo compiled without ipv6.

amd64
http://e-sac.siteseguro.ws/packages/amd64/8/All/squid-3.3.4.tbz

i386
http://e-sac.siteseguro.ws/packages/8/All/squid-3.3.4.tbz

Some lib replacements may require a reboot to avoid squid crashes.

athurdent

Great package, thank you!
On my i386 test KVM I had to copy over the libs and also reinstall squid3-dev afterwards. It would not start because of:

squid[88922]: execvp failed: (2) No such file or directory

Now it's working and I am testing it.

marcelloc

@athurdent:

It would not start because of:
squid[88922]: execvp failed: (2) No such file or directory
Now it's working and I am testing it.

I'm confused, is it working or not ???

Legion

@marcelloc:

@Legion:

DG on pfSense:8080 with Squid as parent
Squid3 on 127.0.0.1:3128

Check if squid is running, on log you sent I can see only warnings.

squid -NsXY on console can show you squid startup error or /var/squid/logs/cache.log

Nope, wasn't running. I tried again and got:

Noticeable points:

I had all my subnets on the ACL allow list (don't remember why)
I had pfsense.org and pfsense.com in ACL allow lists, from problems I had once accessing new package information
I didn't uninstall Squid3 first. But when I noticed that I still had it, I tried to uninstall it and reinstall Squid3-dev and it still didn't work
I get multiple dansguardian[23423]: Error connecting to proxy messages in system.log and no internet connectivity at all
no transparent http or https checked, Squid3-dev listening on localhost:3128 only, NAT rules to redirect 80 to DG, then DG has Squid as parent.

Maybe Squid3-dev works best without DG underneath?

marcelloc

@Legion:

Noticeable points:

I had all my subnets on the ACL allow list (don't remember why)

I had pfsense.org and pfsense.com in ACL allow lists, from problems I had once accessing new package information

I didn't uninstall Squid3 first. But when I noticed that I still had it, I tried to uninstall it and reinstall Squid3-dev and it still didn't work

I get multiple dansguardian[23423]: Error connecting to proxy messages in system.log and no internet connectivity at all

no transparent http or https checked, Squid3-dev listening on localhost:3128 only, NAT rules to redirect 80 to DG, then DG has Squid as parent.

Maybe Squid3-dev works best without DG underneath?

uninstall both and then reinstall squid3-dev.

I've pushed yesterday some fixes to conf generator.

I think it's better to test squid itself and then go to dansguardian integration.

Leave localhost unchecked, it's automatically inserted when using transparent mode. I'll include this warning on gui to prevent some errors.

athurdent

@marcelloc:

@athurdent:
It would not start because of:
squid[88922]: execvp failed: (2) No such file or directory
Now it's working and I am testing it.
I'm confused, is it working or not ???

Sorry for not being clearer. After copying the libs it would not start until I reinstalled it. Just wanted to let others know that maybe a reinstall is needed after putting the libs in place.
It works fine now and I have to dig a little further into ssl filtering ;)