What Squid version and SquidGuard or DansGuardian?

Cino

reinstall squid3 and see if that resolves the issue. i'm wondering if a folder wasn't created to store the av db

MrGlasspoole

That did not help.

$ freshclam
ClamAV update process started at Fri Jan 30 23:32:23 2015
WARNING: Your ClamAV installation is OUTDATED!
WARNING: Local version: 0.98.5 Recommended version: 0.98.6
DON'T PANIC! Read http://www.clamav.net/support/faq
ERROR: Can't download main.cvd from database.clamav.net
Giving up on database.clamav.net...
Update failed. Your network may be down or none of the mirrors listed in /usr/local/etc/freshclam.conf is working.

marcelloc

Check first if there is another freshclam process running before trying to execute another one.

Look foe other squid 3 threads on 2.2. I've posted a really step by step guide to get it working on 64bits version.

MrGlasspoole

I already saw that other threads.

Now i had this:

/var: write failed, filesystem is full
getfile: Can't write 8192 bytes to /var/db/clamav/clamav-2786ca6469a9b9aafef1622f0f0f13be.tmp/clamav-f58a45f6084309de3a81938d410d397b.tmp
WARNING: Can't download main.cvd from database.clamav.net

After i saw the FULL i disabled "Use RAM Disks" and ClamAV works now.
But the squidGuard service still does not start.

Cino

why were you using a ram disk? I can see using it if your using pfSense without any packages… Once you add packages, you need /var to not disappear

anything in the log for squidguard? Which version did you install?

/var/squid/logs/cache.log
/var/squidGuard/log

doktornotor

@Cino:

why were you using a ram disk?

Because people have no clue what they are doing.

MrGlasspoole

@Cino:

why were you using a ram disk?

RAM is faster and takes stress away from the disk.
I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
on again. But when i do that clamd does not start.

@Cino:

anything in the log for squidguard? Which version did you install?

squidGuard-squid3 1.4_7 pkg v.1.9.6

squidGuard/log is empty.

Here is some stuff from squid/logs/cache.log:

Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
2015/01/31 13:50:37 kid1| WARNING: redirector #Hlpr0 exited
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
2015/01/31 13:50:37 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
2015/01/31 13:50:37| pinger: Initialising ICMP pinger ...
2015/01/31 13:50:37|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:37| pinger: Unable to start ICMP pinger.
2015/01/31 13:50:37|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:37| pinger: Unable to start ICMPv6 pinger.
2015/01/31 13:50:37| FATAL: pinger: Unable to open any ICMP sockets.
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
2015/01/31 13:50:40 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 13:50:40| pinger: Initialising ICMP pinger ...
2015/01/31 13:50:40|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:40| pinger: Unable to start ICMP pinger.
2015/01/31 13:50:40|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:40| pinger: Unable to start ICMPv6 pinger.
2015/01/31 13:50:40| FATAL: pinger: Unable to open any ICMP sockets.
2015/01/31 13:50:47 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 13:50:47| pinger: Initialising ICMP pinger ...
2015/01/31 13:50:47|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:47| pinger: Unable to start ICMP pinger.
2015/01/31 13:50:47|  icmp_sock: (1) Operation not permitted
2015/01/31 13:50:47| pinger: Unable to start ICMPv6 pinger.
2015/01/31 13:50:47| FATAL: pinger: Unable to open any ICMP sockets.
FATAL: Received Segment Violation...dying.
CPU Usage: 659.029 seconds = 69.054 user + 589.975 sys
Maximum Resident Size: 154112 KB
Page faults with physical i/o: 12
2015/01/31 14:01:45 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
2015/01/31 14:01:45| pinger: Initialising ICMP pinger ...
2015/01/31 14:01:45|  icmp_sock: (1) Operation not permitted
2015/01/31 14:01:45| pinger: Unable to start ICMP pinger.
2015/01/31 14:01:45|  icmp_sock: (1) Operation not permitted
2015/01/31 14:01:45| pinger: Unable to start ICMPv6 pinger.
2015/01/31 14:01:45| FATAL: pinger: Unable to open any ICMP sockets.
2015/01/31 16:38:35 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
2015/01/31 16:38:36| pinger: Initialising ICMP pinger ...
2015/01/31 16:38:36|  icmp_sock: (1) Operation not permitted
2015/01/31 16:38:36| pinger: Unable to start ICMP pinger.
2015/01/31 16:38:36|  icmp_sock: (1) Operation not permitted
2015/01/31 16:38:36| pinger: Unable to start ICMPv6 pinger.
2015/01/31 16:38:36| FATAL: pinger: Unable to open any ICMP sockets.
2015/01/31 16:10:32 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 16:10:32| pinger: Initialising ICMP pinger ...
2015/01/31 16:10:32|  icmp_sock: (1) Operation not permitted

doktornotor

@MrGlasspoole:

I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
on again. But when i do that clamd does not start.

They are NOT there any more. Guess why. ::) ::) ::)

Sigh. Someone make the package bail out on install when people configure similar BS.

Cino

@MrGlasspoole:

@Cino:

why were you using a ram disk?

RAM is faster and takes stress away from the disk.
I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
on again. But when i do that clamd does not start.

If your going to use squid or any package that writes persistent data to /var, dont use ramdisk. It will wipe /var every time you reboot and will make the packages fail to start. You will then have to re-install the packages again every reboot…

For the other issues. If you have searched the forum; you would have found solutions.

In the Squid config page. check 'Disable ICMP' to get rid of the 'FATAL: pinger: Unable to open any ICMP sockets' errors

for squidGuard run the following commands for amd64... (make sure path /usr/pbi/squidguard-squid3-amd64 exist first)


ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libldap-2.4.so.8 /usr/local/lib/libldap-2.4.so.8
ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libldap-2.4.so.8 /lib/libldap-2.4.so.8
ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libdb-4.6.so.0 /usr/local/lib/libdb-4.6.so.0
ln -s /usr/pbi/squidguard-squid3l-amd64/local/lib/libdb-4.6.so.0 /usr/lib/libdb-4.6.so.0

After you disable ramdisk, UN-install the packages. Reboot, install the packages.. Run the links for squidguard

MrGlasspoole

A search for libldap-2.4.so.2 did return nothing.

After a squid restart and trying to start squidGuart:

FATAL: Received Segment Violation...dying.
CPU Usage: 0.163 seconds = 0.112 user + 0.052 sys
Maximum Resident Size: 101712 KB
Page faults with physical i/o: 0
2015/01/31 17:24:29 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
2015/01/31 17:24:31 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
2015/01/31 17:25:47 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 17:25:49 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 17:26:02 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
2015/01/31 17:26:06 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
FATAL: Received Segment Violation...dying.
CPU Usage: 0.250 seconds = 0.194 user + 0.057 sys
Maximum Resident Size: 103280 KB
Page faults with physical i/o: 0
2015/01/31 17:26:31 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
2015/01/31 17:26:33 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...

But squidGuard service status is still stopped.

marcelloc

What error you get in your browser?

MrGlasspoole

Error in the browser?
I did not setup a blacklist yet cause the service is not running.
I can surf the web normally if i point the browser to wpad.mydomain.net/wpad.dat

doktornotor

It's not running because it segfaults…


FATAL: Received Segment Violation...dying.

Are you still running this from ramdisk?

MrGlasspoole

@doktornotor:

Are you still running this from ramdisk?

No
I thought thats the normal message if you restart squid

MrGlasspoole

Ok, after subscription to squidblacklist.org and this tutorial http://www.legoclan.com/tutorials/#squidblacklist squidGuard is running.

Reason for Squid and SquidGuard for me was:
1. That i thought i can speed up websites if i block ads before they reach the clients.
But it seems that Adblock Plus works better.

2. Block ads and tracking for devices like phones, TVs, consoles…

3. Virus protection for phones, TVs, consoles...
But ClamAV really makes websites slow.

I run pfSense in Hyper-V 2012 R2 Core on a 3.6GHz Core i3-4160 and assigned 2GB to pfSense.
I did set:
Squid Memory cache size: 512
Squid Maximum object size in RAM: 128

Hard disk cache is off cause i was reading it does not help if you have fast internet and not much clients (5-10).

I have a 120 MBit/s internet connection and maybe upgrade to 200.

It would be nice to block:
Virus, Botnet, Malware, Adware, APT, Drive-By Download, Infectious, Espionage, hosts that perform IP tracking for media companies and associations like RIAA/MPAA

Ad the moment i use Malicious, Proxies and the USG Blacklist from squidblacklist.org

Would be nice to to experience how other handle that stuff.

marcelloc

Did you tried any changes on clamav/icap configuration, like improving exclusion, etc?

MrGlasspoole

Ok, step by step.

I have the problems with the clwarn.cgi.
First i changed redirect to:

https://192.168.0.1/clwarn.cgi

as it was suggested. But it's https so i need to accept the non trusted side in Firefox.
Can i use http?

Then my clwarn.cgi is just an empty side?

Next thing is that i get a Squid error site if a URL does not longer exist.
Is it possible to show the defaults browser page?
Or does it have advantages to see a Squid site in such a case?

Read Error
The system returned: (54) Connection reset by peer

Cino

@MrGlasspoole:

Ok, step by step.

I have the problems with the clwarn.cgi.
First i changed redirect to:
https://192.168.0.1/clwarn.cgi
as it was suggested. But it's https so i need to accept the non trusted side in Firefox.
Can i use http?

Then my clwarn.cgi is just an empty side?

try https://192.168.0.1/squid_clwarn.php

MrGlasspoole

@marcelloc:

Did you tried any changes on clamav/icap configuration, like improving exclusion, etc?

I quote myself from another thread:

Yes there are many scenarios but i think it would be nice if some users would post there basic home settings
or there would be some recommendations for example on stuff like Squid Memory cache size based on RAM.
I believe for home use the needs between people do not differentiate to much.
I think there are allot of people here who have experience on what works best.

I'm not sure what files to scan and which not.

@Cino:

try https://192.168.0.1/squid_clwarn.php

Ok, that works. But why does it not point to a php file from the beginning?
But still - is it normal that i need a certificate to show error warnings?

MrGlasspoole

From the log:

/usr/local/bin/squidGuard: can't write to logfile /var/log/squidGuard/squidGuard.log

but it's there and gets updated?

Then:

kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"

normal?

Then i saw:

kid1| WARNING: All 5/5 redirector processes are busy.
kid1| WARNING: 5 pending requests queued
kid1| WARNING: Consider increasing the number of redirector processes in your config file.
kid1| WARNING: All 5/5 redirector processes are busy.
kid1| WARNING: 5 pending requests queued
kid1| WARNING: Consider increasing the number of redirector processes in your config file.

after some google i changed in "/usr/local/pkg/squidguard_configurator.inc":

define('REDIRECTOR_PROCESS_COUNT', '5');

to 10.
Is this correct?

Then the log is full of "init domainlist" and "loading dbfile".
Is that normal?

cache.txt
cache.log.0.txt
squidGuard.txt