What Squid version and SquidGuard or DansGuardian?



  • Hi,

    for me Snort and SquidGuard/DansGuardian are the main reasons to use pfSense.
    If i look around here there are allot of problems when it comes to SquidGuard/DansGuardian.
    Also ClamAV would be nice but if it will work someday: https://forum.pfsense.org/index.php?topic=83404

    The question is: What Squid version plus SquidGuard or DansGuardian combination works?
    I want to use the blacklists from squidblacklist.org.

    And i have another question:
    If you use Adblock Plus in Firefox some sites don't work the way they should.
    Is this also the case with SquidGuard on the router?



  • squid and clamav is working on 2.2 (tested on amd64 version)

    squidguard-devel installs if you disable package signature check on system advanced.

    Dansguardian is not updated since two years. I'm working to port e2guardian to pfsense.
    If you like to help, follow these topics
    https://forum.pfsense.org/index.php?topic=87440.0



  • So the old Squid that isn't updated for a really long time is the way to go?
    And for ClamAV the old HAVP?

    For what do i need squidguard-devel?

    I wonder that all this packages are all still beta after such a long time?
    HAVP is from 2010 and squidGuard from 2009.



  • @MrGlasspoole:

    I wonder that all this packages are all still beta after such a long time?

    pfsense package gui version status beta is different from freebsd ports package version and status



  • But I'm still not sure what to use - what packages?

    And is e2guardian the new dansguardian?

    And what to use for ClamAV cause as in my old thread it did not work: https://forum.pfsense.org/index.php?topic=83404

    Is there some tutorial that works without messing around with hacking stuff?



  • I'm not sure which Squid version is working best in 2.2, but I've never had any problem using any of them with Dansguardian.

    E2Guardian is the new (fork of) Dansguardian. Dansguardian and Squidguard are not functionally equivalent. If you only want blacklist based blocking then Squidguard is fine… if you want content based filtering use Dansguardian (or E2Guardian).



  • What means "content based filtering"?

    Whats the difference between squidGuard and squidGuard-devel?



  • @MrGlasspoole:

    Whats the difference between squidGuard and squidGuard-devel?

    Binary version and gui options



  • E2/Dansguardian will look at the content of pages returned and decide whether to block or allow them based on a weighting calculation of "phrases" in the content. Blacklist blocking only looks at the URL. E2/Dansguardian will do both…



  • Ok, after looking around more i found out you don't need HAVP anymore - why nobody told me that here >:(

    But it does not work and I'm not the only one. It's frustrating if you are new to pfSense and if you think that
    packages you can install are working :(

    It's the latest 2.2 x64 and it's 2.2 since october (no update).
    I installed squid3 and squidGuard-squid3.
    I changed: "Service squid_clamav squidclamav.so"
    I removed: "ldap configuration'Manager:Apassword@ldap.chtsanti.net?o=chtsanti?mermberUid?(&(objectClass=posixGroup)(cn=%s))'"
    I changed to: "redirect https://192.168.0.1/clwarn.cgi"

    But clamd and squidGuard are not starting.
    ps ax | grep -i fresclam or tail -f /var/log/clamav/freshclam.log tells me:

    grep: fresclam: No such file or directory
    grep: or: No such file or directory
    grep: tail: No such file or directory
    

  • Banned

    @MrGlasspoole:

    ps ax | grep -i fresclam or tail -f /var/log/clamav/freshclam.log tells me:

    grep: fresclam: No such file or directory
    grep: or: No such file or directory
    grep: tail: No such file or directory
    

    LULz… :D Those were supposed to be two separate commands....

    
    ps ax | grep -i freshclam
    
    
    
    tail -f /var/log/clamav/freshclam.log
    
    


  • lol - nobody told this guy neither: https://forum.pfsense.org/index.php?topic=87562.msg481208#msg481208

    The output is:

     2560  -  Is     0:01.78 /usr/local/bin/freshclam --daemon -p /var/run/clamav/freshclam.pid
    48264  0  S+     0:00.00 grep -i freshclam
    
    getfile: Can't write 1448 bytes to /var/db/clamav/clamav-e7d674823863b493974d944efdd05df5.tmp/clamav-b69402a7fda033c3acff60eb7da8b732.tmp
    WARNING: Can't download main.cvd from database.clamav.net
    Received signal: wake up
    ClamAV update process started at Thu Jan 29 16:42:08 2015
    WARNING: Your ClamAV installation is OUTDATED!
    WARNING: Local version: 0.98.5 Recommended version: 0.98.6
    DON'T PANIC! Read http://www.clamav.net/support/faq
    getfile: Can't write 1448 bytes to /var/db/clamav/clamav-26487302f658d4bfe55c0f00608d77a0.tmp/clamav-51d39e1a1042c682cfd37d9fe5df5094.tmp
    WARNING: Can't download main.cvd from database.clamav.net
    

    OUTDATED? I just installed it…



  • @MrGlasspoole:

    OUTDATED? I just installed it…

    Minor version changes that need a long update sequence. (clamav developer, freebsd  package maintainer, pfsense package update, pfsense pbi build,…)



  • And what to do now?



  • @MrGlasspoole:

    And what to do now?

    Update virus definitions, the version alert is just a warning, not a service stop.



  • I can't find a tab where i see something like update definitions…



  • @MrGlasspoole:

    I can't find a tab where i see something like update definitions…

    run freshclam via console/ssh



  • still the same message:

    getfile: Can't write 1448 bytes to /var/db/clamav/clamav-e1dc9c51263e0827cd2a0b973ba41d4e.tmp/clamav-d6889ae227e0a4134d824971de0a4a84.tmp
    WARNING: Can't download main.cvd from database.clamav.net
    


  • Can your ping database.clamav.net from your box?



  • Yes

    PING db.other.clamav.net (193.1.193.64): 56 data bytes
    64 bytes from 193.1.193.64: icmp_seq=0 ttl=51 time=35.980 ms
    64 bytes from 193.1.193.64: icmp_seq=1 ttl=51 time=40.412 ms
    64 bytes from 193.1.193.64: icmp_seq=2 ttl=51 time=35.124 ms
    
    --- db.other.clamav.net ping statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 35.124/37.172/40.412/2.318 ms
    


  • And now?



  • reinstall squid3 and see if that resolves the issue. i'm wondering if a folder wasn't created to store the av db



  • That did not help.

    $ freshclam
    ClamAV update process started at Fri Jan 30 23:32:23 2015
    WARNING: Your ClamAV installation is OUTDATED!
    WARNING: Local version: 0.98.5 Recommended version: 0.98.6
    DON'T PANIC! Read http://www.clamav.net/support/faq
    ERROR: Can't download main.cvd from database.clamav.net
    Giving up on database.clamav.net...
    Update failed. Your network may be down or none of the mirrors listed in /usr/local/etc/freshclam.conf is working.
    


  • Check first if there is another freshclam process running before trying to execute another one.

    Look foe other squid 3 threads on 2.2.  I've posted a really step by step guide to get it working on 64bits version.



  • I already saw that other threads.

    Now i had this:

    /var: write failed, filesystem is full
    getfile: Can't write 8192 bytes to /var/db/clamav/clamav-2786ca6469a9b9aafef1622f0f0f13be.tmp/clamav-f58a45f6084309de3a81938d410d397b.tmp
    WARNING: Can't download main.cvd from database.clamav.net
    

    After i saw the FULL i disabled "Use RAM Disks" and ClamAV works now.
    But the squidGuard service still does not start.



  • why were you using a ram disk? I can see using it if your using pfSense without any packages… Once you add packages, you need /var to not disappear

    anything in the log for squidguard? Which version did you install?

    /var/squid/logs/cache.log
    /var/squidGuard/log


  • Banned

    @Cino:

    why were you using a ram disk?

    Because people have no clue what they are doing.



  • @Cino:

    why were you using a ram disk?

    RAM is faster and takes stress away from the disk.
    I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
    on again. But when i do that clamd does not start.

    @Cino:

    anything in the log for squidguard? Which version did you install?

    squidGuard-squid3 1.4_7 pkg v.1.9.6

    squidGuard/log is empty.

    Here is some stuff from squid/logs/cache.log:

    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    2015/01/31 13:50:37 kid1| WARNING: redirector #Hlpr0 exited
    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    2015/01/31 13:50:37 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    2015/01/31 13:50:37| pinger: Initialising ICMP pinger ...
    2015/01/31 13:50:37|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:37| pinger: Unable to start ICMP pinger.
    2015/01/31 13:50:37|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:37| pinger: Unable to start ICMPv6 pinger.
    2015/01/31 13:50:37| FATAL: pinger: Unable to open any ICMP sockets.
    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    Shared object "libldap-2.4.so.2" not found, required by "squidGuard"
    2015/01/31 13:50:40 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 13:50:40| pinger: Initialising ICMP pinger ...
    2015/01/31 13:50:40|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:40| pinger: Unable to start ICMP pinger.
    2015/01/31 13:50:40|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:40| pinger: Unable to start ICMPv6 pinger.
    2015/01/31 13:50:40| FATAL: pinger: Unable to open any ICMP sockets.
    2015/01/31 13:50:47 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 13:50:47| pinger: Initialising ICMP pinger ...
    2015/01/31 13:50:47|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:47| pinger: Unable to start ICMP pinger.
    2015/01/31 13:50:47|  icmp_sock: (1) Operation not permitted
    2015/01/31 13:50:47| pinger: Unable to start ICMPv6 pinger.
    2015/01/31 13:50:47| FATAL: pinger: Unable to open any ICMP sockets.
    FATAL: Received Segment Violation...dying.
    CPU Usage: 659.029 seconds = 69.054 user + 589.975 sys
    Maximum Resident Size: 154112 KB
    Page faults with physical i/o: 12
    2015/01/31 14:01:45 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/01/31 14:01:45| pinger: Initialising ICMP pinger ...
    2015/01/31 14:01:45|  icmp_sock: (1) Operation not permitted
    2015/01/31 14:01:45| pinger: Unable to start ICMP pinger.
    2015/01/31 14:01:45|  icmp_sock: (1) Operation not permitted
    2015/01/31 14:01:45| pinger: Unable to start ICMPv6 pinger.
    2015/01/31 14:01:45| FATAL: pinger: Unable to open any ICMP sockets.
    2015/01/31 16:38:35 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/01/31 16:38:36| pinger: Initialising ICMP pinger ...
    2015/01/31 16:38:36|  icmp_sock: (1) Operation not permitted
    2015/01/31 16:38:36| pinger: Unable to start ICMP pinger.
    2015/01/31 16:38:36|  icmp_sock: (1) Operation not permitted
    2015/01/31 16:38:36| pinger: Unable to start ICMPv6 pinger.
    2015/01/31 16:38:36| FATAL: pinger: Unable to open any ICMP sockets.
    2015/01/31 16:10:32 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 16:10:32| pinger: Initialising ICMP pinger ...
    2015/01/31 16:10:32|  icmp_sock: (1) Operation not permitted
    

  • Banned

    @MrGlasspoole:

    I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
    on again. But when i do that clamd does not start.

    They are NOT there any more. Guess why.  ::) ::) ::)

    Sigh. Someone make the package bail out on install when people configure similar BS.



  • @MrGlasspoole:

    @Cino:

    why were you using a ram disk?

    RAM is faster and takes stress away from the disk.
    I increased the ram disk and thought after the virus definitions are now there i can turn ram disk
    on again. But when i do that clamd does not start.

    If your going to use squid or any package that writes persistent data to /var, dont use ramdisk. It will wipe /var every time you reboot and will make the packages fail to start. You will then have to re-install the packages again every reboot…

    For the other issues. If you have searched the forum; you would have found solutions.

    In the Squid config page. check 'Disable ICMP' to get rid of the 'FATAL: pinger: Unable to open any ICMP sockets' errors

    for squidGuard run the following commands for amd64... (make sure path /usr/pbi/squidguard-squid3-amd64 exist first)

    
    ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libldap-2.4.so.8 /usr/local/lib/libldap-2.4.so.8
    ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libldap-2.4.so.8 /lib/libldap-2.4.so.8
    ln -s /usr/pbi/squidguard-squid3-amd64/local/lib/libdb-4.6.so.0 /usr/local/lib/libdb-4.6.so.0
    ln -s /usr/pbi/squidguard-squid3l-amd64/local/lib/libdb-4.6.so.0 /usr/lib/libdb-4.6.so.0
    
    

    After you disable ramdisk, UN-install the packages. Reboot, install the packages.. Run the links for squidguard



  • A search for libldap-2.4.so.2 did return nothing.

    After a squid restart and trying to start squidGuart:

    FATAL: Received Segment Violation...dying.
    CPU Usage: 0.163 seconds = 0.112 user + 0.052 sys
    Maximum Resident Size: 101712 KB
    Page faults with physical i/o: 0
    2015/01/31 17:24:29 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/01/31 17:24:31 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/01/31 17:25:47 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 17:25:49 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 17:26:02 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    2015/01/31 17:26:06 kid1| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    FATAL: Received Segment Violation...dying.
    CPU Usage: 0.250 seconds = 0.194 user + 0.057 sys
    Maximum Resident Size: 103280 KB
    Page faults with physical i/o: 0
    2015/01/31 17:26:31 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/01/31 17:26:33 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    

    But squidGuard service status is still stopped.



  • What error you get in your browser?



  • Error in the browser?
    I did not setup a blacklist yet cause the service is not running.
    I can surf the web normally if i point the browser to wpad.mydomain.net/wpad.dat


  • Banned

    It's not running because it segfaults…

    
    FATAL: Received Segment Violation...dying.
    
    

    Are you still running this from ramdisk?



  • @doktornotor:

    Are you still running this from ramdisk?

    No
    I thought thats the normal message if you restart squid



  • Ok, after subscription to squidblacklist.org and this tutorial http://www.legoclan.com/tutorials/#squidblacklist squidGuard is running.

    Reason for Squid and SquidGuard for me was:
    1. That i thought i can speed up websites if i block ads before they reach the clients.
        But it seems that Adblock Plus works better.

    2. Block ads and tracking for devices like phones, TVs, consoles…

    3. Virus protection for phones, TVs, consoles...
        But ClamAV really makes websites slow.

    I run pfSense in Hyper-V 2012 R2 Core on a 3.6GHz Core i3-4160 and assigned 2GB to pfSense.
    I did set:
    Squid Memory cache size: 512
    Squid Maximum object size in RAM: 128

    Hard disk cache is off cause i was reading it does not help if you have fast internet and not much clients (5-10).

    I have a 120 MBit/s internet connection and maybe upgrade to 200.

    It would be nice to block:
    Virus, Botnet, Malware, Adware, APT, Drive-By Download, Infectious, Espionage, hosts that perform IP tracking for media companies and associations like RIAA/MPAA

    Ad the moment i use Malicious, Proxies and the USG Blacklist from squidblacklist.org

    Would be nice to to experience how other handle that stuff.



  • Did you tried any changes on clamav/icap configuration, like improving exclusion, etc?



  • Ok, step by step.

    I have the problems with the clwarn.cgi.
    First i changed redirect to:

    https://192.168.0.1/clwarn.cgi
    

    as it was suggested. But it's https so i need to accept the non trusted side in Firefox.
    Can i use http?

    Then my clwarn.cgi is just an empty side?

    Next thing is that i get a Squid error site if a URL does not longer exist.
    Is it possible to show the defaults browser page?
    Or does it have advantages to see a Squid site in such a case?

    Read Error
    The system returned: (54) Connection reset by peer
    


  • @MrGlasspoole:

    Ok, step by step.

    I have the problems with the clwarn.cgi.
    First i changed redirect to:

    https://192.168.0.1/clwarn.cgi
    

    as it was suggested. But it's https so i need to accept the non trusted side in Firefox.
    Can i use http?

    Then my clwarn.cgi is just an empty side?

    try https://192.168.0.1/squid_clwarn.php



  • @marcelloc:

    Did you tried any changes on clamav/icap configuration, like improving exclusion, etc?

    I quote myself from another thread:

    Yes there are many scenarios but i think it would be nice if some users would post there basic home settings
    or there would be some recommendations for example on stuff like Squid Memory cache size based on RAM.
    I believe for home use the needs between people do not differentiate to much.
    I think there are allot of people here who have experience on what works best.

    I'm not sure what files to scan and which not.

    @Cino:

    try https://192.168.0.1/squid_clwarn.php

    Ok, that works. But why does it not point to a php file from the beginning?
    But still - is it normal that i need a certificate to show error warnings?


Log in to reply