Squid / Squidguard



  • I'm running squid3-dev 3.3.10 pkg 2.2.6 so that I can scan with ClamAV and am trying to add in squidGuard-squid3 1.4_4 pkg v1.9.5 to filter web sites but just can't get it to work.  I've followed just about every tutorial I can find but I end up being unable to load any web pages.  They all just time out.  It's been 2 days of trying to figure it out and I've kinda given up.  Should I continue down this road?  Or should I go down to Squid 2.x and squidGuard 2.x and use the HAVP antivirus plugin?  Has anyone gotten this working that could post their configs of how it should be set up?  I'd really appreciate it.  Thanks.

    Version 2.1.4-RELEASE (amd64)
    built on Fri Jun 20 15:48:47 EDT 2014
    FreeBSD 8.3-RELEASE-p16

    You are on the latest version.
    Platform nanobsd (4g)
    NanoBSD Boot Slice pfsense0 / da0s1 (rw)



  • i can help you, try to disable squidguard first and check if squid is running well. Then advise the result



  • Thanks for the help and sorry for the delay.  I'm finally able to get back to this.

    I've removed squidguard and things work fine with just Squid enabled.  I've attached the settings I'm using.

    With AV enabled I get the ICAP protocol error that you can see at:  http://www.censornet.com/assets/images/kb/icaperror.png  I've tried the suggestions at:  http://squidclamav.darold.net/tuning.html which are:

    If you experience Squid "ICAP protocol error" (with bypass enabled) please consider increasing the c-icap following parameters: StartServers, MaxServers, MinSpareThreads, MaxSpareThreads, ThreadsPerChild. Increase also in clamd.conf parameter: MaxThreads may help.

    Other than that I haven't touched the options for the Antivirus.

    I'll follow up with the squidguard info.




  • If I enable SquidGuard it just doesn't seem to filter anything.  Here is what I'm doing:

    In General Settings I've enabled the service and verified it is running.
    In the Blacklist I've downloaded from http://squidguard.mesd.k12.or.us/blacklists.tgz and waited till the log showed:

    Begin blacklist update
    Start download.
    Download archive http://squidguard.mesd.k12.or.us/blacklists.tgz
    Download complete
    Unpack archive
    Scan blacklist categories.
    Found 14 items.
    Start rebuild DB.
    Copy DB to workdir.
    Reconfigure Squid proxy.
    Blacklist update complete.

    I go into Common ACL and see the attached image.  There is no way to choose the categories and I can't seem to get any filtering done.




  • Well, now the SquidGuard service just won't start at all. :'(



  • Well, it looks like the AV issue is shown here:  https://forum.pfsense.org/index.php?topic=73921.0



  • Yet another victim of Squid3-dev….

    btw this question would be better served in the Packages forum.  The Firewall forum is specifically for firewall/rules issues.



  • True, another forum would be more suited.  Sorry.  Mods, can you move this?

    I've dropped down to Squid 2 and it's producing its own set of challenges. SquidGuard is enabled but isn't actually filtering anything.  I can see the Target Rules List and it has a bunch of categories all set to deny and I've tried the Default access as both allow and deny.  Nothing is getting blocked.

    Here is my /var/squid/logs/cache.log file

    2014/09/10 14:02:13| logfileOpen: opening log /var/squid/logs/access.log
    2014/09/10 14:02:13| Store logging disabled
    2014/09/10 14:02:13| Referer logging is disabled.
    2014/09/10 14:02:13| DNS Socket created at 0.0.0.0, port 15676, FD 15
    2014/09/10 14:02:13| Adding domain localdomain from /etc/resolv.conf
    2014/09/10 14:02:13| Adding nameserver 127.0.0.1 from /etc/resolv.conf
    2014/09/10 14:02:13| Adding nameserver 65.32.1.70 from /etc/resolv.conf
    2014/09/10 14:02:13| Adding nameserver 65.32.1.65 from /etc/resolv.conf
    2014/09/10 14:02:13| helperOpenServers: Starting 5 'squidGuard' processes
    2014-09-10 14:02:13 [89236] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:13 [89236] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:13 [89236] INFO: New setting: dbhome: /var/db/squidGuard
    2014/09/10 14:02:13| Accepting proxy HTTP connections at 192.168.1.1, port 3128, FD 24.
    2014/09/10 14:02:13| Accepting transparently proxied HTTP connections at 127.0.0.1, port 3128, FD 25.
    2014/09/10 14:02:13| Accepting HTCP messages on port 4827, FD 27.
    2014/09/10 14:02:13| Accepting SNMP messages on port 3401, FD 28.
    2014/09/10 14:02:13| WCCP Disabled.
    2014-09-10 14:02:13 [89569] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:13 [89569] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:13 [89569] INFO: New setting: dbhome: /var/db/squidGuard
    2014/09/10 14:02:13| Loaded Icons.
    2014-09-10 14:02:13 [90165] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:13 [90043] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014/09/10 14:02:13| Ready to serve requests.
    2014-09-10 14:02:13 [90043] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:13 [90043] INFO: New setting: dbhome: /var/db/squidGuard
    2014-09-10 14:02:13 [89913] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:13 [89913] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:13 [89913] INFO: New setting: dbhome: /var/db/squidGuard
    2014-09-10 14:02:13 [90165] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:13 [90165] INFO: New setting: dbhome: /var/db/squidGuard
    2014/09/10 14:02:38| Reconfiguring Squid Cache (version 2.7.STABLE9)...
    2014/09/10 14:02:38| FD 24 Closing HTTP connection
    2014/09/10 14:02:38| FD 25 Closing HTTP connection
    2014/09/10 14:02:38| FD 27 Closing HTCP socket
    2014/09/10 14:02:38| FD 28 Closing SNMP socket
    2014/09/10 14:02:38| logfileClose: closing log /var/squid/logs/access.log
    2014/09/10 14:02:38| Including Configuration File: /usr/pbi/squid-amd64/etc/squid/squid.conf (depth 0)
    2014/09/10 14:02:38| Initialising SSL.
    2014/09/10 14:02:38| logfileOpen: opening log /var/squid/logs/access.log
    2014/09/10 14:02:38| Store logging disabled
    2014/09/10 14:02:38| Referer logging is disabled.
    2014/09/10 14:02:38| DNS Socket created at 0.0.0.0, port 39900, FD 15
    2014/09/10 14:02:38| Adding domain localdomain from /etc/resolv.conf
    2014/09/10 14:02:38| Adding nameserver 127.0.0.1 from /etc/resolv.conf
    2014/09/10 14:02:38| Adding nameserver 208.67.222.222 from /etc/resolv.conf
    2014/09/10 14:02:38| helperOpenServers: Starting 5 'squidGuard' processes
    2014-09-10 14:02:38 [32376] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:38 [32376] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:38 [32376] INFO: New setting: dbhome: /var/db/squidGuard
    2014-09-10 14:02:38 [32025] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:38 [32025] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:38 [32025] INFO: New setting: dbhome: /var/db/squidGuard
    2014-09-10 14:02:38 [32277] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:38 [32277] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:38 [32277] INFO: New setting: dbhome: /var/db/squidGuard
    2014-09-10 14:02:38 [32609] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:38 [32609] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:38 [32609] INFO: New setting: dbhome: /var/db/squidGuard
    2014/09/10 14:02:38| Accepting proxy HTTP connections at 192.168.1.1, port 3128, FD 24.
    2014/09/10 14:02:38| Accepting transparently proxied HTTP connections at 127.0.0.1, port 3128, FD 25.
    2014/09/10 14:02:38| Accepting HTCP messages on port 4827, FD 27.
    2014/09/10 14:02:38| Accepting SNMP messages on port 3401, FD 28.
    2014/09/10 14:02:38| WCCP Disabled.
    2014/09/10 14:02:38| Loaded Icons.
    2014-09-10 14:02:38 [32755] (squidGuard): can't write to logfile /var/log/squidGuard.log
    2014-09-10 14:02:38 [32755] INFO: New setting: logdir: /var/squidGuard/log
    2014-09-10 14:02:38 [32755] INFO: New setting: dbhome: /var/db/squidGuard
    2014/09/10 14:02:38| Ready to serve requests.
    2014/09/10 14:24:30| WARNING: All url_rewriter processes are busy.
    2014/09/10 14:24:30| WARNING: up to 5 pending requests queued
    
    

    The only thing I see in there is that /var/log/squidGuard.log doesn't exist.  It can't write to it because it doesn't exist.  It can create it if it wants to.

    And some lines from my /var/squid/logs/access.log

    1410358852.730    146 192.168.1.10 TCP_MISS/200 16602 GET http://images.porn.com/sc/1/1669/1669237/promo/crop/302x201/promo_14.jpg - DIRECT/8.26.205.253 image/jpeg
    1410358852.748    113 192.168.1.10 TCP_MISS/200 8793 GET http://images.porn.com/sc/0/61/61007/promo/crop/302x201/promo_7.jpg - DIRECT/8.26.205.253 image/jpeg
    1410358852.751     39 192.168.1.10 TCP_MISS/200 8588 GET http://images.porn.com/sc/1/1569/1569809/promo/crop/302x201/promo_16.jpg - DIRECT/8.26.205.253 image/jpeg
    1410358852.786     74 192.168.1.10 TCP_MISS/200 3001 GET http://images.porn.com/sc/1/1530/1530617/promo/crop/302x201/promo_7.jpg - DIRECT/8.26.204.254 image/jpeg
    1410358852.786     74 192.168.1.10 TCP_MISS/200 4251 GET http://images.porn.com/assets/partner_logos/channels/323.gif - DIRECT/8.26.205.253 image/gif
    1410358852.791     77 192.168.1.10 TCP_MISS/200 4113 GET http://images.porn.com/assets/partner_logos/channels/573.gif - DIRECT/8.26.205.253 image/gif
    1410359070.770     22 192.168.1.10 TCP_MISS/200 7604 GET http://images10.newegg.com/WebResource/Scripts/USA/TP_jQueryPlugin/jquery-migrate-1.2.1.min.js - DIRECT/24.143.206.218 application/javascript
    1410359070.770     28 192.168.1.10 TCP_MISS/200 14568 GET http://images10.newegg.com/WebResource/Themes/2005/CSS/USA/home2013.v1.w.10887.0.css - DIRECT/24.143.206.218 text/css
    1410359070.778     35 192.168.1.10 TCP_MISS/200 22547 GET http://images10.newegg.com/WebResource/Scripts/USA/NeweggJS/NEG.0.2.2.js - DIRECT/24.143.206.218 application/javascript
    1410359070.780     38 192.168.1.10 TCP_MISS/200 5423 GET http://images10.newegg.com/WebResource/Themes/2005/CSS/USA/font-awesome.v1.w.10825.0.css - DIRECT/24.143.206.216 text/css
    1410359070.798     56 192.168.1.10 TCP_MISS/200 51227 GET http://images10.newegg.com/WebResource/Themes/2005/CSS/USA/newegg.v1.w.10825.0.css - DIRECT/24.143.206.218 text/css
    
    

    This is with all sites set to deny in squidguard.



  • When you installed Squid, you did check the Enable logging checkbox under Services - Proxy server?



  • Yes.  I'm assuming that is why there is data in the access log.  I also tested downloading an infected file and it let it come through.  Maybe squid is the one not working since neither squidGuard nor HAVP are catching the data.



  • Sorry, I got confused by how it can't read the access log that you then obviously displayed.

    When working with SquidGuard, it is important that you both Save your changes and then click the Apply button on the General settings tab.



  • I think I've wormed my way through most of the bugs I've come across.  I know these things aren't meant to be run on nanoBSD but with some trickery I can force the heavy reading/writing to happen strictly in RAM so I'm getting close.  My most recent issue is that even though I am allowing and whitelisting I'm still getting the 403 forbidden page.  For instance, I have searchengines set to allow and it seems to be changing the /usr/pbi/squid-amd64/etc/squid/squidGuard.conf file as seen here:

    #
    #
    dest blk_blacklists_searchengines {
            domainlist blk_blacklists_searchengines/domains
            log block.log
    }
    
    #
    dest blk_blacklists_sexuality {
            domainlist blk_blacklists_sexuality/domains
            urllist blk_blacklists_sexuality/urls
            log block.log
    }
    
    

    You can see that something has changed as compared to the sexuality section but it is still being blocked.  What else would need to be changed?  What else it it checking?  What's the next step to look at?  Thanks.



  • Or maybe not.  I changed it back to deny but nothing updated in the /usr/pbi/squid-amd64/etc/squid/squidGuard.conf file.  Grrr >:(



  • OK, so here's what was going on.  I had a startup script that would copy over all of the information from the SD card to the RAM.  It was copying it too late.  I moved it up into rc.bootup file and it's working like a charm now.  I also like that if ClamAV in HAVP doesn't detect a file, I can upload it and within 1-2 days all of my clients are protected from it.  Very nice!



  • Good work.  From what I remember, SquidGuard isn't supported under pfSense nano due to size issues on the RAM disk or something like that.  If you've got it working at all then that's great.


Log in to reply