Squid + Dansguardian issue
-
Apologize that I cannot be of more help on this but I can't replicate the problem. As a matter of fact, I can't seem to get anything to show up as "CONTENTMOD" in my logs… It makes me question whether my rules are even working!
I posted in the thread you referenced previously and asked if others were seeing these log entries and got no response...
-
Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.
-
I'll check the conf file when I get home… pretty sure it is right though because I remember viewing the thread you reference and also remember checking that it was fixed...
However... can you confirm - should I be seeing "CONTENTMOD" (or something similar) in my DG access log? Are you seeing them for situations where the query string is being modified? Thanks!
-
Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.
Hi
I had a quick look into that fix and my /usr/local/pkg/dansguardian.inc file already looks like the one that's been 'fixed'.
Any more ideas?
-
Please post screens of your ACLs that you're using, or the configuration files for them. It sounds like something is still enabled that shouldn't be.
-
I'll upload the logs, where about's can they be found?
-
Er, the logs aren't as important as your actual configs. Screenshots of your ACLs in your GUI config would suffice.
-
Hi
I've attached a zip file of all the ACL lists (there are about 30), I hope these are not overlay complicated to follow. I can upload specific screen shots if needed.
https://www.dropbox.com/s/8lb8w4g7do853bm/ScreenShots.rar
Thanks
-
Could you post your $dansguardian_dir/etc/dansguardian/lists/contentregexplist.* files, and your $dansguardian_dir/etc/dansguardian/dansguardianf*.conf files? This is definitely a case of something getting mangled by something in the "Content Lists" ACL.
-
I've only just seen this reply!
I have done some work, I removed DG and reinstalled it, removed all the config files, ACL's and blocked lists and started again. I have been through all the files that have regex in the file name and turned everything off but the problem still persists. I have just tried google.com and got the below in my syslog.
2014-01-11 08:51:31 User.Info 192.168.1.1 Jan 11 08:51:29 dansguardian[16727]: 2014.1.11 8:51:29 - 192.168.1.37 http://google.com CONTENTMOD GET 219 0 1 301 - Default - -
I'll try and post the files later.
-
-
So looking at your configs, you have a lot of uncommented lines. My first recommendation would be to completely empty your contentregexplist.Default list (cat /dev/null > /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/contentregexplist.Default) and your contentregexplist.g_Default list. Reload DG, test. I don't necessarily see anything in the files that would be doing this, but for testing purposes it's best to just have blank files in general. Let me know what effect, if any, that has, and we can go from there.
-
Done that and tested.
Now nothing shows up in the logs regarding Dansgaurdian/content mod but I still get the same results, Google page loads with the results, click a link, a long Google link is generated and then the page says 'The connection was reset' .
-
Tested it again and found some logs.
-
It's interesting that your log looks so different from what I'd expect. Are you logging to syslog instead of access.log?
When I go to google.co.uk, I am not redirected at all. Here is my log output for searching for "test" and clicking the subsequent Wikipedia link.
1389894645.231 323 1.2.3.4 TCP_MISS/301 221 GET http://google.co.uk abcd DEFAULT_PARENT/127.0.0.1 - 1389894645.404 81 1.2.3.4 TCP_MISS/302 222 GET http://www.google.co.uk abcd DEFAULT_PARENT/127.0.0.1 - 1389894653.163 313 1.2.3.4 TCP_MISS/200 421 GET http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC0QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=-RvYUsz8CdWgsQTq84CoDw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.cWc abcd DEFAULT_PARENT/127.0.0.1 text/html 1389894653.680 379 1.2.3.4 TCP_MISS/200 122872 GET http://en.wikipedia.org/wiki/Test_cricket abcd DEFAULT_PARENT/127.0.0.1 text/html 1389894653.924 195 1.2.3.4 TCP_MISS/200 117 GET http://bits.wikimedia.org/geoiplookup abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894653.983 254 1.2.3.4 TCP_MISS/200 66560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=startup&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894653.984 255 1.2.3.4 TCP_MISS/200 21414 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css 1389894653.998 269 1.2.3.4 TCP_MISS/200 181864 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.rtlcite%2Cwikihiero%7Cext.uls.nojs%7Cext.visualEditor.viewPageTarget.noscript%7Cmediawiki.legacy.commonPrint%2Cshared%7Cmw.PopUpMediaTransform%7Cskins.common.interface%7Cskins.vector.styles&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css 1389894654.056 55 1.2.3.4 TCP_MISS/200 9297 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894654.152 142 1.2.3.4 TCP_MISS/200 180709 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2CloadingSpinner%2CmwEmbedUtil%7Cmw.MwEmbedSupport&only=scripts&skin=vector&version=20140116T030122Z abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894654.291 103 1.2.3.4 TCP_MISS/200 188560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.centralNotice.bannerController%7Cext.centralauth.centralautologin%7Cext.uls.i18n%2Cinit%2Cinterface%2Cmessages%2Cpreferences%2Cwebfonts%7Cext.uls.webfonts.repository%7Cext.visualEditor.viewPageTarget.init%7Cext.wikimediaShopLink.core%7Cjquery.byteLength%2Cclient%2Ccookie%2CdelayedBind%2Ci18n%2CjStorage%2Cjson%2CmwExtension%2Ctipsy%2Cwebfonts%7Cmediawiki.Title%2CUri%2Capi%2Ccldr%2CjqueryMsg%2Clanguage%2Cnotify%2Cuser%2Cutil%7Cmediawiki.language.data%2Cinit%7Cmediawiki.legacy.ajax%2Cwikibits%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.startup%7Cskins.vector.js%7Cwikibase.client.init&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894654.898 148 1.2.3.4 TCP_MISS/200 71 GET http://meta.wikimedia.org/wiki/Special:BannerRandom?uselang=en&sitename=Wikipedia&project=wikipedia&anonymous=true&bucket=0&country=US&device=desktop&slot=10 abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894655.230 470 1.2.3.4 TCP_MISS/200 98901 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.articleFeedbackv5.startup%7Cext.cite%2CeventLogging%2CnavigationTiming%7Cext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.uls.eventlogger%7Cext.wikimediaEvents.moduleStorage%7Cjquery.articleFeedbackv5.utils%7Cjquery.autoEllipsis%2CcheckboxShiftClick%2Chidpi%2ChighlightText%2CmakeCollapsible%2Cmw-jump%2Cplaceholder%2Csuggestions%2CtabIndex%7Cmediawiki.action.view.postEdit%7Cmediawiki.hidpi%2Cinspect%2CsearchSuggest%7Cmediawiki.page.ready%7Cmobile.desktop%7Cmw.MwEmbedSupport.style%7Cmw.PopUpMediaTransform%7Cschema.ModuleStorage%2CNavigationTiming%2CUniversalLanguageSelector%7Cskins.vector.collapsibleNav&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894655.611 57 1.2.3.4 TCP_MISS/200 23313 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery.tablesorter%7Cmediawiki.language.months&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript 1389894657.825 3130 1.2.3.4 TCP_MISS/200 261 GET http://login.wikimedia.org/wiki/Special:CentralAutoLogin/checkLoggedIn?type=script&wikiid=enwiki&proto=http abcd DEFAULT_PARENT/127.0.0.1 text/javascript
I am beyond baffled at your results at this point, it doesn't seem to make any sense. If I were you, I'd try completely removing everything from your ACL configs, confirming that everything is removed with cat /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/*, reloading (dansguardian -r) and retesting. If you still see the same behavior, there is a much deeper issue here. If you do NOT see the same behavior, test after every ACL change that you make to find the culprit. My guess would be some residual rule not getting taken out correctly, but it's very hard to say.
-
I've only just seen this reply!
I have done some work, I removed DG and reinstalled it, removed all the config files, ACL's and blocked lists and started again. I have been through all the files that have regex in the file name and turned everything off but the problem still persists. I have just tried google.com and got the below in my syslog.
2014-01-11 08:51:31 User.Info 192.168.1.1 Jan 11 08:51:29 dansguardian[16727]: 2014.1.11 8:51:29 - 192.168.1.37 http://google.com CONTENTMOD GET 219 0 1 301 - Default - -
I'll try and post the files later.
problem still persists on a clean install? Are you sure there isn't something not being removed on the uninstall?
find / -name *dansguardian*
find / -name *dans* -
I have done a complete re-install, I edited the config.xml (i think?) and deleted all the sections related to DG. I have configured the new ACl's to only block ADs, Tracking and Spyware URL's/Sites. There is no filtering for Porn or Adult material, no regex stuff no content other stuf. Previously I was logging to syslog server but now I'm not, below is a snippet from googling 'test' and choosing the top result (Wikipedia) which fails to load ( the connection was reset).
2014.1.17 13:26:02 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=VC_ZUoL0Eumq7Qb_rYDoAQ&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZGU GET 421 0 1 200 text/html Default - - 2014.1.17 14:22:30 - 192.168.1.15 http://clients1.google.com/ocsp POST 3613 0 1 504 text/html Default - - application/ocsp-request,,107,0,,0; 2014.1.17 14:22:33 - 192.168.1.15 http://clients1.google.com/ocsp POST 3613 0 1 504 text/html Default - - application/ocsp-request,,107,0,,0; 2014.1.17 14:23:11 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=aTzZUoyiNI6ihge74oGQCw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZG4 GET 421 0 1 200 text/html Default - - 2014.1.17 14:29:32 - 192.168.1.15 http://clients1.google.com/ocsp POST 3613 0 1 504 text/html Default - - application/ocsp-request,,107,0,,0; 2014.1.17 14:30:14 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=ED7ZUoX2HI-shQfD_IGwAw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZG4 GET 421 0 1 200 text/html Default - -
-
Does this happen if you don't proxy SSL traffic? I was having a problem with timeouts on SSL traffic in DG at one point and never found an exact cause (and now just let SSL out unfiltered). Adding "dns_v4_first on" to my custom squid directives helped a bit, but didn't 100% fix it. I would try pushing SSL traffic out unproxied and seeing if it exhibits the same behavior.
-
My test Firefox box is configured like so
https://db.tt/GFMeUMC5
-
Working to reproduce your issue, give me a little bit.
Do you have a secondary machine you can set up as a proxy and test on? I've put in your initial rules and still can't reproduce this behavior. I'm thinking there's something going on elsewhere here.
EDIT: AHA! REPRODUCED IT! Steps to fix incoming.
EDIT2: Edit your /usr/pbi/dansguardian-amd64/etc/dansguardian/lists/contentregexplist.g_Default file. Is there text in it? If so, find a line that has this in it:
"