Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid + Dansguardian issue

    Scheduled Pinned Locked Moved pfSense Packages
    33 Posts 4 Posters 7.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      LordCadbury
      last edited by

      I've just gone in and unchecked the Enable tick box, restarted Dansguardian Server and tried again.

      12-11-2013 08:19:04 User.Info 192.168.1.1 Dec 11 08:18:51 dansguardian[79155]: 2013.12.11 8:18:51 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.joules.com%2F&ei=6R-oUpH4GfLQ7Aa6tIDIAQ&usg=AFQjCNGogxNmwosX9d770DUhTMpRsazJXQ&bvm=bv.57799294,d.ZGU CONTENTMOD  GET 383 0  1 200 text/html  Default  - -
      12-11-2013 08:18:51 Local0.Info 192.168.1.1 Dec 11 08:18:38 pf:    192.168.1.15 > 224.0.0.252: igmp v2 report 224.0.0.252
      12-11-2013 08:18:51 Local0.Info 192.168.1.1 Dec 11 08:18:38 pf: 00:00:04.898251 rule 80/8(ip-option): pass in on re0: (tos 0x0, ttl 1, id 13621, offset 0, flags [none], proto IGMP (2), length 32, options (RA))
      12-11-2013 08:18:48 User.Info 192.168.1.1 Dec 11 08:18:36 dansguardian[79155]: 2013.12.11 8:18:36 - 192.168.1.15 http://clients1.google.com/ocsp CONTENTMOD  POST 3613 0  1 504 text/html  Default  - - application/ocsp-request,,107,0,,0;
      12-11-2013 08:18:47 Local0.Info 192.168.1.1 Dec 11 08:18:34 pf:    0.0.0.0 > 224.0.0.1: igmp query v2
      12-11-2013 08:18:47 Local0.Info 192.168.1.1 Dec 11 08:18:34 pf: 00:02:02.601002 rule 3/0(match): block in on re0: (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 32, options (RA))
      12-11-2013 08:18:37 User.Info 192.168.1.1 Dec 11 08:18:24 dansguardian[79155]: 2013.12.11 8:18:24 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.joules.com%2F&ei=oB-oUpCiEYr17Ab-x4GQBw&usg=AFQjCNGogxNmwosX9d770DUhTMpRsazJXQ&bvm=bv.57799294,d.ZGU CONTENTMOD  GET 383 0  1 200 text/html  Default  - -

      From the sys log it looks like the content mod is still being picked up but I cant find where….

      1 Reply Last reply Reply Quote 0
      • L
        LordCadbury
        last edited by

        Can anyone offer any more advice?

        1 Reply Last reply Reply Quote 0
        • R
          rjcrowder
          last edited by

          Apologize that I cannot be of more help on this but I can't replicate the problem. As a matter of fact, I can't seem to get anything to show up as "CONTENTMOD" in my logs… It makes me question whether my rules are even working!

          I posted in the thread you referenced previously and asked if others were seeing these log entries and got no response...

          1 Reply Last reply Reply Quote 0
          • T
            timthetortoise
            last edited by

            Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.

            1 Reply Last reply Reply Quote 0
            • R
              rjcrowder
              last edited by

              I'll check the conf file when I get home… pretty sure it is right though because I remember viewing the thread you reference and also remember checking that it was fixed...

              However... can you confirm - should I be seeing "CONTENTMOD" (or something similar) in my DG access log? Are you seeing them for situations where the query string is being modified? Thanks!

              1 Reply Last reply Reply Quote 0
              • L
                LordCadbury
                last edited by

                @timthetortoise:

                Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.

                Hi

                I had a quick look into that fix and my  /usr/local/pkg/dansguardian.inc file already looks like the one that's been 'fixed'.

                Any more ideas?

                1 Reply Last reply Reply Quote 0
                • T
                  timthetortoise
                  last edited by

                  Please post screens of your ACLs that you're using, or the configuration files for them. It sounds like something is still enabled that shouldn't be.

                  1 Reply Last reply Reply Quote 0
                  • L
                    LordCadbury
                    last edited by

                    I'll upload the logs, where about's can they be found?

                    1 Reply Last reply Reply Quote 0
                    • T
                      timthetortoise
                      last edited by

                      Er, the logs aren't as important as your actual configs. Screenshots of your ACLs in your GUI config would suffice.

                      1 Reply Last reply Reply Quote 0
                      • L
                        LordCadbury
                        last edited by

                        Hi

                        I've attached a zip file of all the ACL lists (there are about 30), I hope these are not overlay complicated to follow. I can upload specific screen shots if needed.

                        https://www.dropbox.com/s/8lb8w4g7do853bm/ScreenShots.rar

                        Thanks

                        1 Reply Last reply Reply Quote 0
                        • T
                          timthetortoise
                          last edited by

                          Could you post your $dansguardian_dir/etc/dansguardian/lists/contentregexplist.* files, and your $dansguardian_dir/etc/dansguardian/dansguardianf*.conf files? This is definitely a case of something getting mangled by something in the "Content Lists" ACL.

                          1 Reply Last reply Reply Quote 0
                          • L
                            LordCadbury
                            last edited by

                            I've only just seen this reply!

                            I have done some work, I removed DG and reinstalled it, removed all the config files, ACL's and blocked lists and started again. I have been through all the files that have regex in the file name and turned everything off but the problem still persists. I have just tried google.com and got the below in my syslog.

                            2014-01-11 08:51:31 User.Info 192.168.1.1 Jan 11 08:51:29 dansguardian[16727]: 2014.1.11 8:51:29 - 192.168.1.37 http://google.com CONTENTMOD  GET 219 0  1 301 -  Default  - -

                            I'll try and post the files later.

                            1 Reply Last reply Reply Quote 0
                            • L
                              LordCadbury
                              last edited by

                              Files Attached.

                              contentregexplist.Default.txt
                              contentregexplist.g_Default.txt
                              contentregexplist.txt
                              dansguardian.conf.txt
                              dansguardianf1.conf.sample.txt
                              dansguardianf1.conf.txt

                              1 Reply Last reply Reply Quote 0
                              • T
                                timthetortoise
                                last edited by

                                So looking at your configs, you have a lot of uncommented lines. My first recommendation would be to completely empty your contentregexplist.Default list (cat /dev/null > /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/contentregexplist.Default) and your contentregexplist.g_Default list. Reload DG, test. I don't necessarily see anything in the files that would be doing this, but for testing purposes it's best to just have blank files in general. Let me know what effect, if any, that has, and we can go from there.

                                1 Reply Last reply Reply Quote 0
                                • L
                                  LordCadbury
                                  last edited by

                                  Done that and tested.

                                  Now nothing shows up in the logs regarding Dansgaurdian/content mod but I still get the same results, Google page loads with the results, click a link, a long Google link is generated and then the page says 'The connection was reset' .

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    LordCadbury
                                    last edited by

                                    Tested it again and found some logs.

                                    Buda.txt

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      timthetortoise
                                      last edited by

                                      It's interesting that your log looks so different from what I'd expect. Are you logging to syslog instead of access.log?

                                      When I go to google.co.uk, I am not redirected at all. Here is my log output for searching for "test" and clicking the subsequent Wikipedia link.

                                      1389894645.231    323 1.2.3.4 TCP_MISS/301 221 GET http://google.co.uk abcd DEFAULT_PARENT/127.0.0.1 -
1389894645.404     81 1.2.3.4 TCP_MISS/302 222 GET http://www.google.co.uk abcd DEFAULT_PARENT/127.0.0.1 -
1389894653.163    313 1.2.3.4 TCP_MISS/200 421 GET http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC0QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=-RvYUsz8CdWgsQTq84CoDw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.cWc abcd DEFAULT_PARENT/127.0.0.1 text/html
1389894653.680    379 1.2.3.4 TCP_MISS/200 122872 GET http://en.wikipedia.org/wiki/Test_cricket abcd DEFAULT_PARENT/127.0.0.1 text/html
1389894653.924    195 1.2.3.4 TCP_MISS/200 117 GET http://bits.wikimedia.org/geoiplookup abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894653.983    254 1.2.3.4 TCP_MISS/200 66560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=startup&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894653.984    255 1.2.3.4 TCP_MISS/200 21414 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css
1389894653.998    269 1.2.3.4 TCP_MISS/200 181864 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.rtlcite%2Cwikihiero%7Cext.uls.nojs%7Cext.visualEditor.viewPageTarget.noscript%7Cmediawiki.legacy.commonPrint%2Cshared%7Cmw.PopUpMediaTransform%7Cskins.common.interface%7Cskins.vector.styles&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css
1389894654.056     55 1.2.3.4 TCP_MISS/200 9297 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.152    142 1.2.3.4 TCP_MISS/200 180709 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2CloadingSpinner%2CmwEmbedUtil%7Cmw.MwEmbedSupport&only=scripts&skin=vector&version=20140116T030122Z abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.291    103 1.2.3.4 TCP_MISS/200 188560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.centralNotice.bannerController%7Cext.centralauth.centralautologin%7Cext.uls.i18n%2Cinit%2Cinterface%2Cmessages%2Cpreferences%2Cwebfonts%7Cext.uls.webfonts.repository%7Cext.visualEditor.viewPageTarget.init%7Cext.wikimediaShopLink.core%7Cjquery.byteLength%2Cclient%2Ccookie%2CdelayedBind%2Ci18n%2CjStorage%2Cjson%2CmwExtension%2Ctipsy%2Cwebfonts%7Cmediawiki.Title%2CUri%2Capi%2Ccldr%2CjqueryMsg%2Clanguage%2Cnotify%2Cuser%2Cutil%7Cmediawiki.language.data%2Cinit%7Cmediawiki.legacy.ajax%2Cwikibits%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.startup%7Cskins.vector.js%7Cwikibase.client.init&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.898    148 1.2.3.4 TCP_MISS/200 71 GET http://meta.wikimedia.org/wiki/Special:BannerRandom?uselang=en&sitename=Wikipedia&project=wikipedia&anonymous=true&bucket=0&country=US&device=desktop&slot=10 abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894655.230    470 1.2.3.4 TCP_MISS/200 98901 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.articleFeedbackv5.startup%7Cext.cite%2CeventLogging%2CnavigationTiming%7Cext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.uls.eventlogger%7Cext.wikimediaEvents.moduleStorage%7Cjquery.articleFeedbackv5.utils%7Cjquery.autoEllipsis%2CcheckboxShiftClick%2Chidpi%2ChighlightText%2CmakeCollapsible%2Cmw-jump%2Cplaceholder%2Csuggestions%2CtabIndex%7Cmediawiki.action.view.postEdit%7Cmediawiki.hidpi%2Cinspect%2CsearchSuggest%7Cmediawiki.page.ready%7Cmobile.desktop%7Cmw.MwEmbedSupport.style%7Cmw.PopUpMediaTransform%7Cschema.ModuleStorage%2CNavigationTiming%2CUniversalLanguageSelector%7Cskins.vector.collapsibleNav&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894655.611     57 1.2.3.4 TCP_MISS/200 23313 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery.tablesorter%7Cmediawiki.language.months&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894657.825   3130 1.2.3.4 TCP_MISS/200 261 GET http://login.wikimedia.org/wiki/Special:CentralAutoLogin/checkLoggedIn?type=script&wikiid=enwiki&proto=http abcd DEFAULT_PARENT/127.0.0.1 text/javascript

                                      I am beyond baffled at your results at this point, it doesn't seem to make any sense. If I were you, I'd try completely removing everything from your ACL configs, confirming that everything is removed with cat /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/*, reloading (dansguardian -r) and retesting. If you still see the same behavior, there is a much deeper issue here. If you do NOT see the same behavior, test after every ACL change that you make to find the culprit. My guess would be some residual rule not getting taken out correctly, but it's very hard to say.

                                      1 Reply Last reply Reply Quote 0
                                      • B
                                        bryan.paradis
                                        last edited by

                                        @LordCadbury:

                                        I've only just seen this reply!

                                        I have done some work, I removed DG and reinstalled it, removed all the config files, ACL's and blocked lists and started again. I have been through all the files that have regex in the file name and turned everything off but the problem still persists. I have just tried google.com and got the below in my syslog.

                                        2014-01-11 08:51:31 User.Info 192.168.1.1 Jan 11 08:51:29 dansguardian[16727]: 2014.1.11 8:51:29 - 192.168.1.37 http://google.com CONTENTMOD  GET 219 0  1 301 -  Default  - -

                                        I'll try and post the files later.

                                        problem still persists on a clean install? Are you sure there isn't something not being removed on the uninstall?

                                        find / -name *dansguardian*
                                        find / -name *dans*

                                        1 Reply Last reply Reply Quote 0
                                        • L
                                          LordCadbury
                                          last edited by

                                          I have done a complete re-install, I edited the config.xml (i think?) and deleted all the sections related to DG. I have configured the new ACl's to only block ADs, Tracking and Spyware URL's/Sites. There is no filtering for Porn or Adult material, no regex stuff no content other stuf. Previously I was logging to syslog server but now I'm not, below is a snippet from googling 'test' and choosing the top result (Wikipedia) which fails to load ( the connection was reset).

                                          
2014.1.17 13:26:02 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=VC_ZUoL0Eumq7Qb_rYDoAQ&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZGU  GET 421 0  1 200 text/html  Default   - - 
2014.1.17 14:22:30 - 192.168.1.15 http://clients1.google.com/ocsp  POST 3613 0  1 504 text/html  Default   - - application/ocsp-request,,107,0,,0;
2014.1.17 14:22:33 - 192.168.1.15 http://clients1.google.com/ocsp  POST 3613 0  1 504 text/html  Default   - - application/ocsp-request,,107,0,,0;
2014.1.17 14:23:11 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=aTzZUoyiNI6ihge74oGQCw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZG4  GET 421 0  1 200 text/html  Default   - - 
2014.1.17 14:29:32 - 192.168.1.15 http://clients1.google.com/ocsp  POST 3613 0  1 504 text/html  Default   - - application/ocsp-request,,107,0,,0;
2014.1.17 14:30:14 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=ED7ZUoX2HI-shQfD_IGwAw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.ZG4  GET 421 0  1 200 text/html  Default   - -
                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            timthetortoise
                                            last edited by

                                            Does this happen if you don't proxy SSL traffic? I was having a problem with timeouts on SSL traffic in DG at one point and never found an exact cause (and now just let SSL out unfiltered). Adding "dns_v4_first on" to my custom squid directives helped a bit, but didn't 100% fix it. I would try pushing SSL traffic out unproxied and seeing if it exhibits the same behavior.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.