Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid + Dansguardian issue

    Scheduled Pinned Locked Moved pfSense Packages
    33 Posts 4 Posters 7.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rjcrowder
      last edited by

      I tried this on my squid/dg setup and didn't have any issue…

      From this site -http://contentfilter.futuragts.com/wiki/doku.php?id=the_access.log_files  it appears that perhaps you have something in your content regular expression list that is modifying the returned content?

      1 Reply Last reply Reply Quote 0
      • L
        LordCadbury
        last edited by

        I've added the following http://forum.pfsense.org/index.php?topic=68975.0 to ACl' >URL Lists > Default Url Access List > Modify Section, Enable is ticked.

        1 Reply Last reply Reply Quote 0
        • R
          rjcrowder
          last edited by

          I will look at my rewrite rules when I get home… The difference in how I'm setup vs. what you are doing is that I force non-SSL google search using DNS overrides.

          Regardless... it seems that the rewrite stuff is what is messing you up. Can you disable it and test that things work?

          1 Reply Last reply Reply Quote 0
          • L
            LordCadbury
            last edited by

            I've just gone in and unchecked the Enable tick box, restarted Dansguardian Server and tried again.

            12-11-2013 08:19:04 User.Info 192.168.1.1 Dec 11 08:18:51 dansguardian[79155]: 2013.12.11 8:18:51 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.joules.com%2F&ei=6R-oUpH4GfLQ7Aa6tIDIAQ&usg=AFQjCNGogxNmwosX9d770DUhTMpRsazJXQ&bvm=bv.57799294,d.ZGU CONTENTMOD  GET 383 0  1 200 text/html  Default  - -
            12-11-2013 08:18:51 Local0.Info 192.168.1.1 Dec 11 08:18:38 pf:    192.168.1.15 > 224.0.0.252: igmp v2 report 224.0.0.252
            12-11-2013 08:18:51 Local0.Info 192.168.1.1 Dec 11 08:18:38 pf: 00:00:04.898251 rule 80/8(ip-option): pass in on re0: (tos 0x0, ttl 1, id 13621, offset 0, flags [none], proto IGMP (2), length 32, options (RA))
            12-11-2013 08:18:48 User.Info 192.168.1.1 Dec 11 08:18:36 dansguardian[79155]: 2013.12.11 8:18:36 - 192.168.1.15 http://clients1.google.com/ocsp CONTENTMOD  POST 3613 0  1 504 text/html  Default  - - application/ocsp-request,,107,0,,0;
            12-11-2013 08:18:47 Local0.Info 192.168.1.1 Dec 11 08:18:34 pf:    0.0.0.0 > 224.0.0.1: igmp query v2
            12-11-2013 08:18:47 Local0.Info 192.168.1.1 Dec 11 08:18:34 pf: 00:02:02.601002 rule 3/0(match): block in on re0: (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 32, options (RA))
            12-11-2013 08:18:37 User.Info 192.168.1.1 Dec 11 08:18:24 dansguardian[79155]: 2013.12.11 8:18:24 - 192.168.1.15 http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.joules.com%2F&ei=oB-oUpCiEYr17Ab-x4GQBw&usg=AFQjCNGogxNmwosX9d770DUhTMpRsazJXQ&bvm=bv.57799294,d.ZGU CONTENTMOD  GET 383 0  1 200 text/html  Default  - -

            From the sys log it looks like the content mod is still being picked up but I cant find where….

            1 Reply Last reply Reply Quote 0
            • L
              LordCadbury
              last edited by

              Can anyone offer any more advice?

              1 Reply Last reply Reply Quote 0
              • R
                rjcrowder
                last edited by

                Apologize that I cannot be of more help on this but I can't replicate the problem. As a matter of fact, I can't seem to get anything to show up as "CONTENTMOD" in my logs… It makes me question whether my rules are even working!

                I posted in the thread you referenced previously and asked if others were seeing these log entries and got no response...

                1 Reply Last reply Reply Quote 0
                • T
                  timthetortoise
                  last edited by

                  Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.

                  1 Reply Last reply Reply Quote 0
                  • R
                    rjcrowder
                    last edited by

                    I'll check the conf file when I get home… pretty sure it is right though because I remember viewing the thread you reference and also remember checking that it was fixed...

                    However... can you confirm - should I be seeing "CONTENTMOD" (or something similar) in my DG access log? Are you seeing them for situations where the query string is being modified? Thanks!

                    1 Reply Last reply Reply Quote 0
                    • L
                      LordCadbury
                      last edited by

                      @timthetortoise:

                      Does your relevant dansguardianfX.conf file look correct? If not, please try this fix, re-save your ACLs, and try again. Make absolutely sure you re-save your rules. Really, it should not need this fix since it was merged before release - but you never know.

                      Hi

                      I had a quick look into that fix and my  /usr/local/pkg/dansguardian.inc file already looks like the one that's been 'fixed'.

                      Any more ideas?

                      1 Reply Last reply Reply Quote 0
                      • T
                        timthetortoise
                        last edited by

                        Please post screens of your ACLs that you're using, or the configuration files for them. It sounds like something is still enabled that shouldn't be.

                        1 Reply Last reply Reply Quote 0
                        • L
                          LordCadbury
                          last edited by

                          I'll upload the logs, where about's can they be found?

                          1 Reply Last reply Reply Quote 0
                          • T
                            timthetortoise
                            last edited by

                            Er, the logs aren't as important as your actual configs. Screenshots of your ACLs in your GUI config would suffice.

                            1 Reply Last reply Reply Quote 0
                            • L
                              LordCadbury
                              last edited by

                              Hi

                              I've attached a zip file of all the ACL lists (there are about 30), I hope these are not overlay complicated to follow. I can upload specific screen shots if needed.

                              https://www.dropbox.com/s/8lb8w4g7do853bm/ScreenShots.rar

                              Thanks

                              1 Reply Last reply Reply Quote 0
                              • T
                                timthetortoise
                                last edited by

                                Could you post your $dansguardian_dir/etc/dansguardian/lists/contentregexplist.* files, and your $dansguardian_dir/etc/dansguardian/dansguardianf*.conf files? This is definitely a case of something getting mangled by something in the "Content Lists" ACL.

                                1 Reply Last reply Reply Quote 0
                                • L
                                  LordCadbury
                                  last edited by

                                  I've only just seen this reply!

                                  I have done some work, I removed DG and reinstalled it, removed all the config files, ACL's and blocked lists and started again. I have been through all the files that have regex in the file name and turned everything off but the problem still persists. I have just tried google.com and got the below in my syslog.

                                  2014-01-11 08:51:31 User.Info 192.168.1.1 Jan 11 08:51:29 dansguardian[16727]: 2014.1.11 8:51:29 - 192.168.1.37 http://google.com CONTENTMOD  GET 219 0  1 301 -  Default  - -

                                  I'll try and post the files later.

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    LordCadbury
                                    last edited by

                                    Files Attached.

                                    contentregexplist.Default.txt
                                    contentregexplist.g_Default.txt
                                    contentregexplist.txt
                                    dansguardian.conf.txt
                                    dansguardianf1.conf.sample.txt
                                    dansguardianf1.conf.txt

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      timthetortoise
                                      last edited by

                                      So looking at your configs, you have a lot of uncommented lines. My first recommendation would be to completely empty your contentregexplist.Default list (cat /dev/null > /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/contentregexplist.Default) and your contentregexplist.g_Default list. Reload DG, test. I don't necessarily see anything in the files that would be doing this, but for testing purposes it's best to just have blank files in general. Let me know what effect, if any, that has, and we can go from there.

                                      1 Reply Last reply Reply Quote 0
                                      • L
                                        LordCadbury
                                        last edited by

                                        Done that and tested.

                                        Now nothing shows up in the logs regarding Dansgaurdian/content mod but I still get the same results, Google page loads with the results, click a link, a long Google link is generated and then the page says 'The connection was reset' .

                                        1 Reply Last reply Reply Quote 0
                                        • L
                                          LordCadbury
                                          last edited by

                                          Tested it again and found some logs.

                                          Buda.txt

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            timthetortoise
                                            last edited by

                                            It's interesting that your log looks so different from what I'd expect. Are you logging to syslog instead of access.log?

                                            When I go to google.co.uk, I am not redirected at all. Here is my log output for searching for "test" and clicking the subsequent Wikipedia link.

                                            1389894645.231    323 1.2.3.4 TCP_MISS/301 221 GET http://google.co.uk abcd DEFAULT_PARENT/127.0.0.1 -
1389894645.404     81 1.2.3.4 TCP_MISS/302 222 GET http://www.google.co.uk abcd DEFAULT_PARENT/127.0.0.1 -
1389894653.163    313 1.2.3.4 TCP_MISS/200 421 GET http://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC0QFjAA&url=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTest_cricket&ei=-RvYUsz8CdWgsQTq84CoDw&usg=AFQjCNGPXOKClui7vHgzV25lOsr4nAq50g&bvm=bv.59568121,d.cWc abcd DEFAULT_PARENT/127.0.0.1 text/html
1389894653.680    379 1.2.3.4 TCP_MISS/200 122872 GET http://en.wikipedia.org/wiki/Test_cricket abcd DEFAULT_PARENT/127.0.0.1 text/html
1389894653.924    195 1.2.3.4 TCP_MISS/200 117 GET http://bits.wikimedia.org/geoiplookup abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894653.983    254 1.2.3.4 TCP_MISS/200 66560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=startup&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894653.984    255 1.2.3.4 TCP_MISS/200 21414 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css
1389894653.998    269 1.2.3.4 TCP_MISS/200 181864 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.rtlcite%2Cwikihiero%7Cext.uls.nojs%7Cext.visualEditor.viewPageTarget.noscript%7Cmediawiki.legacy.commonPrint%2Cshared%7Cmw.PopUpMediaTransform%7Cskins.common.interface%7Cskins.vector.styles&only=styles&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/css
1389894654.056     55 1.2.3.4 TCP_MISS/200 9297 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=scripts&skin=vector&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.152    142 1.2.3.4 TCP_MISS/200 180709 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2CloadingSpinner%2CmwEmbedUtil%7Cmw.MwEmbedSupport&only=scripts&skin=vector&version=20140116T030122Z abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.291    103 1.2.3.4 TCP_MISS/200 188560 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.centralNotice.bannerController%7Cext.centralauth.centralautologin%7Cext.uls.i18n%2Cinit%2Cinterface%2Cmessages%2Cpreferences%2Cwebfonts%7Cext.uls.webfonts.repository%7Cext.visualEditor.viewPageTarget.init%7Cext.wikimediaShopLink.core%7Cjquery.byteLength%2Cclient%2Ccookie%2CdelayedBind%2Ci18n%2CjStorage%2Cjson%2CmwExtension%2Ctipsy%2Cwebfonts%7Cmediawiki.Title%2CUri%2Capi%2Ccldr%2CjqueryMsg%2Clanguage%2Cnotify%2Cuser%2Cutil%7Cmediawiki.language.data%2Cinit%7Cmediawiki.legacy.ajax%2Cwikibits%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.startup%7Cskins.vector.js%7Cwikibase.client.init&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894654.898    148 1.2.3.4 TCP_MISS/200 71 GET http://meta.wikimedia.org/wiki/Special:BannerRandom?uselang=en&sitename=Wikipedia&project=wikipedia&anonymous=true&bucket=0&country=US&device=desktop&slot=10 abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894655.230    470 1.2.3.4 TCP_MISS/200 98901 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.articleFeedbackv5.startup%7Cext.cite%2CeventLogging%2CnavigationTiming%7Cext.gadget.DRN-wizard%2CReferenceTooltips%2Ccharinsert%2Cteahouse%7Cext.uls.eventlogger%7Cext.wikimediaEvents.moduleStorage%7Cjquery.articleFeedbackv5.utils%7Cjquery.autoEllipsis%2CcheckboxShiftClick%2Chidpi%2ChighlightText%2CmakeCollapsible%2Cmw-jump%2Cplaceholder%2Csuggestions%2CtabIndex%7Cmediawiki.action.view.postEdit%7Cmediawiki.hidpi%2Cinspect%2CsearchSuggest%7Cmediawiki.page.ready%7Cmobile.desktop%7Cmw.MwEmbedSupport.style%7Cmw.PopUpMediaTransform%7Cschema.ModuleStorage%2CNavigationTiming%2CUniversalLanguageSelector%7Cskins.vector.collapsibleNav&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894655.611     57 1.2.3.4 TCP_MISS/200 23313 GET http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery.tablesorter%7Cmediawiki.language.months&skin=vector&version=20140116T030123Z&* abcd DEFAULT_PARENT/127.0.0.1 text/javascript
1389894657.825   3130 1.2.3.4 TCP_MISS/200 261 GET http://login.wikimedia.org/wiki/Special:CentralAutoLogin/checkLoggedIn?type=script&wikiid=enwiki&proto=http abcd DEFAULT_PARENT/127.0.0.1 text/javascript

                                            I am beyond baffled at your results at this point, it doesn't seem to make any sense. If I were you, I'd try completely removing everything from your ACL configs, confirming that everything is removed with cat /usr/pbi/dansguardian-xxx/etc/dansguardian/lists/*, reloading (dansguardian -r) and retesting. If you still see the same behavior, there is a much deeper issue here. If you do NOT see the same behavior, test after every ACL change that you make to find the culprit. My guess would be some residual rule not getting taken out correctly, but it's very hard to say.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.