Problem with web uploads in new dansguardian 2.12.0.3



  • Since upgrading to the new version of dansguardian, I've been unable to get anything that does a web based upload to work (instagram or faxorama for example). No matter what I set "maxuploadsize" to, uploads always fail with the denied message "Web upload is banned." I can get around it by entering an "exception" for each site, but that's not what i want… I want to be able to do uploads.

    Any idea why this is not working? I've tried setting maxuploadsize to a high value and to -1 in both the general config and the config for my groups... but nothing seems to make any difference.

    Thanks!



  • Just as some added information. I have confirmed that this is only an issue for the 2.12.0.3 version of dansguardian. I copied the dans executable from the packages that were created for 2.12.0.0 and 2.12.0.2 and both work just fine using the exact same config files - just copying over the executable.

    My guess is that this bug (fairly confident it is a bug) has something to do with the "fix" relating to maxuploadsize that is noted in the 2.12.0.3 release notes…



  • Looked at the "diff" file for Patch 11 (the dg patch relating to maxuploadsize) and it seems obvious that something is amiss… I'm not sure why FOptionContainer.cpp isn't picking up the maxuploadsize from my group config (dansguardianf1.conf) file but it definitely is not...

    BTW Marcelo, if you can get 2.12.0.3 working the diff makes it clear that you can remove maxuploadsize from the dansguardian.conf (and the limits screen). The code to read it in OptionContainer.cpp was removed!

    As for me, I'm currently using the 2.12.0.2 dansguardian executable - although I'm a little nervous about the "memory leaks" that were supposedly fixed in 2.12.0.3.



  • Are you using package version 0.1.7_1?



  • Yes. 0.1.7_1  Even tried uninstall/reinstall several times. Per my previous messages, I've tried every possible combination I can think of for setting the maxuploadsize value in both the limit and group settings. I also checked that the GUI was setting maxuploadsize correctly in dansguardian.conf and dansguardianf1.conf. Bottom line… everything is working fine from the GUI package perspective, but the 2.12.0.3 dansguardian executable doesn't seem to read the setting - no matter what I try. If I simply swap the dansguardian executable for the 2.12.0.2 or 2.12.0.0 version, everything works fine.

    Also... like I said in my previous post, when/if we actually get 2.12.0.3 working - you can see from the patch diff file that the maxuploadsize setting in dansguardian.conf is no longer used. You could remove it from the "limits" page...

    @marcelloc:

    Are you using package version 0.1.7_1?



  • It's not working even if you set it to -1 on group options?



  • I had said the same last week when I installed the newly released version.

    http://forum.pfsense.org/index.php/topic,43786.msg312070.html#msg312070



  • @marcelloc:

    It's not working even if you set it to -1 on group options?

    Tried that already. Does not work.



  • Agreed…. it does not work with -1 in the group options... I also tried setting it to a value... doesn't work.

    @asterix:

    @marcelloc:

    It's not working even if you set it to -1 on group options?

    Tried that already. Does not work.



  • I've fixed the code.  ;D

    The patch 11 was malformed and broke up web upload check.

    I did not pushed it to dansguardian devel or freebsd patch yet.

    Test my dansguardian package and see if it works

    I've also included the code to limit upload size(was not working on 2.12.0.2 AFAIK)

    amd64
    http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.3_1.tbz

    i386
    http://e-sac.siteseguro.ws/packages/8/All/dansguardian-2.12.0.3_1.tbz

    Do not forget to kill dansguardian process before trying new packge binaries.



  • Thanks Marcello… that's awesome!  I'm currently traveling - so I won't be able to check it till Friday. Will let you know as soon as I do though...

    @marcelloc:

    I've fixed the code.  ;D

    The patch 11 was malformed and broke up web upload check.

    I did not pushed it to dansguardian devel or freebsd patch yet.

    Test my dansguardian package and see if it works

    I've also included the code to limit upload size(was not working on 2.12.0.2 AFAIK)

    amd64
    http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.3_1.tbz

    i386
    http://e-sac.siteseguro.ws/packages/8/All/dansguardian-2.12.0.3_1.tbz

    Do not forget to kill dansguardian process before trying new packge binaries.



  • Marcello when will you push the patch?



  • @samham:

    Marcello when will you push the patch?

    I've pushed it to dansguardian sourceforge project.

    But not for freebsd ports.

    I'm waiting feedbacks to see if there are other bugs that I can help to fix.



  • Seems to work. Thanks!



  • I am a noob to freebsd but faced the same problem above, I downlaoded the new package (modified) extracted dansguardian from sbin and pasted it in sbin of my pfsense box and it worked
    Is it necessary to replace the other files?



  • no, just dansguardian bin.

    I've uploaded dansguardian bin to my personal repo :)

    i386
    killall dansguardian
    cd /usr/local/sbin
    fetch http://e-sac.siteseguro.ws/pfsense/8/dansguardian

    amd64
    killall dansguardian
    cd /usr/local/sbin
    fetch http://e-sac.siteseguro.ws/pfsense/8/amd64/dansguardian



  • patched binary worked for me, Thanks!



  • ditto on the fix, thanks!  ;D



  • I have just set up Dansguardian for a large site and have experienced the same "web upload is banned" as reported in this thread.

    Package Manager shows 2.12.0.3pkg v.0.1.7_3.

    I've tried downloading the patched binary from http://e-sac.sitesguro.ws/pfsense/8/amd64/dansguardian
    after killing dansguardian.
    When I try to start the new binary I get the message:

    /usr/local/sbin/dansguardian: 1: Syntax error: ")" unexpected
    ./dansguardian.sh WARNING: failed to start dansguardian

    Are you able to help me here, and in particular what steps do I need to follow to resolve the "web upload is banned" problem - thanks

    PS.  to be able to save this post, I had to put an exception for forum.pfsense.org into the sites exception list, to avoid having this post blocked by the same error for which I am seeking help!



  • @neil:

    I've tried downloading the patched binary from http://e-sac.sitesguro.ws/pfsense/8/amd64/dansguardian
    after killing dansguardian.
    When I try to start the new binary I get the message:

    /usr/local/sbin/dansguardian: 1: Syntax error: ")" unexpected
    ./dansguardian.sh WARNING: failed to start dansguardian

    If you copied only dansguardian binary, it should start as it does not touch dansguardian.sh file.

    try to save dansguardian config again to recreate startup script.



  • Thanks for the quick reply - I did as you suggested, made a change in DG/PFsense, and saved the change - presumably this recreates the dansguardian.sh script - unfortunately no change.

    Just to recap all I did was:

    1. killall dansguardian
    2. cd /usr/local/sbin
    3. mv dansguardian dansguardian.orig (ie backed up the old version)
    4. fetch https://e-sac….....etc
    This retrieved a file of size 994216 dated feb 8.
    The permissions on this file did not match those on the old version and in particular were not executable.
    So I typed chmod 555 dansguardian and the permissions then matched (ie -r-xr-xr-x)
    5. cd /usr/local/etc/rc.d
    6. ./dansguardian.sh start

    This resulted in the error I reported.

    When I copy the original dansguardian binary back again, the startup script runs without any error.

    Any further thoughts?



  • @neil:

    Any further thoughts?

    No. :( Except for the mv dansguardian dansguardian.orig I've did it again here and it started up fine.



  • Hmmmm - I'm not getting anywhere here.
    If I copy the dansguardian.orig back to dansguardian and run ./dansguardian.sh start , DG starts without error.
    The "syntax error: " etc seems to be coming from the binary not from the script.

    I've tried downloading the binary again, and there is no improvement.

    Could you send the binary to me at neil@darfield.school.nz?

    Thanks



  • DOH this was my problem - my production servers are all x64 while my development server is x86.
    I was trying to apply the x64 version of DG to an x86 PFsense install.
    I've downloaded the x86 version and this starts correctly.
    Thanks for your replies



  • Also had problems uploading, also downloaded your patch, also worked. Thanks marcelloc.



  • @marcelloc:

    @samham:

    Marcello when will you push the patch?

    I've pushed it to dansguardian sourceforge project.

    But not for freebsd ports.

    I'm waiting feedbacks to see if there are other bugs that I can help to fix.

    Estava com esse problema aqui no dansguardian, só fiz substituir o arquivo que você disponibilizou e pronto!! Problema resolvido !!

    Valeuz Marcello



  • Me too! Thanks marcelloc.  8)



  • I have the same problem, but I can not download the patch. When I try ato fetch, I get a: fetch http://e-sac.sitesguro.ws/pfsense/8/dansguardian: Operation timed out

    I tried a traceroute on http://e-sac.sitesguro.ws and got the following:

    traceroute e-sac.stesguro.ws
    traceroute to e-sac.stesguro.ws (64.70.19.198), 64 hops max, 52 byte packets
    1  10.1.1.1 (10.1.1.1)  0.463 ms  0.335 ms  0.281 ms
    nexthop.qld.iinet.net.au (203.215.9.250)  14.452 ms  14.227 ms  16.294 ms
    3  150.101.33.158 (150.101.33.158)  31.716 ms  13.708 ms  16.490 ms
    ae1.br1.syd4.on.ii.net (150.101.33.18)  29.600 ms  28.967 ms  31.846 ms
    te0-0-0-1.br1.lax1.on.ii.net (203.16.213.69)  178.061 ms
        te0-1-1-2.bd1.lax1.on.ii.net (203.16.213.65)  206.041 ms
        te0-0-0-1.br1.lax1.on.ii.net (203.16.213.69)  177.792 ms
    6  144.223.30.1 (144.223.30.1)  218.953 ms  207.542 ms  203.939 ms
    sl-crs2-ana-0-15-0-0.sprintlink.net (144.232.19.226)  175.235 ms
        144.232.1.177 (144.232.1.177)  176.961 ms  179.663 ms
    sl-st21-la-13-0-0.sprintlink.net (144.232.20.69)  208.509 ms
        sl-st21-la-0-0-0.sprintlink.net (144.232.20.206)  210.669 ms  210.212 ms
    9  208.174.196.113 (208.174.196.113)  211.137 ms  208.013 ms  208.514 ms
    10  pr2-so-6-1-0.losangelesequinix.savvis.net (204.70.200.97)  191.378 ms
        pr2-so-6-0-0.losangelesequinix.savvis.net (204.70.200.93)  195.916 ms
        pr2-so-6-2-0.losangelesequinix.savvis.net (204.70.200.101)  195.221 ms
    11  hr1-te-1-0-0.irvine2oc2.savvis.net (204.70.204.189)  190.690 ms  191.125 ms  190.078 ms
    12  hr2-te-1-0-1.irvine2oc2.savvis.net (204.70.204.166)  218.562 ms  220.108 ms  221.215 ms
    13  hr1-te-1-0-0.elsegundola1.savvis.net (204.70.204.194)  188.445 ms  189.647 ms  188.262 ms
    14  das1-v3006.la1.savvis.net (64.70.11.54)  187.990 ms  190.962 ms  191.201 ms
    15  * * *
    16  * * *
    17  * * *
    18  * * *
    19  * * *
    (this went up to 44 * * * before a did a ctrl-C)

    Is this patch accessible from Australia?

    Thanks.
    Paul.



  • @engellion:

    Is this patch accessible from Australia?

    Yes, but sometimes marcelloc might turn off his server. I remember once I couldn't get something either, but OTOH I've downloaded lots of dansguardian and squid stuff from his site at other times. In fact, I've just browsed to his site now and it's accessible to me.



  • Hi there,

    As of today - September 23, 2013 - Is this patch still necessary, or has it been merged into the package?

    Phob



  • Still the same.. Hasn't changed yet..



  • Thanks for this. I tried following the instructions, but it didn't seem to work. The DG process kept crapping out after a few minutes, and browsing was quite slow. Also, DG started blocking content I didn't ask it to.

    I removed the package, but it left behind the dansguardian file in the sbin folder. I manually deleted that and have gone back to the package version for now.

    Do the permissions need to be adjusted, or should it work without further intervention?



  • Hi again,

    So I'm still having trouble with the process from earlier in this thread that fetches the new executable via fetch.

    Via SSH I did :

    1. killall dansguardian
    2. cd /usr/local/sbin
    3. fetch http://e-sac.sitesguro.ws/pfsense/8/dansguardian

    I noted Neil's earlier post, so I also:

    4. chmod 555 dansguardian
    5. cd /usr/local/etc/rc.d
    6. ./dansguardian.sh start

    I then restarted the process from the GUI (Status>Services>Dansguardian)  Seeing no improvement, I also rebooted the firewall (pfSense) box.

    I still see very slow browsing performance, and DG no longer seems to respond to the configuration changes I make in the pfSense GUI.  I've tried clicking through each tab and clicking save thinking that there could be a new configuration version or something, but it still doesn't seem to want to work.

    What is stranger, is that DG starts blocking things that couldn't possibly match what I've defined, which is essentially to only block pornography.  It even started blocking sites I used to originally learn about the solution:

    http://old.theninjageek.co.za/2013/07/02/pfsense-squid3-and-dansguardian-a-better-alternative-to-squidguard/

    For now, I'm back to the "stock" package available through the pfSense GUI.  I'm running:

    2.1-RELEASE (i386)
    built on Wed Sep 11 18:16:50 EDT 2013
    FreeBSD 8.3-RELEASE-p11

    Dansguardian Services 2.12.0.3 pkg v.0.1.8
    squid3 Network 3.1.20 pkg 2.0.6

    If anybody has some other tips to try, please let me know!

    Thanks,

    Phob



  • It appears this is still an issue in the latest release. I've tried the new binary fix, and it seems to hose the DG install beyond repair, resulting in having to completely remove and reinstall the DG package.



  • @timthetortoise:

    It appears this is still an issue in the latest release. I've tried the new binary fix, and it seems to hose the DG install beyond repair, resulting in having to completely remove and reinstall the DG package.

    Works fine if you do the following:
      1.) download the fixed dansguardian file
      2.) extract the dansguardian executable
      3.) stop dansguardian
      4.) copy the new executable over the old one
      5.) start dansguardian

    Make sure you download the proper fixed dg. I saw some posts where someone was installing the wrong binary (32 bit vs. 64 bit).



  • Indeed that was what I did. It then complained that it couldn't find dansguardian.pbiopt, and couldn't load any of the filter files. I'm guessing the patched executable wants the /usr/local/blahblahblah instead of /usr/pbi/blahblahblah directory.

    Edit: To be more precise, here is my log when I start DG after using the patched amd64 binary:

    
    Oct  3 11:20:02 firewall php: dansguardian_ldap.php: Starting Dansguardian
    Oct  3 11:20:02 firewall dansguardian[58327]: Error reading /usr/local/etc/dansguardian/dansguardianf3.conf
    Oct  3 11:20:02 firewall dansguardian[58327]: Error opening filter group config: /usr/local/etc/dansguardian/dansguardianf3.conf
    Oct  3 11:20:02 firewall dansguardian[58327]: Error reading filter group conf file(s).
    Oct  3 11:20:02 firewall dansguardian[58327]: Error parsing the dansguardian.conf file or other DansGuardian configuration files
    Oct  3 11:20:02 firewall root: /usr/local/etc/rc.d/dansguardian.sh: WARNING: failed to start dansguardian
    Oct  3 11:20:02 firewall php: dansguardian_ldap.php: The command '/usr/local/etc/rc.d/dansguardian.sh start' returned exit code '1', the output was 'kern.ipc.somaxconn: 16384 -> 16384 kern.maxfiles: 131072 -> 131072 kern.maxfilesperproc: 104856 -> 104856 kern.threads.max_threads_per_proc: 4096 -> 4096 Starting dansguardian. Error reading: /usr/local/etc/dansguardian/dansguardianf3.conf Error opening filter group config: /usr/local/etc/dansguardian/dansguardianf3.conf Error reading filter group conf file(s). Error parsing the dansguardian.conf file or other DansGuardian configuration files /usr/local/etc/rc.d/dansguardian.sh: WARNING: failed to start dansguardian' 
    
    

    I will be testing this on a different VM to see if this is amd64-specific, but it does not happen on a test 386 box.

    Edit2: not sure what the deal is on my production system, but it worked fine on my new amd64 install.
    I ended up fixing it with this command-set:

    
    rm -rf /usr/local/etc/dansguardian
    ln -s /usr/pbi/dansguardian/etc/dansguardian /usr/local/etc/dansguardian
    killall dansguardian
    cd /usr/local/sbin
    fetch http://e-sac.siteseguro.ws/pfsense/8/amd64/dansguardian
    /usr/local/etc/rc.d/dansguardian.sh start
    
    

    Now to test whether uploads are fixed or not!

    Edit3: Test

    Edit4: Hooray, it's fixed! Many thanks for the modified binary.



  • Yea… I was just in the process of posting back about the sym links. Glad you got it working!



  • I had to make a slight adjustment to what timthetortoise did for x64 on 2.1:

    rm -rf /usr/local/etc/dansguardian
    ln -s /usr/pbi/dansguardian-amd64/etc/dansguardian /usr/local/etc/dansguardian
    killall dansguardian
    cd /usr/local/sbin
    fetch http://e-sac.siteseguro.ws/pfsense/8/amd64/dansguardian
    chmod +x /usr/local/sbin/dansguardian
    /usr/local/etc/rc.d/dansguardian.sh start
    

    and now I am uploading this post via dansguardian.  ;D



  • Oops, forgot about chmod +x! Good call!



  • It will need to push a fix to freebsd ports before a new compile run.

    To have more then 1024 clients it also needs some changes on freebsd too.

    I'll try again to build a new pbi and put on my repo.


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy