Dansguardian unusable

totalimpact

I really like this package, and I bet Marcello has done a lot of hard work on it (thanks), but it simply does not work… using DG 2.12.0.3 and pf2.1.

It all boils down to Bypass... DG when set to where it should be, can have some false positives - this makes having a by-pass method mandatory, and the by-pass must have at least basic security.

1. MAJOR error, many sites when bypassed receive a browser error, both Chrome and FF (maybe others) give some sort of error about the headers having a comma in them, when a semicolon is expected, exact error here:
http://stackoverflow.com/questions/12348242/debugging-multiple-disposition-headers
Results in a url like so: http://www.youtube.com/?GBYPASS=BDFCC0B18D31A41D235D902707ECCD6F1383756560
error: Duplicate headers received from server

2. Bypass never prompts for a password

3. Cannot get auth to work in any manner other than IP
need some way of securing bypass

4. Minor: clam directories and perms are out of wack, I can manually clean this up, but shouldnt that be built in to the package

If anyone has a working guide for DG and pf2.1, I would love to try it, but I have read through several other posts, and none seem to work.

I need it to block pages, and provide a link to bypass...when the user clicks bypass, there needs to be either a password prompt, or an LDAP/AD auth check to see if this user is allowed to bypass, then it MUST properly show the web page without errors.

serialdie

@totalimpact:

I really like this package, and I bet Marcello has done a lot of hard work on it (thanks), but it simply does not work… using DG 2.12.0.3 and pf2.1.

It all boils down to Bypass... DG when set to where it should be, can have some false positives - this makes having a by-pass method mandatory, and the by-pass must have at least basic security.

1. MAJOR error, many sites when bypassed receive a browser error, both Chrome and FF (maybe others) give some sort of error about the headers having a comma in them, when a semicolon is expected, exact error here:
http://stackoverflow.com/questions/12348242/debugging-multiple-disposition-headers
Results in a url like so: http://www.youtube.com/?GBYPASS=BDFCC0B18D31A41D235D902707ECCD6F1383756560
error: Duplicate headers received from server

2. Bypass never prompts for a password

3. Cannot get auth to work in any manner other than IP
need some way of securing bypass

4. Minor: clam directories and perms are out of wack, I can manually clean this up, but shouldnt that be built in to the package

If anyone has a working guide for DG and pf2.1, I would love to try it, but I have read through several other posts, and none seem to work.

I need it to block pages, and provide a link to bypass...when the user clicks bypass, there needs to be either a password prompt, or an LDAP/AD auth check to see if this user is allowed to bypass, then it MUST properly show the web page without errors.

Dansguardian works just fine for me.

squid transparent mode
forward traffic to 8080

totalimpact

serialdie: That is great to hear.

Are you using the Bypass function?

How do you have Bypass setup?

Which "Transparent" method are you using? via Squid Transparent setting, or via NAT rule to redirect 80 > 8080?

marcelloc

check this post to fix some problems on 2.1

http://forum.pfsense.org/index.php/topic,58442.msg376363.html#msg376363

dansguardian code has no updates for a while and some fixes need to be pushed to freebsd ports before a new compile run for 2.1

totalimpact

Thanks for your response Marecello, I had noticed that thread the other day, just noticed its been pretty active.

I just did a fresh install, and ran the recent patch from that post you linked:
pf 2.1-RELEASE amd64
DG 2.12.0.3
Squid 3.1.20

All auth methods disabled on DG and squid, transparent mode off in squid, using NAT redirect, Bypass enabled.

Still getting error, which I think is supposed to be avoided by checking Disable VIA in Squid, but I think this setting is being ignored.

if I go to match.com (or facebook) and bypass the filter warning, I get this:

rjcrowder

The first time I implemented bypass, I used the DG bypass feature. However, there were some situations where either DG or squid were still causing problems with whatever site I was trying to hit - even though DG was not filtering.

So… I've implemented a completely different bypass approach that meets my needs... I'll describe and you can let me know if you are interested in the code. In a nutshell, I created a bypass page that does not use the DG bypass feature at all. Instead, the bypass page adds the users IP address to an exception list on the rule that redirects to port 8080 (i.e. a "not" of an alias in the rule that redirects to the DG port). A cron job removes the IP address from the exception list after 15 minutes.

In order to "authenticate" a user to perform the bypass, the page checks two things. First it compares the entered id/password to the pfsense admin id/password. If that doesn't get a match, it looks for the id/password in a text file. I modified the DG report/log screen to allow editing of the id/password text file.

The advantage of this approach is that it completely removes DG/Squid from the flow - there is no way that DG or squid will interfere with the users (i.e. that IP address) hitting any site they want for the time period that the IP address is on the exception list.

This approach will work under the following circumstances...
1.) You are redirecting (transparent redirect) users to the DG port.
2.) Every user has a unique IP address (i.e. not coming through a proxy).
3.) You are comfortable allowing an incoming IP to completely bypass the filter for a time period (i.e. the bypass is not site specific).

totalimpact

YES - LOVE IT!…. I saw your post on this previously, but not having "easy" access to the user list would be a problem, if you have it coded to be shown in the web ui - that would be perfect.

How can I get your updated code?

...thanks in advance

rjcrowder

@totalimpact:

YES - LOVE IT!…. I saw your post on this previously, but not having "easy" access to the user list would be a problem, if you have it coded to be shown in the web ui - that would be perfect.

How can I get your updated code?

...thanks in advance

I'm out of town right now… Soon as I get back home I will figure out how to package it up for you.

rjcrowder

So, TotalImpact… I was working on packaging up this change for you and I'm trying to figure out how much I need to automate the setup. How familiar are you with working at the Unix command prompt? Basically, there are two directories and about a half dozen files, and a patch that needs to be applied (to one of the dansguardian xml files).

marcelloc

I do not have that free time but I can help. :)

totalimpact

I would say high level intermediate… more familiar with red Hat based stuff than bsd, definitely not an expert there... but I already have vhosts installed, an alias prepped, and I suppose I will need to edit some cron jobs and other stuff...

give me the quick and dirty and will let you know if I miss anything.

rjcrowder

OK… I'm gonna try this by just attaching some files and giving instructions. First of all, you don't actually need to install vHosts. I recently switched to using a directory under /usr/local/www and it works fine. So... here goes.

I've attached several files as follows (rename to remove the ".txt"):

dgbypass.tar.gz - should be untar'd in the directory /usr/local/. Will create /usr/local/dgbypass
dgbypass_www.tar.gz - should be untar'd in the directory /usr/local/www. Will create /usr/local/www/dgbypass
dansguardian.inc.patch - should be applied to dansguardian.inc
dansguardian_log.xml.patch - should be applied to dansguardian_log.xml
my_logrotation.tar.gz should be untar'd in the directory /usr/local. Will create /usr/local/my_logrotation (not required).
Two screenshots - one of the modified dansguardian log page and the other of the bypass page.

Create a URL alias called "internal_unfiltered" and point it to the url "http://192.168.5.1/dgbypass/unfiltered" (change 192.168.5.1 to the address of your pfsense server). Note that I have one address range already in the file called unfiltered - it is a range that I assign to devices that should never be filtered (such as the ROKU's and the xBox). Address are added to the unfiltered file by the accessdenied.php page (it calls /usr/local/dgbypass/dgbypass) to be allowed to temporarily bypass the filter. They are tracked and removed by an entry in /usr/local/dgbypass/current_bypass.list.

I assume you already have a firewall rule that transparently redirects users to port 8080 (or whatever port you are using for dansguardian). In that rule, add a "not" source of your alias called "internal_unfiltered".

Edit the following files and change all reference to "192.168.5." to your domain (such as "192.168.1")
/usr/local/www/dgbypass/unfiltered
/usr/local/www/dgbypass/gold_unfiltered
/usr/local/www/dgbypass/accessdenied.php
/usr/local/dgbypass/

Also, if you want to have a different range of addresses allowed to bypass DG all the time, then change the address range in /usr/local/www/dgbypass/unfiltered and /usr/local/dgbypass/gold_unfiltered.

Apply the patches to /usr/local/pkg/dansguardian.inc and /usr/local/pkg/dansguardian_log.xml (patch -N file_name < /patches/common/file_name.patch).

Now go into the dansguardian "report and log" page and setup the entries as they look in the attached screenshot. Of course, you'll need to change the IP address to be the IP of your machine.

Create a directory "/var/log/dgbypass"

Add a cron entry to check for removing the bypass addresses every 2 minutes (or more frequently if you want). The cron entry looks like this...
"*/2 * * * * root /usr/local/dgbypass/expire_bypass_ips >> /var/log/dgbypass/expire_bypass_ips.log". The easiest way to enter it is to just install the cron package.

Believe that should do it! When you attempt to hit a blocked site, you should get the bypass page pictured in the last screenshot. The tar file for "my_logrotation" is optional - it's a shell script that I schedule via cron for rotating logs...

Note that I have all of the above steps automated via scripts and patches for any "fresh" install that I do... It would be easier if I could just pass that script to you, but it does way more than just the changes described above...

![Screenshot from 2013-11-09 19:51:40.png](/public/imported_attachments/1/Screenshot from 2013-11-09 19:51:40.png)
![Screenshot from 2013-11-09 19:51:40.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-09 19:51:40.png_thumb)
![Screenshot from 2013-11-09 19:53:22.png](/public/imported_attachments/1/Screenshot from 2013-11-09 19:53:22.png)
![Screenshot from 2013-11-09 19:53:22.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-09 19:53:22.png_thumb)
dgbypass.tar.gz.txt
dgbypass_www.tar.gz.txt
my_logrotation.tar.gz.txt
dansguardian.inc.patch.txt
dansguardian_log.xml.patch.txt

totalimpact

Thanks so much…this looks like a piece of cake, will try it out later today.

rjcrowder

Yea… let me know if I missed something. More than happy to help. Prob a little bit of a pain to setup, but once it is setup, you can just add/delete usernames and passwords on the report/log page (or just use admin/pfsense_password).

Like I said before, the big advantage is that you are completely bypassing DG with this approach...

totalimpact

ok…got everything setup, i made an initial mistake on making the alias, forgot to make it a url 'table', got that straightened out, but the override function doesnt work. The access denied page opens, then I submit a username/password and it just bounces back to the access denied page again...

i found the override script, and running it manually works perfect:

/usr/local/dgbypass(45): ./dgbypass 10.0.9.11 john match.com
1 addresses added.

The alias table then has my ip 10.0.9.11 in it, and I can browse 100% unfiltered.

i verified my username/pass are in /usr/local/dgbypass/passwords.txt, i cleared everything out of this file except the user:

[2.1-RELEASE][root@router2.localdomain]/usr/local/dgbypass(11): cat passwords.txt 
john 123

it seems the accessdenied.php page is not properly calling the dgbypass script, and I am not having much luck debugging the php.

rjcrowder

OK… if it is just bouncing back to the accessdenied.php page, then it is not validating your id/password combination. If you enter valid id/password it goes to a "proceed" page (see attached). Note... the "proceed" page is in the same php file.

Try your pfsense admin id/password first (it gets it from config.xml) and see if that works. If the php cannot find it that way, then it goes to passwords.txt and looks for a userid/password there...

![Screenshot from 2013-11-10 15:58:15.png](/public/imported_attachments/1/Screenshot from 2013-11-10 15:58:15.png)
![Screenshot from 2013-11-10 15:58:15.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-10 15:58:15.png_thumb)

rjcrowder

btw…
The code to check config.xml starts at line 186 of the accessdenied.php.
The code to check "passwords.txt" starts at line 197
The code to execute dgbypass starts at line 217

totalimpact

Yes, I read through all your php, I can decipher it all, but I am just worthless at modifying or validating it, I saw that several functions/pages are in that one.

Using the pfsense admin login does not work either, I even made a 2nd pfsense admin account, dont work…any more ideas?

rjcrowder

The attached version writes a bunch of log statements to /var/log/dgbypass/accessdenied.log. Can you copy it into /usr/local/www/dgbypass and let me know what is written to the log?

BTW… be sure to change the IP address in the php (192.168.4.1) to the address of your pfsense server...

accessdenied.php.txt

totalimpact

was on the road all day… here is what i got:

cat /var/log/dgbypass/accessdenied.log
[11-Nov-2013 21:04:55 America/Los_Angeles] Starting
[11-Nov-2013 21:04:55 America/Los_Angeles] clientip [10.0.9.11]
[11-Nov-2013 21:04:55 America/Los_Angeles] url2     [DENIEDURL==http%3a%2f%2fmatch%2ecom::IP==10.0.9.11::USER==10.0.9.11::CATEGORIES==::GBYPASS==69555D8FAFA8A886D99A6227DDEA2FF41384232725::REASON==Banned%20site%3a%20match%2ecom]
[11-Nov-2013 21:05:03 America/Los_Angeles] Checking ID/Password
[11-Nov-2013 21:05:03 America/Los_Angeles] username []
[11-Nov-2013 21:05:03 America/Los_Angeles] passwd   []
[11-Nov-2013 21:05:03 America/Los_Angeles] exec-1   []
[11-Nov-2013 21:05:03 America/Los_Angeles] ID/Password NOT found in config.xml
[11-Nov-2013 21:05:03 America/Los_Angeles] ID/Password NOT found in passwords.txt
[11-Nov-2013 21:05:03 America/Los_Angeles] Starting
[11-Nov-2013 21:05:03 America/Los_Angeles] clientip [10.0.9.11]
[11-Nov-2013 21:05:03 America/Los_Angeles] url2     [::DENIEDURL==http%3a%2f%2fmatch%2ecom::IP==10.0.9.11::USER==10.0.9.11::CATEGORIES==::GBYPASS==69555D8FAFA8A886D99A6227DDEA2FF41384232725::REASON==Banned%20site%3a%20match%2ecom]

Used this user/password:

cat /usr/local/dgbypass/passwords.txt 
john zxcv