Dansguardian unusable

totalimpact

I really like this package, and I bet Marcello has done a lot of hard work on it (thanks), but it simply does not work… using DG 2.12.0.3 and pf2.1.

It all boils down to Bypass... DG when set to where it should be, can have some false positives - this makes having a by-pass method mandatory, and the by-pass must have at least basic security.

1. MAJOR error, many sites when bypassed receive a browser error, both Chrome and FF (maybe others) give some sort of error about the headers having a comma in them, when a semicolon is expected, exact error here:
http://stackoverflow.com/questions/12348242/debugging-multiple-disposition-headers
Results in a url like so: http://www.youtube.com/?GBYPASS=BDFCC0B18D31A41D235D902707ECCD6F1383756560
error: Duplicate headers received from server

2. Bypass never prompts for a password

3. Cannot get auth to work in any manner other than IP
need some way of securing bypass

4. Minor: clam directories and perms are out of wack, I can manually clean this up, but shouldnt that be built in to the package

If anyone has a working guide for DG and pf2.1, I would love to try it, but I have read through several other posts, and none seem to work.

I need it to block pages, and provide a link to bypass...when the user clicks bypass, there needs to be either a password prompt, or an LDAP/AD auth check to see if this user is allowed to bypass, then it MUST properly show the web page without errors.

serialdie

@totalimpact:

I really like this package, and I bet Marcello has done a lot of hard work on it (thanks), but it simply does not work… using DG 2.12.0.3 and pf2.1.

It all boils down to Bypass... DG when set to where it should be, can have some false positives - this makes having a by-pass method mandatory, and the by-pass must have at least basic security.

1. MAJOR error, many sites when bypassed receive a browser error, both Chrome and FF (maybe others) give some sort of error about the headers having a comma in them, when a semicolon is expected, exact error here:
http://stackoverflow.com/questions/12348242/debugging-multiple-disposition-headers
Results in a url like so: http://www.youtube.com/?GBYPASS=BDFCC0B18D31A41D235D902707ECCD6F1383756560
error: Duplicate headers received from server

2. Bypass never prompts for a password

3. Cannot get auth to work in any manner other than IP
need some way of securing bypass

4. Minor: clam directories and perms are out of wack, I can manually clean this up, but shouldnt that be built in to the package

If anyone has a working guide for DG and pf2.1, I would love to try it, but I have read through several other posts, and none seem to work.

I need it to block pages, and provide a link to bypass...when the user clicks bypass, there needs to be either a password prompt, or an LDAP/AD auth check to see if this user is allowed to bypass, then it MUST properly show the web page without errors.

Dansguardian works just fine for me.

squid transparent mode
forward traffic to 8080

totalimpact

serialdie: That is great to hear.

Are you using the Bypass function?

How do you have Bypass setup?

Which "Transparent" method are you using? via Squid Transparent setting, or via NAT rule to redirect 80 > 8080?

marcelloc

check this post to fix some problems on 2.1

http://forum.pfsense.org/index.php/topic,58442.msg376363.html#msg376363

dansguardian code has no updates for a while and some fixes need to be pushed to freebsd ports before a new compile run for 2.1

totalimpact

Thanks for your response Marecello, I had noticed that thread the other day, just noticed its been pretty active.

I just did a fresh install, and ran the recent patch from that post you linked:
pf 2.1-RELEASE amd64
DG 2.12.0.3
Squid 3.1.20

All auth methods disabled on DG and squid, transparent mode off in squid, using NAT redirect, Bypass enabled.

Still getting error, which I think is supposed to be avoided by checking Disable VIA in Squid, but I think this setting is being ignored.

if I go to match.com (or facebook) and bypass the filter warning, I get this:

rjcrowder

The first time I implemented bypass, I used the DG bypass feature. However, there were some situations where either DG or squid were still causing problems with whatever site I was trying to hit - even though DG was not filtering.

So… I've implemented a completely different bypass approach that meets my needs... I'll describe and you can let me know if you are interested in the code. In a nutshell, I created a bypass page that does not use the DG bypass feature at all. Instead, the bypass page adds the users IP address to an exception list on the rule that redirects to port 8080 (i.e. a "not" of an alias in the rule that redirects to the DG port). A cron job removes the IP address from the exception list after 15 minutes.

In order to "authenticate" a user to perform the bypass, the page checks two things. First it compares the entered id/password to the pfsense admin id/password. If that doesn't get a match, it looks for the id/password in a text file. I modified the DG report/log screen to allow editing of the id/password text file.

The advantage of this approach is that it completely removes DG/Squid from the flow - there is no way that DG or squid will interfere with the users (i.e. that IP address) hitting any site they want for the time period that the IP address is on the exception list.

This approach will work under the following circumstances...
1.) You are redirecting (transparent redirect) users to the DG port.
2.) Every user has a unique IP address (i.e. not coming through a proxy).
3.) You are comfortable allowing an incoming IP to completely bypass the filter for a time period (i.e. the bypass is not site specific).

totalimpact

YES - LOVE IT!…. I saw your post on this previously, but not having "easy" access to the user list would be a problem, if you have it coded to be shown in the web ui - that would be perfect.

How can I get your updated code?

...thanks in advance

rjcrowder

@totalimpact:

YES - LOVE IT!…. I saw your post on this previously, but not having "easy" access to the user list would be a problem, if you have it coded to be shown in the web ui - that would be perfect.

How can I get your updated code?

...thanks in advance

I'm out of town right now… Soon as I get back home I will figure out how to package it up for you.

rjcrowder

So, TotalImpact… I was working on packaging up this change for you and I'm trying to figure out how much I need to automate the setup. How familiar are you with working at the Unix command prompt? Basically, there are two directories and about a half dozen files, and a patch that needs to be applied (to one of the dansguardian xml files).

marcelloc

I do not have that free time but I can help. :)

totalimpact

I would say high level intermediate… more familiar with red Hat based stuff than bsd, definitely not an expert there... but I already have vhosts installed, an alias prepped, and I suppose I will need to edit some cron jobs and other stuff...

give me the quick and dirty and will let you know if I miss anything.

rjcrowder

OK… I'm gonna try this by just attaching some files and giving instructions. First of all, you don't actually need to install vHosts. I recently switched to using a directory under /usr/local/www and it works fine. So... here goes.

I've attached several files as follows (rename to remove the ".txt"):

dgbypass.tar.gz - should be untar'd in the directory /usr/local/. Will create /usr/local/dgbypass
dgbypass_www.tar.gz - should be untar'd in the directory /usr/local/www. Will create /usr/local/www/dgbypass
dansguardian.inc.patch - should be applied to dansguardian.inc
dansguardian_log.xml.patch - should be applied to dansguardian_log.xml
my_logrotation.tar.gz should be untar'd in the directory /usr/local. Will create /usr/local/my_logrotation (not required).
Two screenshots - one of the modified dansguardian log page and the other of the bypass page.

Create a URL alias called "internal_unfiltered" and point it to the url "http://192.168.5.1/dgbypass/unfiltered" (change 192.168.5.1 to the address of your pfsense server). Note that I have one address range already in the file called unfiltered - it is a range that I assign to devices that should never be filtered (such as the ROKU's and the xBox). Address are added to the unfiltered file by the accessdenied.php page (it calls /usr/local/dgbypass/dgbypass) to be allowed to temporarily bypass the filter. They are tracked and removed by an entry in /usr/local/dgbypass/current_bypass.list.

I assume you already have a firewall rule that transparently redirects users to port 8080 (or whatever port you are using for dansguardian). In that rule, add a "not" source of your alias called "internal_unfiltered".

Edit the following files and change all reference to "192.168.5." to your domain (such as "192.168.1")
/usr/local/www/dgbypass/unfiltered
/usr/local/www/dgbypass/gold_unfiltered
/usr/local/www/dgbypass/accessdenied.php
/usr/local/dgbypass/

Also, if you want to have a different range of addresses allowed to bypass DG all the time, then change the address range in /usr/local/www/dgbypass/unfiltered and /usr/local/dgbypass/gold_unfiltered.

Apply the patches to /usr/local/pkg/dansguardian.inc and /usr/local/pkg/dansguardian_log.xml (patch -N file_name < /patches/common/file_name.patch).

Now go into the dansguardian "report and log" page and setup the entries as they look in the attached screenshot. Of course, you'll need to change the IP address to be the IP of your machine.

Create a directory "/var/log/dgbypass"

Add a cron entry to check for removing the bypass addresses every 2 minutes (or more frequently if you want). The cron entry looks like this...
"*/2 * * * * root /usr/local/dgbypass/expire_bypass_ips >> /var/log/dgbypass/expire_bypass_ips.log". The easiest way to enter it is to just install the cron package.

Believe that should do it! When you attempt to hit a blocked site, you should get the bypass page pictured in the last screenshot. The tar file for "my_logrotation" is optional - it's a shell script that I schedule via cron for rotating logs...

Note that I have all of the above steps automated via scripts and patches for any "fresh" install that I do... It would be easier if I could just pass that script to you, but it does way more than just the changes described above...

![Screenshot from 2013-11-09 19:51:40.png](/public/imported_attachments/1/Screenshot from 2013-11-09 19:51:40.png)
![Screenshot from 2013-11-09 19:51:40.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-09 19:51:40.png_thumb)
![Screenshot from 2013-11-09 19:53:22.png](/public/imported_attachments/1/Screenshot from 2013-11-09 19:53:22.png)
![Screenshot from 2013-11-09 19:53:22.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-09 19:53:22.png_thumb)
dgbypass.tar.gz.txt
dgbypass_www.tar.gz.txt
my_logrotation.tar.gz.txt
dansguardian.inc.patch.txt
dansguardian_log.xml.patch.txt

totalimpact

Thanks so much…this looks like a piece of cake, will try it out later today.

rjcrowder

Yea… let me know if I missed something. More than happy to help. Prob a little bit of a pain to setup, but once it is setup, you can just add/delete usernames and passwords on the report/log page (or just use admin/pfsense_password).

Like I said before, the big advantage is that you are completely bypassing DG with this approach...

totalimpact

ok…got everything setup, i made an initial mistake on making the alias, forgot to make it a url 'table', got that straightened out, but the override function doesnt work. The access denied page opens, then I submit a username/password and it just bounces back to the access denied page again...

i found the override script, and running it manually works perfect:

/usr/local/dgbypass(45): ./dgbypass 10.0.9.11 john match.com
1 addresses added.

The alias table then has my ip 10.0.9.11 in it, and I can browse 100% unfiltered.

i verified my username/pass are in /usr/local/dgbypass/passwords.txt, i cleared everything out of this file except the user:

[2.1-RELEASE][root@router2.localdomain]/usr/local/dgbypass(11): cat passwords.txt 
john 123

it seems the accessdenied.php page is not properly calling the dgbypass script, and I am not having much luck debugging the php.

rjcrowder

OK… if it is just bouncing back to the accessdenied.php page, then it is not validating your id/password combination. If you enter valid id/password it goes to a "proceed" page (see attached). Note... the "proceed" page is in the same php file.

Try your pfsense admin id/password first (it gets it from config.xml) and see if that works. If the php cannot find it that way, then it goes to passwords.txt and looks for a userid/password there...

![Screenshot from 2013-11-10 15:58:15.png](/public/imported_attachments/1/Screenshot from 2013-11-10 15:58:15.png)
![Screenshot from 2013-11-10 15:58:15.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-10 15:58:15.png_thumb)

rjcrowder

btw…
The code to check config.xml starts at line 186 of the accessdenied.php.
The code to check "passwords.txt" starts at line 197
The code to execute dgbypass starts at line 217

totalimpact

Yes, I read through all your php, I can decipher it all, but I am just worthless at modifying or validating it, I saw that several functions/pages are in that one.

Using the pfsense admin login does not work either, I even made a 2nd pfsense admin account, dont work…any more ideas?

rjcrowder

The attached version writes a bunch of log statements to /var/log/dgbypass/accessdenied.log. Can you copy it into /usr/local/www/dgbypass and let me know what is written to the log?

BTW… be sure to change the IP address in the php (192.168.4.1) to the address of your pfsense server...

accessdenied.php.txt

totalimpact

was on the road all day… here is what i got:

cat /var/log/dgbypass/accessdenied.log
[11-Nov-2013 21:04:55 America/Los_Angeles] Starting
[11-Nov-2013 21:04:55 America/Los_Angeles] clientip [10.0.9.11]
[11-Nov-2013 21:04:55 America/Los_Angeles] url2     [DENIEDURL==http%3a%2f%2fmatch%2ecom::IP==10.0.9.11::USER==10.0.9.11::CATEGORIES==::GBYPASS==69555D8FAFA8A886D99A6227DDEA2FF41384232725::REASON==Banned%20site%3a%20match%2ecom]
[11-Nov-2013 21:05:03 America/Los_Angeles] Checking ID/Password
[11-Nov-2013 21:05:03 America/Los_Angeles] username []
[11-Nov-2013 21:05:03 America/Los_Angeles] passwd   []
[11-Nov-2013 21:05:03 America/Los_Angeles] exec-1   []
[11-Nov-2013 21:05:03 America/Los_Angeles] ID/Password NOT found in config.xml
[11-Nov-2013 21:05:03 America/Los_Angeles] ID/Password NOT found in passwords.txt
[11-Nov-2013 21:05:03 America/Los_Angeles] Starting
[11-Nov-2013 21:05:03 America/Los_Angeles] clientip [10.0.9.11]
[11-Nov-2013 21:05:03 America/Los_Angeles] url2     [::DENIEDURL==http%3a%2f%2fmatch%2ecom::IP==10.0.9.11::USER==10.0.9.11::CATEGORIES==::GBYPASS==69555D8FAFA8A886D99A6227DDEA2FF41384232725::REASON==Banned%20site%3a%20match%2ecom]

Used this user/password:

cat /usr/local/dgbypass/passwords.txt 
john zxcv

totalimpact

ok… i figured it out, not sure why it was showing the accessdenied page at all... the default pfsense port 80>443 redirect rule was messing with it, i specified the bypass page on port 80, it would display for some reason, then choke when bypassing because i didnt specify https in the php code.

THANKS SO MUCH - this is excellent.

rjcrowder

Glad you got it working…

Just in case you're interested, I've spent a lot of time trying to make pfSense the "ultimate home filtering solution". I've even gone so far as creating stripped down menu and rewritten a number of the screens in order to make it easier for a non-technical person to administer. My intention is to offer pre-configured cheap atom boxes for anyone interested in the uber home filtering solution (really as a ministry - not to make money).

Some features of the home filter box setup...
o Very simplified screens focused purely on:
- applying web access time restrictions
- administering the content filter
- removal of anything that could be confusing to the "non-technical"
o Screen to easily assign MAC addresses to "IP Group Aliases".
o Screen to apply time blocking schedules to IP Group Aliases
o Bypass feature for the content filter
o Interface to query/view the content filter logs (dglog2)

Obviously, this setup is very limited in features intended for a specific purpose... but it is also very simple and difficult to "break". It assumes a very specific configuration (i.e. squid, dg, two interfaces of LAN/WAN, no VPN or traffic shaping, etc.) However, it's also easy to switch back to the default pfSense menu when necessary and turn on more features. I also have instructions and scripts that pretty much automate the setup...

Some of the more technical things that I've implemented "under the covers" that you might be interested in:
o The filter bypass (based on IP address not being redirected to the filter)
o Layer 3 checking (using ipfw) of mac/IP combinations to make sure no one "hijacks" an unfiltered IP address
o DNS entries to force non-SSL google search (so it can be content filtered)
o Dynamic update of the addresses it resolves to for non-SSL search (in case they change)
o Implementation of dglog2.pl script for querying/reporting the content filter logs
o Block usage of any name servers other than OpenDNS

Anyway... probably total overkill. Just thought you or others might be interested. I've included links to some screen shots.

totalimpact

That looks awesome… I definitely dont want to encroach on your potential business, but if you wished to offer it, I would be interested some of those ssl search redirect features you have setup.

Right now this thing does exactly what I was in dire need of with dg, I probably wouldnt use your gui setup, because there are some other features I use, but I wouldn't mind tinkering with the ssl redirect.

rjcrowder

@totalimpact:

That looks awesome… I definitely dont want to encroach on your potential business, but if you wished to offer it, I would be interested some of those ssl search redirect features you have setup.

Right now this thing does exactly what I was in dire need of with dg, I probably wouldnt use your gui setup, because there are some other features I use, but I wouldn't mind tinkering with the ssl redirect.

That one is pretty simple…

1.) Create a directory /usr/local/update_dns_overrides
2.) Create a log directory - I use /var/log/update_dns_overrides
3.) Copy attached php file (minus .txt) into the directory
4.) Create the "host override" entries on the attached screenshot.
5.) Run the php script via cron on whatever timeframe you want

My cron entry is:
/usr/local/bin/php -q /usr/local/update_dns_overrides/update_dns_overrides.php >> /var/log/update_dns_overrides/update_dns_overrides.log

Note that this is a little bit of a hack. The script looks for the description starting with "ip=" and updates the override address to the address that URL following "=" resolves to...

Obviously, you wouldn't have to have my little update script - you could just create the entries. However, this covers you if the address for your override ever changes.

![Screenshot from 2013-11-12 12:00:07.png](/public/imported_attachments/1/Screenshot from 2013-11-12 12:00:07.png)
![Screenshot from 2013-11-12 12:00:07.png_thumb](/public/imported_attachments/1/Screenshot from 2013-11-12 12:00:07.png_thumb)
update_dns_overrides.php.txt

totalimpact

great, thanks for the tip, I will check that out sometime soon…