Dansguardian Bypass

bilbo

Is it possible to set DG up so that a standard password has to be entered in order to by pass the block page?

At the moment I have put in the -BYPASS- variable in the std block page, and in standard group 300 second time out and entered a secret key, this seems to produce a bypass url which works but I'd prefer a password box and a button. :D

bilbo

Test: webupload.

edit: worked.

bilbo

@rjcrowder:

Dansguardian override works like a champ… Here is what I did.

1. Installed the vhosts package.
I had one minor issue with this. The service status page doesn't seem to correctly display the fact that it is running. I found a workaround on the forums to fix it http://forum.pfsense.org/index.php/topic,33804.0.html.

2. Followed the instructions for setting up the override page from here http://honestpchelp.com/2011/clearos-dansguardian-accessdenied-php-bypass-script/.
This was pretty straight forward, I just had to change the directories to be appropriate to the light http web server. For example, I put the accessdenied.php file in the directory /usr/local/vhosts/vhost01.local/. Of course, I also had to change the URL's to be appropriate to my box and port. I put the password text file in /var/etc/.

I found this (Thanks RJ) and followed it through but my conf files get overwritten when DG restarts, how do I get around this?

bilbo

Ahh nevermind, I found it in the gui! dufus.

bilbo

This bypass method has been working great for me. However I have been experimenting and decided to enter the proxy settings manually in my web browser.

The problem with this however is that the bypass page seems to result in a redirect loop from what I can tell. This does not happen in transparent mode, any ideas why this may be? ???

rjcrowder

I recently switched to a different bypass approach. Mainly because there seems to be issues on some web sites with traffic flowing through dg/squid even if it is not filtering. See this http://forum.pfsense.org/index.php/topic,68872.msg377435.html#msg377435.

This approach works by dynamicaly changing the list of addresses that are redirected to dg. Note that it ONLY works if you are doing a transparent redirect. It also allows the source IP to hit any site. However - it has the advantage of completely removing dg and squid from the traffic flow when the IP is bypassing…

bilbo

Thanks RJ, you saved this thread from being completely embarrassing when I responded to my self again! And thanks for your suggestion it looks great. I'll bear that in mind if what I am using currently fails.

It seems that the proxy exceptions I had specified in the wpad.dat were wrong and so the bypass page was getting filtered again and again, seems okay for now.

The last issue I am having is how to get pfsense to host the wpad file as I kept getting redirected to the ssl webgui page when I tried hosting it in the web folder on pfsense and so have resorted in serving it from another machine for now.

rjcrowder

Hmmm… seems to me that I had a wpad file under www at one point. However, I wasn't using https for the web configurator. Not sure if that makes a difference.. Seems to me that there may be a redirect rule for it..

bilbo

Bypass is working fine and with explicit proxy setup I can block ssl sites. The only problem with ssl sites is that the block/bypass page does not show up, just a blank page when the ssl site it blocked. No redirect to the blocked page. Is this typical when blocking ssl sites?

rjcrowder

@bilbo:

Bypass is working fine and with explicit proxy setup I can block ssl sites. The only problem with ssl sites is that the block/bypass page does not show up, just a blank page when the ssl site it blocked. No redirect to the blocked page. Is this typical when blocking ssl sites?

Honestly don't know… Since you cannot easily filter SSL sites (i.e. the content filtering doesn't work unless you do a "man in the middle" setup), I've opted to do a transparent redirect. I didn't want the hassle of having to setup an explicit proxy on all machines. I tried using a wpad file for a while, but I still ran into issues where some machines would not use it (i.e. they would need explicit proxy setting).

I accomplish basically the same thing by using dans only for http and then counting on the OpenDNS name servers to blacklist https (of course it also ads an additional layer of protection on http).

vernonspangler

rjcrowder & bilbo,

So to ensure that I am inline with what you accomplishing here. Please correct me if this feed is not that conversation.

I would like to regulate what my children are authorized to see by utilizing web filtering.

Web filtering is transparent so that all traffic from a the DHCP side is being filter while my other static IP Address is unfiltered. (Note that I can setup this with Dansguardian and Squid 2.7.9)
Within the filtering process the "Access is Denied" page would need a simple password text field/Submit button that visitors could get a temporary password that will allow them to view what they would like to see. (Your thoughts?)
What I am noticing is that network performances are dramatically being reduced when I setup everything up.

Please note that my current setup is as followed.

Modem –> PFSENSE (Firewall / Gateway / OpenVPN) --> Home Switch
PFSense is a two interface setup inline with the modem to switch (WAN/LAN)

Thanks for any assistance in advance.

rjcrowder

There are tons of options here… I can describe what I have setup.

If you happen to be interested in this config, I have a script that will take a vanilla pfSense install and create what I believe is the ultimate home filtering solution. It's fairly simple to setup and I have a readme that describes how to apply it to a base pfSense install.

Keep in mind that it is very focused on creating a home filtering solution - so... single LAN/WAN, Squid, Dansguardian, OpenDNS. The script will install in one of two "flavors". The first "flavor" leaves all of the pfSense menus and screens in place but pre-configures squid, DG, some IP alias groups, and firewall rules, DGLog2 for DG reporting, etc.. The second "flavor" takes it to another level by replacing some of the pfSense screens to simplify the static IP setup, adding simple screens for creating IP based time restrictions, and removing any of the pfSense menu items that are not necessary.

So... here is what I setup on my config.

1. Squid and DG in transparent mode. A firewall rule is used to redirect all HTTP traffic to DG. DG is configured to do both blacklist and phrase filtering and downloads the lists from shalla.
2. DG is configured for IP based authentication. At the moment, I don't do anything to put different IP ranges into different DG filtering groups, but it would be very easy to do.
3. OpenDNS is used as the DNS resolution service. This gives me blacklist based HTTPS filtering.
4. I've pre-created IP aliases for ranges to allow internet access rules within the firewall. MAC addresses can be assigned to specific IP's (or ranges if you use my modified screens) and the ranges are used in rules to do time based restriction.
5. IP alias ranges that are not filtered so that you can assign devices an unfiltered IP address (such as an XBox or ROKU).
6. I've created a DG bypass page that prompts for an ID/Password. The ID/Password is either a valid pfSense user ID/Password or an ID/Password combo from a text file. The text file can be edited on the DG logging screen. If a user enters a valid ID/Password, then their IP address is allowed to fully bypass the filter for approximately 15 minutes. The bypass is implemented by excluding the IP from the the firewall redirect rule.
7. Host overrides for Google to disable HTTPS search.
8. Layer 3 checks (using IPFW) to make sure that no one tries to hijack an unfiltered IP address (compares MAC to the static assignment).
9. DGLog2 for querying and reporting activity on the DG access logs. A menu item is added under pfSense but it is also accessible without logging into the DG web console.

I'll post some links to screenshots and the setup script in case you are interested...

rjcrowder

OK… I've attached a bunch of links.

The screenshots are all from what I call an "appliance" install. The appliance setup can be created by running the apply custom script in "appliance" mode (./apply_custom.sh -a). This is my attempt to create a pfSense install and user interface that is easy and usable as a home filtering solution but also nearly impossible to break. However, even with the stripped down web UI, all of the pfSense core and packages (including the original screens) is still installed and available - just hidden.

You can also run the script to create a "base" install (./apply_custom.sh -b). The base install sets up everything I described on the previous post but leaves almost all of the pfSense menus in place.

Install files (see readme.txt for instructions)…
https://dl.dropboxusercontent.com/u/55672566/install_files/readme.txt
https://dl.dropboxusercontent.com/u/55672566/install_files/apply_custom.sh
https://dl.dropboxusercontent.com/u/55672566/install_files/pfsense_custom.tar.gz
https://dl.dropboxusercontent.com/u/55672566/install_files/fetch_blacklist.sh

vernonspangler

rjcrowder

I tried both appliance and base on your install script. I did this on a solid state HDD and I am running into a few issues.
Actually 1 major issue. It installs with no issues but after installation I have no connectivity internal via pfsense web portal in order to make any adjustments, nor do i have access to the outside internet. I do however have the ability to ssh from my workstation to the pfsense.

WAN = DHCP from ISP
LAN = 192.168.20.1/24
DHCP 192.168.20.100 - 192.268.20.200

Please keep in mind I am doing this work on a demo system that is confirmed to work with pfsense in order to work out all the issues prior to applying this to my home network.

rjcrowder

OK… Hmmm... haven't seen that particular issue. Did you follow the readme exactly? You should do an install, set the IP address for the LAN, and then run the install... That's it.

The install script copies a config.xml over the /conf/config.xml that was originally created. It then replaces the IP address range with the range you had put in your original config.xml (in this case it sounds like you would have put 192.168.20.1/24. Your DHCP range will also be overwritten to be (in this case) 192.168.20.32 -> 192.168.20.95. The IP address of your pfsense server should (of course) be 192.168.20.1

I'd ssh to the box and check the IP addresses in /conf/config.xml. It should have created everything in the 20.x range. The other thing to check... do an "ipfw -x Dummy show" and make sure that all the ipfw rules are created for the 20.x range...

vernonspangler

@rjcrowder:

The install script copies a config.xml over the /conf/config.xml that was originally created. It then replaces the IP address range with the range you had put in your original config.xml (in this case it sounds like you would have put 192.168.20.1/24. Your DHCP range will also be overwritten to be (in this case) 192.168.20.32 -> 192.168.20.95. The IP address of your pfsense server should (of course) be 192.168.20.1

Okay so I am able to get it to work now. So I have a few more questions if you do not mind.

I am noticing that I have to authenticate twice in order to access a banned webpage. (I think this is just a browser issue)
I am getting a few errors that the unable to resolve name from browser (This might be me as I have this setup currently behind another FW/GW in order to stage a quick swap)
Why xxx.xxx.20.32 -> xxx.xxx.20.95 would it make a difference if I utilized xxx.xxx.2.254 for my pfsense?

As I review your setup, to understand what it is going to take for me if I so desire to change the pfsense ip address without locking me out again.

I would say that I would have to disable Dans and squid first.
Setup firewall rules and filters on the LAN to allow the new network scheme.
Adjust my DHCP server to new IP Scheme (ensure workstation has static IP of current IP scheme)
Change the LAN IP
Adjust my workstation to match new LAN IP scheme
Adjust Dans and Squid to new IP Scheme and start services back up.

(Ultimately it would be easier for me to setup with the IP scheme that I would to utilize as the pfsense LAN ID. Your thoughts?)

I am highly impressed with the filtering and scheduling of IP ranges that was going to be my next challenge after i figured out the web filtering.

Note:
You might want to update your readme.txt as the following.

1. Install pfsense 2.1 using normal USB memstick install (Currently being tested on HDD setup)

5. Copy the following files to the box
scp apply_custom.sh root@192.168.4.1:/root/.
scp pfsense_custom.tar.gz root@192.168.4.1:/root/.
scp pkg-install.php root@192.168.4.1:/root/.
scp fetch_blacklist.sh root@192.168.4.1:/root/.

Once again I would like express my high apprieciation for the work that you have compiled.

rjcrowder

First of all… you're welcome. Glad you got it working and I'm glad that someone else sees some value in it.

You can easily change the subnet that is used (say from 192.168.20.x to 192.168.2.x) by doing the following:
1.) Edit your current config.xml and change the "ipaddr" value under the lan interface.
2.) Re-run the "apply_custom.sh" script with the -i command line option

The script goes out and changes IP addresses that are kept in some other things I added. For example, addresses can be kept in the ipfw custom rules and they are also set in the DG bypass pages. However, the script only changes the subnet (not the last part of the IP address)... So, I modified it a little bit to also change the pfSense machine address. You can run the following script (save it as whatever.sh and make it executable). Be careful... 254 should work fine, but you don't want to change it to an address that is within one of the ranges that is being using for rules...

#!/bin/sh

#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
update_ip() {
  config_file=/root/config.xml
  cp /cf/conf/config.xml $config_file

  # Prompt the user for the new lan ipv4 address and domain
  #
  echo 'Enter new LAN IP address of server (ex: 192.168.3.1):'
  read new_ip
  new_ip1=`echo $new_ip | cut -f1 -d"."`
  new_ip2=`echo $new_ip | cut -f2 -d"."`
  new_ip3=`echo $new_ip | cut -f3 -d"."`
  new_ip4=`echo $new_ip | cut -f4 -d"."`

  cfg_ip=`xmllint --xpath '/pfsense/interfaces/lan/ipaddr/text()' $config_file`
  cfg_ip1=`echo $cfg_ip | cut -f1 -d"."`
  cfg_ip2=`echo $cfg_ip | cut -f2 -d"."`
  cfg_ip3=`echo $cfg_ip | cut -f3 -d"."`
  cfg_ip4=`echo $cfg_ip | cut -f4 -d"."`

  cat $config_file | \
      sed -e "s/$cfg_ip1\.$cfg_ip2\.$cfg_ip3\.$cfg_ip4      sed -e "s/$cfg_ip1\.$cfg_ip2\.$cfg_ip3\./$new_ip1\.$new_ip2\.$new_ip3\./g" > \
      /cf/conf/config.xml

  pfs_ip=$cfg_ip
  pfs_ip1=`echo $pfs_ip | cut -f1 -d"."`
  pfs_ip2=`echo $pfs_ip | cut -f2 -d"."`
  pfs_ip3=`echo $pfs_ip | cut -f3 -d"."`
  pfs_ip4=`echo $pfs_ip | cut -f4 -d"."`

  # Update some other config files with the proper IP address
  #
  cfg_ip=`cat /usr/local/ipfw_custom_rules/checked_ranges.conf | grep -v "^#" | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/ipfw_custom_rules/checked_ranges.conf $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/ipfw_custom_rules/macip_additions.conf | grep -v "^#" | head -1 | awk '{ print $2 }'`
  update_lan_ip /usr/local/ipfw_custom_rules/macip_additions.conf $cfg_ip $new_ip

  cfg_ip=`fgrep 'action="http' /usr/local/www/dgbypass/accessdenied.php | cut -f2 -d"=" | cut -f3 -d'/'`
  update_lan_ip /usr/local/www/dgbypass/accessdenied.php $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/www/dgbypass/unfiltered | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/www/dgbypass/unfiltered $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/dgbypass/gold_unfiltered | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/dgbypass/gold_unfiltered $cfg_ip $new_ip

  cfg_ip=`fgrep 'src="http' /usr/local/www/content_filter_logs.php | cut -f2 -d"=" | cut -f3 -d'/'`
  update_lan_ip /usr/local/www/content_filter_logs.php $cfg_ip $new_ip
}

#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
update_lan_ip() {
  in_file=$1
  if [ "$2" != "" ]; then
    repl_ip1=`echo $2 | cut -f1 -d"."`
    repl_ip2=`echo $2 | cut -f2 -d"."`
    repl_ip3=`echo $2 | cut -f3 -d"."`
    new_ip1=`echo $3 | cut -f1 -d"."`
    new_ip2=`echo $3 | cut -f2 -d"."`
    new_ip3=`echo $3 | cut -f3 -d"."`
    new_ip4=`echo $3 | cut -f4 -d"."`
    mv $in_file $in_file.orig
    cat $in_file.orig | \
        sed -e "s/$pfs_ip1\.$pfs_ip2\.$pfs_ip3\.$pfs_ip4/$new_ip1\.$new_ip2\.$new_ip3\.$new_ip4/g" | \
        sed -e "s/$repl_ip1\.$repl_ip2\.$repl_ip3\./$new_ip1\.$new_ip2\.$new_ip3\./g" > \
        $in_file
  fi 
}

#-----------------------------------------------------------------------
# Main
#-----------------------------------------------------------------------

echo "Changing IP address and subnet of pfSense"

update_ip
/etc/rc.reboot

To your other question… there's no particular reason for the 32-95 range for dynamically assigned addresses other than the fact that it falls on maskable boundaries... You could change it to whatever you want as long as it doesn't conflict with ranges being used for other rules.

rjcrowder

OK… Updated the readme.txt file. Also changed the "-i" option of "apply_custom.sh" so that it will prompt you for the IP address to set the server to... it will also validate that the IP address is in a valid range (i.e. won't conflict with any of the pre-configured rules or the dynamically assigned range).

vernonspangler

This is great and helps out a ton. I was wondering if you have a script that setup for your post http://forum.pfsense.org/index.php/topic,68872.msg377435.html#msg377435

While I do appreciate this highly intense setup. Personally I do not need all the subnet rules.

My network setup is as followed and I am sure that others have something similar.

pfSense/GW/FW: LAN IP address = 192.168.x.x Subnet = 255.255.255.0
pfSense DHCP Range: 192.168.x.x - 192.168.x.x (whatever they feel that they want to issues out with the worries of subnets.)

The web filtering applies to the subnet range that applies to the interface you wish to filter… i.e. LAN, OPT.
From there a person could setup up there own aliases with a network range and apply FW rules and schedules.

Feel free to shot me a call anytime as I am off for the holidays and would enjoy speaking with you more on this...

;D ;D ;D ;D ;D ;D ;D ;D ;D

rjcrowder

At the moment, I don't have a script that only sets up the IP based dansguardian bypass. I could probably create one, but just haven't done it yet.

The script I have (in base/appliance mode) was created for several reasons. For the appliance setup, I was trying to come up with something that was a simple (hence getting the "alias groups", removal of DHCP page, replacement of the rules page, removal of most of the screens, menus, etc.) yet very solid filtering solution. My goal was to come up with something that was based solely on open source software but offered the ultimate in filtering for a home user who had a fairly standard config that was Modem <–-> pfSense <---> AccessPoint. My target was something similar to the functionality of this http://pandorashope.com/ commercial product but with a separate firewall/access point (and no yearly subscription).

The other mode of install - what I called "base" - is nothing other than what I use... It's accomplishes the same thing as the "appliance" install, but you have to know what you are doing...

There are a few limitations to what I'm adding on top of pfSense. Probably the biggest one is that I'm messing with the captive portal functionality by adding my own IPFW firewall rules (see the directory /usr/local/ipfw_custom_rules). In theory, the captive portal should still work if you change the value of "skip_captiveportal_rules" to be "false" (in my script), but I don't use the captive portal and therefore haven't tested it. The DHCP page has also been modified to add my custom rules every time you save - since the rules should be re-created every time you add a static address assignment.

I did one thing on the "appliance" setup that I would have preferred avoiding, but it made life easier. I added an element called "ipalias" into the xml for a static mapping. It just made it easier to track what alias group an IP was assigned to. If you happen to edit the static mapping with the default DHCP edit page, you will lose the ipalias value because it obviously doesn't save it...

Anyway... there are several other "features" that I added. If you want an explanation of how (or why) any of them are created the way they are - just let me know. I'd be happy to discuss.

Finally, I've thought about trying to distribute this somehow. I'm not really looking to make money, more as a ministry - I think it would be a great service to families if it could be made simple enough to setup. Certainly open to any ideas or help!