Dansguardian Bypass

vernonspangler

rjcrowder

I tried both appliance and base on your install script. I did this on a solid state HDD and I am running into a few issues.
Actually 1 major issue. It installs with no issues but after installation I have no connectivity internal via pfsense web portal in order to make any adjustments, nor do i have access to the outside internet. I do however have the ability to ssh from my workstation to the pfsense.

WAN = DHCP from ISP
LAN = 192.168.20.1/24
DHCP 192.168.20.100 - 192.268.20.200

Please keep in mind I am doing this work on a demo system that is confirmed to work with pfsense in order to work out all the issues prior to applying this to my home network.

rjcrowder

OK… Hmmm... haven't seen that particular issue. Did you follow the readme exactly? You should do an install, set the IP address for the LAN, and then run the install... That's it.

The install script copies a config.xml over the /conf/config.xml that was originally created. It then replaces the IP address range with the range you had put in your original config.xml (in this case it sounds like you would have put 192.168.20.1/24. Your DHCP range will also be overwritten to be (in this case) 192.168.20.32 -> 192.168.20.95. The IP address of your pfsense server should (of course) be 192.168.20.1

I'd ssh to the box and check the IP addresses in /conf/config.xml. It should have created everything in the 20.x range. The other thing to check... do an "ipfw -x Dummy show" and make sure that all the ipfw rules are created for the 20.x range...

vernonspangler

@rjcrowder:

The install script copies a config.xml over the /conf/config.xml that was originally created. It then replaces the IP address range with the range you had put in your original config.xml (in this case it sounds like you would have put 192.168.20.1/24. Your DHCP range will also be overwritten to be (in this case) 192.168.20.32 -> 192.168.20.95. The IP address of your pfsense server should (of course) be 192.168.20.1

Okay so I am able to get it to work now. So I have a few more questions if you do not mind.

I am noticing that I have to authenticate twice in order to access a banned webpage. (I think this is just a browser issue)
I am getting a few errors that the unable to resolve name from browser (This might be me as I have this setup currently behind another FW/GW in order to stage a quick swap)
Why xxx.xxx.20.32 -> xxx.xxx.20.95 would it make a difference if I utilized xxx.xxx.2.254 for my pfsense?

As I review your setup, to understand what it is going to take for me if I so desire to change the pfsense ip address without locking me out again.

I would say that I would have to disable Dans and squid first.
Setup firewall rules and filters on the LAN to allow the new network scheme.
Adjust my DHCP server to new IP Scheme (ensure workstation has static IP of current IP scheme)
Change the LAN IP
Adjust my workstation to match new LAN IP scheme
Adjust Dans and Squid to new IP Scheme and start services back up.

(Ultimately it would be easier for me to setup with the IP scheme that I would to utilize as the pfsense LAN ID. Your thoughts?)

I am highly impressed with the filtering and scheduling of IP ranges that was going to be my next challenge after i figured out the web filtering.

Note:
You might want to update your readme.txt as the following.

1. Install pfsense 2.1 using normal USB memstick install (Currently being tested on HDD setup)

5. Copy the following files to the box
scp apply_custom.sh root@192.168.4.1:/root/.
scp pfsense_custom.tar.gz root@192.168.4.1:/root/.
scp pkg-install.php root@192.168.4.1:/root/.
scp fetch_blacklist.sh root@192.168.4.1:/root/.

Once again I would like express my high apprieciation for the work that you have compiled.

rjcrowder

First of all… you're welcome. Glad you got it working and I'm glad that someone else sees some value in it.

You can easily change the subnet that is used (say from 192.168.20.x to 192.168.2.x) by doing the following:
1.) Edit your current config.xml and change the "ipaddr" value under the lan interface.
2.) Re-run the "apply_custom.sh" script with the -i command line option

The script goes out and changes IP addresses that are kept in some other things I added. For example, addresses can be kept in the ipfw custom rules and they are also set in the DG bypass pages. However, the script only changes the subnet (not the last part of the IP address)... So, I modified it a little bit to also change the pfSense machine address. You can run the following script (save it as whatever.sh and make it executable). Be careful... 254 should work fine, but you don't want to change it to an address that is within one of the ranges that is being using for rules...

#!/bin/sh

#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
update_ip() {
  config_file=/root/config.xml
  cp /cf/conf/config.xml $config_file

  # Prompt the user for the new lan ipv4 address and domain
  #
  echo 'Enter new LAN IP address of server (ex: 192.168.3.1):'
  read new_ip
  new_ip1=`echo $new_ip | cut -f1 -d"."`
  new_ip2=`echo $new_ip | cut -f2 -d"."`
  new_ip3=`echo $new_ip | cut -f3 -d"."`
  new_ip4=`echo $new_ip | cut -f4 -d"."`

  cfg_ip=`xmllint --xpath '/pfsense/interfaces/lan/ipaddr/text()' $config_file`
  cfg_ip1=`echo $cfg_ip | cut -f1 -d"."`
  cfg_ip2=`echo $cfg_ip | cut -f2 -d"."`
  cfg_ip3=`echo $cfg_ip | cut -f3 -d"."`
  cfg_ip4=`echo $cfg_ip | cut -f4 -d"."`

  cat $config_file | \
      sed -e "s/$cfg_ip1\.$cfg_ip2\.$cfg_ip3\.$cfg_ip4      sed -e "s/$cfg_ip1\.$cfg_ip2\.$cfg_ip3\./$new_ip1\.$new_ip2\.$new_ip3\./g" > \
      /cf/conf/config.xml

  pfs_ip=$cfg_ip
  pfs_ip1=`echo $pfs_ip | cut -f1 -d"."`
  pfs_ip2=`echo $pfs_ip | cut -f2 -d"."`
  pfs_ip3=`echo $pfs_ip | cut -f3 -d"."`
  pfs_ip4=`echo $pfs_ip | cut -f4 -d"."`

  # Update some other config files with the proper IP address
  #
  cfg_ip=`cat /usr/local/ipfw_custom_rules/checked_ranges.conf | grep -v "^#" | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/ipfw_custom_rules/checked_ranges.conf $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/ipfw_custom_rules/macip_additions.conf | grep -v "^#" | head -1 | awk '{ print $2 }'`
  update_lan_ip /usr/local/ipfw_custom_rules/macip_additions.conf $cfg_ip $new_ip

  cfg_ip=`fgrep 'action="http' /usr/local/www/dgbypass/accessdenied.php | cut -f2 -d"=" | cut -f3 -d'/'`
  update_lan_ip /usr/local/www/dgbypass/accessdenied.php $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/www/dgbypass/unfiltered | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/www/dgbypass/unfiltered $cfg_ip $new_ip

  cfg_ip=`cat /usr/local/dgbypass/gold_unfiltered | head -1 | cut -f1 -d'/'`
  update_lan_ip /usr/local/dgbypass/gold_unfiltered $cfg_ip $new_ip

  cfg_ip=`fgrep 'src="http' /usr/local/www/content_filter_logs.php | cut -f2 -d"=" | cut -f3 -d'/'`
  update_lan_ip /usr/local/www/content_filter_logs.php $cfg_ip $new_ip
}

#-----------------------------------------------------------------------
#-----------------------------------------------------------------------
update_lan_ip() {
  in_file=$1
  if [ "$2" != "" ]; then
    repl_ip1=`echo $2 | cut -f1 -d"."`
    repl_ip2=`echo $2 | cut -f2 -d"."`
    repl_ip3=`echo $2 | cut -f3 -d"."`
    new_ip1=`echo $3 | cut -f1 -d"."`
    new_ip2=`echo $3 | cut -f2 -d"."`
    new_ip3=`echo $3 | cut -f3 -d"."`
    new_ip4=`echo $3 | cut -f4 -d"."`
    mv $in_file $in_file.orig
    cat $in_file.orig | \
        sed -e "s/$pfs_ip1\.$pfs_ip2\.$pfs_ip3\.$pfs_ip4/$new_ip1\.$new_ip2\.$new_ip3\.$new_ip4/g" | \
        sed -e "s/$repl_ip1\.$repl_ip2\.$repl_ip3\./$new_ip1\.$new_ip2\.$new_ip3\./g" > \
        $in_file
  fi 
}

#-----------------------------------------------------------------------
# Main
#-----------------------------------------------------------------------

echo "Changing IP address and subnet of pfSense"

update_ip
/etc/rc.reboot

To your other question… there's no particular reason for the 32-95 range for dynamically assigned addresses other than the fact that it falls on maskable boundaries... You could change it to whatever you want as long as it doesn't conflict with ranges being used for other rules.

rjcrowder

OK… Updated the readme.txt file. Also changed the "-i" option of "apply_custom.sh" so that it will prompt you for the IP address to set the server to... it will also validate that the IP address is in a valid range (i.e. won't conflict with any of the pre-configured rules or the dynamically assigned range).

vernonspangler

This is great and helps out a ton. I was wondering if you have a script that setup for your post http://forum.pfsense.org/index.php/topic,68872.msg377435.html#msg377435

While I do appreciate this highly intense setup. Personally I do not need all the subnet rules.

My network setup is as followed and I am sure that others have something similar.

pfSense/GW/FW: LAN IP address = 192.168.x.x Subnet = 255.255.255.0
pfSense DHCP Range: 192.168.x.x - 192.168.x.x (whatever they feel that they want to issues out with the worries of subnets.)

The web filtering applies to the subnet range that applies to the interface you wish to filter… i.e. LAN, OPT.
From there a person could setup up there own aliases with a network range and apply FW rules and schedules.

Feel free to shot me a call anytime as I am off for the holidays and would enjoy speaking with you more on this...

;D ;D ;D ;D ;D ;D ;D ;D ;D

rjcrowder

At the moment, I don't have a script that only sets up the IP based dansguardian bypass. I could probably create one, but just haven't done it yet.

The script I have (in base/appliance mode) was created for several reasons. For the appliance setup, I was trying to come up with something that was a simple (hence getting the "alias groups", removal of DHCP page, replacement of the rules page, removal of most of the screens, menus, etc.) yet very solid filtering solution. My goal was to come up with something that was based solely on open source software but offered the ultimate in filtering for a home user who had a fairly standard config that was Modem <–-> pfSense <---> AccessPoint. My target was something similar to the functionality of this http://pandorashope.com/ commercial product but with a separate firewall/access point (and no yearly subscription).

The other mode of install - what I called "base" - is nothing other than what I use... It's accomplishes the same thing as the "appliance" install, but you have to know what you are doing...

There are a few limitations to what I'm adding on top of pfSense. Probably the biggest one is that I'm messing with the captive portal functionality by adding my own IPFW firewall rules (see the directory /usr/local/ipfw_custom_rules). In theory, the captive portal should still work if you change the value of "skip_captiveportal_rules" to be "false" (in my script), but I don't use the captive portal and therefore haven't tested it. The DHCP page has also been modified to add my custom rules every time you save - since the rules should be re-created every time you add a static address assignment.

I did one thing on the "appliance" setup that I would have preferred avoiding, but it made life easier. I added an element called "ipalias" into the xml for a static mapping. It just made it easier to track what alias group an IP was assigned to. If you happen to edit the static mapping with the default DHCP edit page, you will lose the ipalias value because it obviously doesn't save it...

Anyway... there are several other "features" that I added. If you want an explanation of how (or why) any of them are created the way they are - just let me know. I'd be happy to discuss.

Finally, I've thought about trying to distribute this somehow. I'm not really looking to make money, more as a ministry - I think it would be a great service to families if it could be made simple enough to setup. Certainly open to any ideas or help!

mrnadj

rjcrowder & all,

I could really use some help.
I have installed the script with the -b option, and it almost works.

My configuration is:
WAN = 192.168.1.100 /24 + ( GW: 192.168.1.1=adsl )
LAN = 192.168.1.2 /24 + (No Gateway) + (dhcpServer 192.168.1.34 - 192.168.1.64)

I had to create a new rule in NAT:
: -> *:80 redirect-> 192.168.1.2:8080
basically any traffic, redirect to DANS on port 8080

I have tested on a VirtualBox Client, and it gets a DHCP assigned address, with the 192.168.1.2 as gateway,
and it all seems to work. Its requests are even logged by Dans.

But when I test with a Actual Laptop, it gets the DHCP assigned address, with the 192.168.1.2 as gateway,
but no HTTP Traffic seems to work, and its requests are not logged by Dans.

I have tried apply_custom.sh -i, but it did not fix anything.
I have also tried adding PASS rules to firewall and disabling any Block rules, still no go.

As I understand it, this should only require like 1 NAT Rule, and maybe 1 LAN FW Rule, 1 WAN FW Rule?

rjcrowder

From what you describe, I can't figure what is wrong. Let me make a couple of statements (that might be helpful) and throw out one potential issue that you can check…

1.) The install script I created copies a config.xml in place. That config.xml sets the DHCP assigned range based on the fact that other ranges of addresses are used for specific devices. It sounds like you stuck with the address setup that I had configured so that's good. The config.xml had some pre-created rules that redirected to dans (port 8080) so you should not have needed to create your own rule. However, I don't know that I ever tried it with an address other than a ".1" for the gateway (think you did ".2").

2. One of the other modification I did was turning on the captive portal to enable the IPFW firewall. It then creates rules to skip the normal portal rules and check certain address ranges to make sure that no one is "hijacking" mac addresses. This is particularly important for the address ranges that are unfiltered - you don't want someone to get around the filter by manually setting their IP to one that is unfiltered. One of the things I've discovered is that there are situations where pfSense will see multiple MAC addresses for the same IP!!! The two things that I know can cause this to occur are a wireless access point configured as a wireless adapter (such as for an xbox) and a virtual machine running on a host with its own MAC address. I made a way to add additional "valid" MAC's for the same IP by adding them to a file that is used when the IPFW rules are created. The file is manually editable under /usr/local/ipfw_macip directory. In a newer version of my modifications (not posted on dropbox yet), I also added it to the DHCP screen.

You can easily check to see if traffic is hitting the box for a MAC and also see if it is being blocked in IPFW by looking at the IPFW rule listing. I believe the command is "ipfw -x Dummy list" (don't remember for sure... might be "show" instead of list).

Hope that's helpful...

series_rp

i found this problem too… thank for shareing...guy :D

mrnadj

rjcrowder ,

Thank you again for your guide, and assistance!

I thought this was impossible for me, after already spending about 5 days on it.

I gave it another go today, I was able to get Firefox, with manual HTTP Proxy to connect to Dans on 8080.

Once I had that working I set the Laptop to use PFS as Gateway.
With all the firewall rules deleted, it simply forwarded all traffic directed at the gateway.
I could see this on the console, option 10), pf Logs

I then added a Reject All traffic rule
Then I added a NAT rule for port 80.
Then a FW Allow rule for 443.

It now works well. Thank you rjcrowder. I am a programmer, and even I found this beyond challenging!

rjcrowder

Hmmm… sorry about that. I intended these scripts to be an easy way to setup a very specific configuration. I've never had any issue as long as I've stuck strictly to the intended use case. The downside of that approach is that I haven't tried a lot of variations (multiple gateway boxes, different gateway addresses, etc.) and I'm sure there are multiple ways it could be broken. However, if you can pin down issues with the install process or instructions (or give me enough info that I can find them) I'd love to know what they were so that I can try to fix them.

I'm a software guy by trade as well. What I've learned about networking has been purely by playing with stuff like this. Nice to see someone else branching out...