Filter porn, virus's and ads with squid3, HAVP, Dansguardian and privoxy.

[berrance]

Create an internet filter using Pfsense, squid3, HAVP, Dansguardian and privoxy. This filter will screen incoming web traffic for virus's, banner ads and inappropriate content such as porn. The filter will also be set up transparently so there is no client configuration necessary or that kids/employees can bypass the filter.

To set the filter up you will ideally need an old PC with 2 network ports. The machine used in this howto is an old P4 1.7Ghz with 1GB ram. One network port will plug directly into your network and one into your modem/router. This way there is no direct path from the network to the internet without going though the filter box.

The Main reason for writing this howto is because I simply couldn’t find a howto out on the interwobble on this setup. I could find plenty of howto's to set up Pfsense with virus and porn blocking but non whatsoever on setting up Pfsense with ad blocking, let alone all 3 combined. I used to use IPcop with this setup for years instead of Pfsense but my network outgrew what could be done with IPcop.

Please note that Privoxy is not an official Pfsense addon so you may not get any support for it on these forums. Also I originally wrote this howto for howto forge (http://www.howtoforge.com/node/7237). All of this howto except this paragraph is a direct copy of the howto forge howto, which is also a direct copy and past from my personal internal docs wiki.

First head over to www.pfsense.org and get your self a Pfsense ISO and burn it to disk. Once you have done that install it onto what will become your filter box. There are plenty of howto's out there on how to burn ISO's and default install of Pfsense so this will not be covered in this howto. Also make sure you have installed any updates for Pfsense! The version of Pfsense used in this howto is 2.0.1.

Network parameters for this howto

WAN

*IP - 192.168.1.10
*netmask - 255.255.255.0
*gateway - 192.168.1.1

LAN

*IP - 10.20.99.254
*netmask - 255.255.0.0
*This is the gateway so no gateway is needed however once the filter is set up clients on your lan will use this machine as there gateway.

You may need to change these perimeters for your own network

Once you have a default install of pfsense first make sure you can access the internet from a client inside the network through the pfsense box.

Client configuration

*IP - 10.20.99.5
*Netmask - 255.255.0.0
*Gateway - 10.20.99.254 (Pfsense lan IP)

If you can access the internet things are looking good and you have a default install of Pfsense up and running.

Now for the real work to begin.

Head over to your Pfsense lan IP in your browser, 10.20.99.254 and login (defaults are user: admin Pass; pfsense)

First we need to change the default port on which Pfsense serves the config pages. In the system menu click on advanced and change the TCP port, I used 8443. Save the setup and enter the new address (10.20.99.254:8443) in your web browser and relogin. We have done this because to make the filter transparent we need to use and redirect port 80.

Now in the system menu select packages, and then click the available packages tab. Install Squid3, this is our proxy server to cache websites and help speed things up a little (Very handy when a throttled 2.5Mbps connection, even more annoying when 2 streets away you get 350Mbps fibre). Once Squid is installed the config page will appear in the services menu under proxy server.

The only things we have to change in squid is the listening interface. Make sure only the loopback interface is selected. DO NOT tick the transparent proxy check box! And add the lines

acl all src 0.0.0.0/0.0.0.0
never_direct allow allsrc
cache_peer 127.0.0.1 parent 3125 0 name=havp no-query no-digest no-netdb-exchange default

to the custom options section and save the settings

You can now see if squid is running by clicking on services in the status menu.

Next we setup the virus scanning.

We need to install the 'HAVP antivirus' package. We do this the same as we did for squid. System menu and select packages and click on the available packages tab and install 'HAVP antivirus'. Once installed the config page will be under Antivirus in the services menu.

In the config page for the antivirus click on the http proxy tab and make sure that Enable is ticked, proxy mode is set to 'standard' (DO NOT SELECT "PARENT FOR SQUID", this can cause problems after a reboot.) and proxy interface is set to loopback. Set Parent proxy to 127.0.0.1:8118. Then save the settings. You can again check that HAVP is running by viewing the services page in the status menu.

Now we will set up Dansguardian.

This will block sites deemed as inappropriate such as porn. Again we do this by going to the system menu, select packages and click on the available packages tab. Then install Dansguardian. The Dansguardian config is in the service menu under you guessed it in Dansguardian.

Again not a lot to set up in the config just make sure the 'Enable Dansguardian' checkbox is ticked and the listening interface is loopback. Because Dansguardian runs before squid we set dansguardian to pass its traffic to squid by setting the proxy IP to 127.0.0.1 and the proxy port to 3128 and save the config.

You can check that Dansguardian is running by checking Status > Services.

So far so good. Now for the banner ad blocking. Unfortunately there is no Pfsense package for doing this which makes setting up privoxy a little more complex but well worth it. However Privoxy is a great piece of software for doing this. To install Privoxy you need shell access to the Pfsence box either directly sat at the machine with a keyboard or via SSH.

at the Pfsense command prompt enter

pkg_add -r ftp://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/Latest/privoxy.tbz

This should install privoxy.

While still in the command line run the following commands

mkdir /var/run/privoxy
chown privoxy:privoxy /var/run/privoxy
touch /usr/local/etc/rc.d/privoxy.sh
chmod +x /usr/local/etc/rc.d/privoxy.sh
squid -z

Now back in the Pfsense webconfig head to Edit file in the diagnostics menu and click browse. Select the /usr/local/etc/privoxy/config file. Due to some funny things inside pfsense it causes problems for privoxy reading the log file after a reboot, so I will disable loggings by commenting out the logdir and logfile directives by changing the lines

logdir /var/log/privoxy

to

#logdir /var/log/privoxy

and the line

logfile logfile

to

#logfile logfile

then save the file.

Now we need to edit the /usr/local/etc/rc.d/privoxy.sh file. Click browse again and select /usr/local/etc/rc.d/privoxy.sh.

The file will be empty. Past the following into the file and then save it

#!/bin/sh

rc_start() {
/usr/local/etc/rc.d/privoxy forcestart

}

rc_stop() {
/usr/local/etc/rc.d/privoxy stop
}

case $1 in
    start)
        rc_start
        ;;
    stop)
        rc_stop
        ;;
    restart)
        rc_stop
        rc_start
        ;;
esac

Now back on the command line on your Pfsense box issue the command

sh /usr/local/etc/rc.d/privoxy.sh start

This should start privoxy.

You can check that privoxy is running by running

/usr/local/etc/rc.d/privoxy forcestatus

We are nearly done!

We have set each service to only listen on the loopback service. This is to stop users on your network manually setting proxy settings in there browser and bypassing any of the filtering. However this makes it so no one can access any of the filters. But we we want to set it up transparently which just so happens to fix this problem. To do this go to Firewall > NAT in the web config. Click on the + icon on right to create a new nat rule.

Set interface to LAN
Set Destination to any
Set Destination port range to HTTP
Set redirect target IP to 127.0.0.1
Set redirect target port to 8080
Leave all other settings as they are

and save the rule and click apply changes

To make sure all is fine reboot your pfsense box leave it 5 mins to reboot and try accessing the internet, make sure porn, virus's and ads are still blocked.

You should now have a working Pfsense box filtering out ads, Virus's and porn.

You can check that virus's are blocked by trying to download the test virus from http://www.eicar.org/85-0-Download.html You should get an access is denied message when trying to download one of the test virus's.

Check porn access by heading over to a porn site such as www.redtube.com. Again you should get an access is denied message.

Now check for ad and banner removal by heading over to a site known to have ads on it such as www.slashdot.org. All being well there shouldn't be any ads at the top or sides of the page.

If all is well you now have a very powerful internet filter built around Pfsense.

If you find any errors or improvements please post them below.

[KeltecRFB]

Berrance, is this the correct Parent proxy for HAVP?  Parent proxy to 127.0.0.1:8118?  Shouldn't be 3125, 3128, or 8080?

[berrance]

I believe so. The setup is,

Client > Nat to dansguardian (127.0.0.1:8080) > squid (127.0.0.1:3128) > Havp (127.0.0.1:3125) > Privoxy (127.0.0.1:8118) > internet

With no parent proxy in Havp you will get the internet but not running through privoxy so no ad blocking.

Please drop another line if your having problems.

Berrance

[KeltecRFB]

I was able to get Dansguardian, Squid, and HAVP working before system resources were getting tight so I did not install Privoxy. Although my configuration was a little different.  I will post it later.

EDIT:

I did almost everything up above except Privoxy (for now?) and the following:

1.  I place Squid on Transparent instead of turning it off.
2.  I clicked LAN\Opt1 interfaces on Squid and HAVP.  Squid proxy point at HAVP (I tried loopback but did not work)
3.  I did set HAVP as parent for squid. (I tried standard and transparent but did not work)
4.  I clicked LAN\Opt1\Loopback for Dansguardian - Proxy server point at Squid (tried loopback but did not work)
5.  I NAT'd Opt1 traffic to Opt1 interface IP address and LAN Subnet to the LAN interface IP address instead of the Loopback. (tried 127.0.0.1  but did not work).
6.  Added ignore_expect_100 on to Squid Proxy customizations.
7.  Tested connectivity and "questionable material" was blocked and eicar test was blocked as well.

On a side note, do you know how many resources does Privoxy take up?  I am hovering around 80% with all of the services I have running.

The firewall hardware consists of dual P-III 850 and 1 gig of RAM (everything is maxed).

[berrance]

Glad to hear you've got going. Just a few notes on your points from my experience (may help other people). I could only get it working by doing it in the order from my howto did a reinstall to check but thats not to say I have missed something out.

1. For me placing squid on transparent made the web traffic bypass Dansguardian as Dansguardian was running before squid in the filter chain. Squid transparent mode redirected all http traffic to Squid which is why I added a nat rule for all traffic on lan1 port 80 and redirect it to 127.0.0.1:8080 provided dansguardian is listening on loopback making it transparent

2. When squid is in transparent mode You may need squid listening on Lan.  If you had Havp set as parent for squid and listening on loopback what IP is shown in the Integrations section on the squid config page? You may need to change it from your lan address to 127.0.0.1

3. For me it sorta worked with parent for Squid. But for me and a big But, after installing Dansguardian and restarting squid or rebooting the machine I had to re add the line

acl all src 0.0.0.0/0.0.0.0

to squid.conf before the lines

never_direct allow allsrc
cache_peer 127.0.0.1 parent 3125 0 name=havp no-query no-digest no-netdb-exchange default

that Havp added when set to Parent for Squid and restart squid. Which again didnt survive a restart. To fix this I set Havp to S5.  I NAT'd Opt1 traffic to Opt1 interface IP address and LAN Subnet to the LAN interface IP address tandard and manually add  the above lines in order to the Custom Options section on the proxy conf page making sure the integrations section is empty. To get the integrations section to stay empty after Havp had been set to Parent for squid I had to uninstall and reinstall Havp (Possibly Squid aswell I cant remember). For testing purposes I just set my browsers proxy to squid also for testing I did enable Squid to listen on Lan aswell.

4. see 1.

5. What address/port where you listening on/redirecting to? provided Dansguardian is listening on loopback port 8080 you should be fine redirecting traffic from Lan/Opt1 port 80 to 127.0.0.1 port 8080

7. All should well work well with all services running whilst listening on Lan but should a user know the ip and port numbers of the services running they could bypass vital parts of the chain by changing their browsers proxy settings.

I used to run a similar set up but without Havp on a dual P-III 600Mhz with 750 MBs ram set up with IPCop a few years ago and that used to max out pretty easy. The setup I used in the howto is an old P4 1.7Ghz with 1GB ram. Still running now with that how to.

[marcelloc]

berrance,
Great contribution.  :)

Just some notes:

• Dansguardian already has antivirus integration, you do not need HAVP.  ;)

• Are you sure dansguardian blacklists can't deny ads?

• passing traffic through a lot of daemons, may slow down internet access.

• HTTPS access will not be filtered on this setup

[KeltecRFB]

Marcello,

What is your recommended configuration?

Thanks!

[marcelloc]

@KeltecRFB:

What is your recommended configuratio?

User -> dansguardian -> squid -> internet
                (With clamav)

[KeltecRFB]

Any good documentation on just using Squid and Dansguardian or point me to the best thread because everything I have tried fails.

[marcelloc]

@KeltecRFB:

Any good documentation on just using Squid and Dansguardian or point me to the best thread because everything I have tried fails.

Did you tried this?

Installing the Dansguardian package in PFSense - One user's experience

[KeltecRFB]

@marcelloc:

Did you tried this?

Installing the Dansguardian package in PFSense - One user's experience

Thank you, I figured out what was wrong … :-)

Now to figure out anti-virus.

[marcelloc]

@KeltecRFB:

Now to figure out anti-virus.

just select clamdscan on system -> dansguardian -> general and save config

[KeltecRFB]

@marcelloc:

@KeltecRFB:

Now to figure out anti-virus.

just select clamdscan on system -> dansguardian -> general and save config

I did just that but then Danguardian pops up with a message stating it is blocking the site because it can not scan it, Categories:  Content scan.  What else needs to be done; do I also need to disable a categories filter?

Thanks!

[marcelloc]

Force a frashclam update on console and then restart dansguardian(or save config)

[KeltecRFB]

@marcelloc:

Force a frashclam update on console and then restart dansguardian(or save config)

Did that, same response - website blocked, unable to scan content…

In addition, if I turn on clamdscan, and then turn it off because it is not working properly yet, I have to reboot the box for internet connectivity to be restored.

Should I try to enable it and reboot?

[marcelloc]

Are you on latest dansguardian package version?

[KeltecRFB]

@marcelloc:

Are you on latest dansguardian package version?

2.12.0.3 pkg v.0.1.7

[KeltecRFB]

I see other people have had this problem, but nothing sticks out yet.

[KeltecRFB]

Marcello,

I tried doing this and the download was successful, however, the deny page for Cisco is still displayed when clamdscan is activated.

http://forum.pfsense.org/index.php/topic,52163.0.html

[KeltecRFB]

Problem resolved…

I made a change thinking that was the problem for something else but it wasn't and caused a problem with clamdscan working properly.  Now everything is working.