Ad Blocking via dnsmasq and httpd-proxy_mod_security

stephalosaurus

I know the first prevailing thought - use Squid. Frankly, I simply don't want to, as I don't like, don't trust, and can't "afford" the weight of squid on my home router. (feel free to take that up with me privately, this thread is for an alternate solution!)

That being said, this page is more of a marker for my own future reference on how I went about making pfSense 2.1.2 utilize DNS-based ad blocking with minimal headache. Some remaining work will need to be performed - I'll update this thread accordingly.

The theory is the same as those who use dnsmasq or bind to redirect a list of undesirable servers to a particular IP, except rather than using pixelserv, I'm using httpd. I know httpd would consume more resources, but it can also handle a higher level of concurrency than pixelserv. I set httpd to bind to the lo0 interface (127.0.0.1) on TCP/2010, then essentially redirect all traffic using PF from the DNS redirect (10.254.254.254 in my configuration) to the firewall's lo0 (127.0.0.1) on TCP/2010. Apache then serves a transparent pixel and goes back to idle. CPU usage seems largely unaffected, though Apache does consume significantly more memory than pixelserv. If someone would rather write a pixelserv.php that can handle a relatively heavy connection rate, I'll happily include it as the better option. You could just as easily create a virtualhost and serve an index.html with an img tag, or rewrite all URLs to transparent_pixel.gif, I simply chose to use the ErrorDocument feature to do my dirtywork. Logging is disabled.

I'm using the nanoBSD CF image, so please bear with me if you're not - I'll get to that later.

Install the Apache with mod_security-dev and cron packages via the webConfigurator via System -> Package Manager -> Available Packages.
Via SSH, log into you router (presumably as root, but if you want to use a user + sudo, install the sudo package and go that route), then mount your filesystem read-write.

/etc/rc.conf_mount_rw

Configure dnsmasq:

3a) Create the dnsmasq configuration directory:

mkdir /usr/local/etc/dnsmasq.d

Edit /usr/local/etc/dnsmasq.conf:


domain-needed
bogus-priv
local=/localhost/
domain=localdomain
expand-hosts
cache-size=10000
dns-forward-max=300
resolv-file=/var/etc/resolv.conf
conf-dir=/usr/local/etc/dnsmasq.d

3b) Create the shell script /usr/local/bin/update_adblock_list.sh that will create and udpate the ad blocking list (this will eventually be placed as a cron job). At this point, select an unused IP address from some network you have no intention of using. I chose 10.254.254.254, but feel free to do as you please if you're using the full 10/8 for something else.


#!/bin/sh
REDIR_TO='10.254.254.254'
ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext'
ADBLOCK_CONF='/usr/local/etc/dnsmasq.d/adblock.conf'
TMP_FILE='/tmp/adblock.conf'

echo Fetching AD Blocking list from $ADBLOCK_URL...
/usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" | /usr/bin/sort | /usr/bin/uniq > $TMP_FILE
echo Analyzing for changes...
if ! cmp -s "$TMP_FILE" "$ADBLOCK_CONF"; then
  echo Changes detected!
  echo Mounting filesystem read-write...
  /etc/rc.conf_mount_rw
  echo Updating $ADBLOCK_CONF with latest entries...
  /usr/bin/sort $TMP_FILE $ADBLOCK_CONF | /usr/bin/uniq > $ADBLOCK_CONF
  echo Mounting filesystem read-only...
  /etc/rc.conf_mount_ro
  echo Restarting dnsmasq...
  /usr/local/bin/dnsmasq_restart.php 
  echo Update completed.
else
 echo No updates required.
fi

3c) Create /usr/local/bin/dnsmasq_restart.php, a php-based script used to restart dnsmasq:


#!/usr/local/bin/php -q
require_once("functions.inc");
$retval |= services_dnsmasq_configure();
?>

3d) Make the dnsmasq_restart.php and /usr/local/bin/update_adblock_list.sh scripts executable:

chmod 755 /usr/local/bin/dnsmasq_restart.php /usr/local/bin/update_adblock_list.sh

3e) Execute update_adblock_list.sh, this will create the ad blocking list (adblock.conf) in /usr/local/etc/dnsmasq.d, query the pgl.yoyo.org list, update and restart dnsmasq:

/usr/local/bin/update_adblock_list.sh

Configure httpd:

4a) Copy the transparent_pixel.gif file from the pfsense_ng theme directory, then remove index.html:


cp /usr/local/www/themes/pfsense_ng/images/transparent_pixel.gif /usr/pbi/proxy_mod_security-i386/www/apache24/data/
rm /usr/pbi/proxy_mod_security-i386/www/apache24/data/index.html

4b) Create a new httpd.conf file with minimal modules, etc.:


cp /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf.bak
vi /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf


ServerRoot "/usr/pbi/proxy_mod_security-i386"
Listen 127.0.0.1:2010
LoadModule authz_core_module libexec/apache24/mod_authz_core.so
LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so
LoadModule unixd_module libexec/apache24/mod_unixd.so

 <ifmodule unixd_module="">User www
Group www</ifmodule> 

 <directory>AllowOverride none
    Require all denied</directory> 

DocumentRoot "/usr/pbi/proxy_mod_security-i386/www/apache24/data"
 <directory "="" usr="" pbi="" proxy_mod_security-i386="" www="" apache24="" data"="">AllowOverride None
    Require all granted</directory> 

ErrorLog "/var/log/httpd-error.log"
LogLevel emerg

ErrorDocument 402 "/transparent_pixel.gif"
ErrorDocument 403 "/transparent_pixel.gif"
ErrorDocument 404 "/transparent_pixel.gif"
ErrorDocument 500 "/transparent_pixel.gif"

alternately, edit /usr/local/pkg/apache.template for the ErrorLog, LogLevel and ErrorDocument functions above (comment out access logs if desired) and bind to 127.0.0.1/2010 via web interface (Services -> Mod_Security+Apache+Proxy). This will persist the modifications across changes made via the web interface.

4c) Start httpd:


httpd

Configure webConfigurator:

5a) Go to Firewall -> NAT, then add a new NAT rule with the following options:


Interface: LAN
Protocol: TCP
Destination: Single host or alias
Destination address: 10.254.254.254 (same as what was designated above!)
Destination Port Range: 80 (http)
Redirect target IP: 127.0.0.1
Redirect target Port: 2010

5b) Click Save, then click Apply to reload the ruleset. At this point, your ad filtration should now be operational.

Cleanup!

6a) To add a cron job that automatically updates the ad blocking list for dnsmasq, go to Services -> cron and add as follows:


0 4 * * 1 root /usr/bin/nice -n20 /usr/local/bin/update_adblock_list.sh

This will call the update_adblock_list.sh script at 04h00 every Monday morning.

6b) Mount filesystem read-only:


/etc/rc.conf_mount_ro

Zflash76

This is genius. Gonna try this for sure!

stephalosaurus

Beware the httpd package, it looks as if the apache.template file is grotesquely broken/misconfigured for 2.4. I'm working on a fixed version, but the developers should be aware that the web interface, when re-writing the configuration, WILL break the package.

In short, it's pretty much awful - for now, stay with a manual configuration and do NOT touch the config via the webConfigurator.

stephalosaurus

I guess this forum doesn't allow edits after some undefined period of time, or not that I can find - brilliant, absolutely brilliant. ::)

I made a few changes to the adblock update script, got rid of the ugly sort/uniq operators, lots of local logging junk (now logs via syslog to /tmp/resolver.log properly) and no longer requires the assistance of a php script to restart dnsmasq.


#!/usr/local/bin/bash
REDIR_TO='10.254.254.254'
ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext'
CONF_DNSMASQ='/usr/local/etc/dnsmasq.conf'
CONF_DNSMASQ_DIR='/usr/local/etc/dnsmasq.d'
CONF_ADBLOCK='/usr/local/etc/dnsmasq.d/adblock.conf'
CONF_ADBLOCK_TEMP='/tmp/adblock.conf'
CONF_ADBLOCK_BACKUP='/tmp/adblock.conf.orig'
LOCAL_DEBUG=false

daemonlog () {
  logger -p daemon.info -i -t dnsmasq $1
  $LOCAL_DEBUG && echo $1
}

restart_dnsmasq () {
  echo '' | php -q
}

if [ ! -d $CONF_DNSMASQ_DIR ]; then
  daemonlog "Initializing ad blocking configuration, mounting filesystem read-write"
  /etc/rc.conf_mount_rw
  mkdir -p $CONF_DNSMASQ_DIR

  if [ ! -r $CONF_DNSMASQ ]; then
    daemonlog "Creating dnsmasq configuration"
    echo "conf-dir=$CONF_DNSMASQ_DIR" > $CONF_DNSMASQ
  else
    daemonlog "dnsmasq configuration exists, adding configuration directory"
    echo "conf-dir=$CONF_DNSMASQ_DIR" >> $CONF_DNSMASQ
  fi

  daemonlog "Initializing ad blocking repository, mounting filesystem read-only"
  touch $CONF_ADBLOCK
  /etc/rc.conf_mount_ro
fi

daemonlog "Fetching ad blocking list from $ADBLOCK_URL"
/usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" > $CONF_ADBLOCK_TEMP
daemonlog "Analyzing for changes"
if ! /usr/bin/cmp -s "$CONF_ADBLOCK_TEMP" "$CONF_ADBLOCK"; then
  daemonlog "Changes detected, mounting filesystem read-write"
  /etc/rc.conf_mount_rw
  daemonlog "Updating $CONF_ADBLOCK with latest entries"
  cp $CONF_ADBLOCK $CONF_ADBLOCK_BACKUP
  cp $CONF_ADBLOCK_TEMP $CONF_ADBLOCK
  daemonlog "Restarting dnsmasq"
  restart_dnsmasq
  if ! pgrep -q dnsmasq; then
    daemonlog "dnsmasq failed to restart, reverting to previous ad blocking configuration"
    cp $CONF_ADBLOCK_BACKUP $CONF_ADBLOCK
    restart_dnsmasq
  fi
  daemonlog "Update completed. Re-mounting filesystem read-only"
  /etc/rc.conf_mount_ro
else
  daemonlog "No ad blocking updates required"
fi