Ad Blocking via dnsmasq and httpd-proxy_mod_security



  • I know the first prevailing thought - use Squid.  Frankly, I simply don't want to, as I don't like, don't trust, and can't "afford" the weight of squid on my home router. (feel free to take that up with me privately, this thread is for an alternate solution!)

    That being said, this page is more of a marker for my own future reference on how I went about making pfSense 2.1.2 utilize DNS-based ad blocking with minimal headache.  Some remaining work will need to be performed - I'll update this thread accordingly.

    The theory is the same as those who use dnsmasq or bind to redirect a list of undesirable servers to a particular IP, except rather than using pixelserv, I'm using httpd.  I know httpd would consume more resources, but it can also handle a higher level of concurrency than pixelserv.  I set httpd to bind to the lo0 interface (127.0.0.1) on TCP/2010, then essentially redirect all traffic using PF from the DNS redirect (10.254.254.254 in my configuration) to the firewall's lo0 (127.0.0.1) on TCP/2010.  Apache then serves a transparent pixel and goes back to idle.  CPU usage seems largely unaffected, though Apache does consume significantly more memory than pixelserv.  If someone would rather write a pixelserv.php that can handle a relatively heavy connection rate, I'll happily include it as the better option.  You could just as easily create a virtualhost and serve an index.html with an img tag, or rewrite all URLs to transparent_pixel.gif, I simply chose to use the ErrorDocument feature to do my dirtywork.  Logging is disabled.

    I'm using the nanoBSD CF image, so please bear with me if you're not - I'll get to that later.

    1. Install the Apache with mod_security-dev and cron packages via the webConfigurator via System -> Package Manager -> Available Packages.

    2. Via SSH, log into you router (presumably as root, but if you want to use a user + sudo, install the sudo package and go that route), then mount your filesystem read-write.

    /etc/rc.conf_mount_rw
    
    1. Configure dnsmasq:

    3a) Create the dnsmasq configuration directory:

    mkdir /usr/local/etc/dnsmasq.d
    
    • Edit /usr/local/etc/dnsmasq.conf:
    
    domain-needed
    bogus-priv
    local=/localhost/
    domain=localdomain
    expand-hosts
    cache-size=10000
    dns-forward-max=300
    resolv-file=/var/etc/resolv.conf
    conf-dir=/usr/local/etc/dnsmasq.d
    
    

    3b) Create the shell script /usr/local/bin/update_adblock_list.sh that will create and udpate the ad blocking list (this will eventually be placed as a cron job).  At this point, select an unused IP address from some network you have no intention of using.  I chose 10.254.254.254, but feel free to do as you please if you're using the full 10/8 for something else.

    
    #!/bin/sh
    REDIR_TO='10.254.254.254'
    ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext'
    ADBLOCK_CONF='/usr/local/etc/dnsmasq.d/adblock.conf'
    TMP_FILE='/tmp/adblock.conf'
    
    echo Fetching AD Blocking list from $ADBLOCK_URL...
    /usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" | /usr/bin/sort | /usr/bin/uniq > $TMP_FILE
    echo Analyzing for changes...
    if ! cmp -s "$TMP_FILE" "$ADBLOCK_CONF"; then
      echo Changes detected!
      echo Mounting filesystem read-write...
      /etc/rc.conf_mount_rw
      echo Updating $ADBLOCK_CONF with latest entries...
      /usr/bin/sort $TMP_FILE $ADBLOCK_CONF | /usr/bin/uniq > $ADBLOCK_CONF
      echo Mounting filesystem read-only...
      /etc/rc.conf_mount_ro
      echo Restarting dnsmasq...
      /usr/local/bin/dnsmasq_restart.php 
      echo Update completed.
    else
     echo No updates required.
    fi
    
    

    3c) Create /usr/local/bin/dnsmasq_restart.php, a php-based script used to restart dnsmasq:

    
    #!/usr/local/bin/php -q
    require_once("functions.inc");
    $retval |= services_dnsmasq_configure();
    ?>
    
    

    3d) Make the dnsmasq_restart.php and /usr/local/bin/update_adblock_list.sh scripts executable:

    chmod 755 /usr/local/bin/dnsmasq_restart.php /usr/local/bin/update_adblock_list.sh
    

    3e) Execute update_adblock_list.sh, this will create the ad blocking list (adblock.conf) in /usr/local/etc/dnsmasq.d, query the pgl.yoyo.org list, update and restart dnsmasq:

    /usr/local/bin/update_adblock_list.sh
    
    1. Configure httpd:

    4a) Copy the transparent_pixel.gif file from the pfsense_ng theme directory, then remove index.html:

    
    cp /usr/local/www/themes/pfsense_ng/images/transparent_pixel.gif /usr/pbi/proxy_mod_security-i386/www/apache24/data/
    rm /usr/pbi/proxy_mod_security-i386/www/apache24/data/index.html
    
    

    4b) Create a new httpd.conf file with minimal modules, etc.:

    
    cp /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf.bak
    vi /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf
    
    
    
    ServerRoot "/usr/pbi/proxy_mod_security-i386"
    Listen 127.0.0.1:2010
    LoadModule authz_core_module libexec/apache24/mod_authz_core.so
    LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so
    LoadModule unixd_module libexec/apache24/mod_unixd.so
    
     <ifmodule unixd_module="">User www
    Group www</ifmodule> 
    
     <directory>AllowOverride none
        Require all denied</directory> 
    
    DocumentRoot "/usr/pbi/proxy_mod_security-i386/www/apache24/data"
     <directory "="" usr="" pbi="" proxy_mod_security-i386="" www="" apache24="" data"="">AllowOverride None
        Require all granted</directory> 
    
    ErrorLog "/var/log/httpd-error.log"
    LogLevel emerg
    
    ErrorDocument 402 "/transparent_pixel.gif"
    ErrorDocument 403 "/transparent_pixel.gif"
    ErrorDocument 404 "/transparent_pixel.gif"
    ErrorDocument 500 "/transparent_pixel.gif"
    
    
    • alternately, edit /usr/local/pkg/apache.template for the ErrorLog, LogLevel and ErrorDocument functions above (comment out access logs if desired) and bind to 127.0.0.1/2010 via web interface (Services -> Mod_Security+Apache+Proxy). This will persist the modifications across changes made via the web interface.

    4c) Start httpd:

    
    httpd
    
    
    1. Configure webConfigurator:

    5a) Go to Firewall -> NAT, then add a new NAT rule with the following options:

    
    Interface: LAN
    Protocol: TCP
    Destination: Single host or alias
    Destination address: 10.254.254.254 (same as what was designated above!)
    Destination Port Range: 80 (http)
    Redirect target IP: 127.0.0.1
    Redirect target Port: 2010
    
    

    5b) Click Save, then click Apply to reload the ruleset.  At this point, your ad filtration should now be operational.

    1. Cleanup!

    6a) To add a cron job that automatically updates the ad blocking list for dnsmasq, go to Services -> cron and add as follows:

    
    0 4 * * 1 root /usr/bin/nice -n20 /usr/local/bin/update_adblock_list.sh
    
    

    This will call the update_adblock_list.sh script at 04h00 every Monday morning.

    6b) Mount filesystem read-only:

    
    /etc/rc.conf_mount_ro
    
    


  • This is genius. Gonna try this for sure!



  • Beware the httpd package, it looks as if the apache.template file is grotesquely broken/misconfigured for 2.4.  I'm working on a fixed version, but the developers should be aware that the web interface, when re-writing the configuration, WILL break the package.

    In short, it's pretty much awful - for now, stay with a manual configuration and do NOT touch the config via the webConfigurator.



  • I guess this forum doesn't allow edits after some undefined period of time, or not that I can find - brilliant, absolutely brilliant.  ::)

    I made a few changes to the adblock update script, got rid of the ugly sort/uniq operators, lots of local logging junk (now logs via syslog to /tmp/resolver.log properly) and no longer requires the assistance of a php script to restart dnsmasq.

    
    #!/usr/local/bin/bash
    REDIR_TO='10.254.254.254'
    ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext'
    CONF_DNSMASQ='/usr/local/etc/dnsmasq.conf'
    CONF_DNSMASQ_DIR='/usr/local/etc/dnsmasq.d'
    CONF_ADBLOCK='/usr/local/etc/dnsmasq.d/adblock.conf'
    CONF_ADBLOCK_TEMP='/tmp/adblock.conf'
    CONF_ADBLOCK_BACKUP='/tmp/adblock.conf.orig'
    LOCAL_DEBUG=false
    
    daemonlog () {
      logger -p daemon.info -i -t dnsmasq $1
      $LOCAL_DEBUG && echo $1
    }
    
    restart_dnsmasq () {
      echo '' | php -q
    }
    
    if [ ! -d $CONF_DNSMASQ_DIR ]; then
      daemonlog "Initializing ad blocking configuration, mounting filesystem read-write"
      /etc/rc.conf_mount_rw
      mkdir -p $CONF_DNSMASQ_DIR
    
      if [ ! -r $CONF_DNSMASQ ]; then
        daemonlog "Creating dnsmasq configuration"
        echo "conf-dir=$CONF_DNSMASQ_DIR" > $CONF_DNSMASQ
      else
        daemonlog "dnsmasq configuration exists, adding configuration directory"
        echo "conf-dir=$CONF_DNSMASQ_DIR" >> $CONF_DNSMASQ
      fi
    
      daemonlog "Initializing ad blocking repository, mounting filesystem read-only"
      touch $CONF_ADBLOCK
      /etc/rc.conf_mount_ro
    fi
    
    daemonlog "Fetching ad blocking list from $ADBLOCK_URL"
    /usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" > $CONF_ADBLOCK_TEMP
    daemonlog "Analyzing for changes"
    if ! /usr/bin/cmp -s "$CONF_ADBLOCK_TEMP" "$CONF_ADBLOCK"; then
      daemonlog "Changes detected, mounting filesystem read-write"
      /etc/rc.conf_mount_rw
      daemonlog "Updating $CONF_ADBLOCK with latest entries"
      cp $CONF_ADBLOCK $CONF_ADBLOCK_BACKUP
      cp $CONF_ADBLOCK_TEMP $CONF_ADBLOCK
      daemonlog "Restarting dnsmasq"
      restart_dnsmasq
      if ! pgrep -q dnsmasq; then
        daemonlog "dnsmasq failed to restart, reverting to previous ad blocking configuration"
        cp $CONF_ADBLOCK_BACKUP $CONF_ADBLOCK
        restart_dnsmasq
      fi
      daemonlog "Update completed. Re-mounting filesystem read-only"
      /etc/rc.conf_mount_ro
    else
      daemonlog "No ad blocking updates required"
    fi