Ad Blocking via dnsmasq and httpd-proxy_mod_security
-
I know the first prevailing thought - use Squid. Frankly, I simply don't want to, as I don't like, don't trust, and can't "afford" the weight of squid on my home router. (feel free to take that up with me privately, this thread is for an alternate solution!)
That being said, this page is more of a marker for my own future reference on how I went about making pfSense 2.1.2 utilize DNS-based ad blocking with minimal headache. Some remaining work will need to be performed - I'll update this thread accordingly.
The theory is the same as those who use dnsmasq or bind to redirect a list of undesirable servers to a particular IP, except rather than using pixelserv, I'm using httpd. I know httpd would consume more resources, but it can also handle a higher level of concurrency than pixelserv. I set httpd to bind to the lo0 interface (127.0.0.1) on TCP/2010, then essentially redirect all traffic using PF from the DNS redirect (10.254.254.254 in my configuration) to the firewall's lo0 (127.0.0.1) on TCP/2010. Apache then serves a transparent pixel and goes back to idle. CPU usage seems largely unaffected, though Apache does consume significantly more memory than pixelserv. If someone would rather write a pixelserv.php that can handle a relatively heavy connection rate, I'll happily include it as the better option. You could just as easily create a virtualhost and serve an index.html with an img tag, or rewrite all URLs to transparent_pixel.gif, I simply chose to use the ErrorDocument feature to do my dirtywork. Logging is disabled.
I'm using the nanoBSD CF image, so please bear with me if you're not - I'll get to that later.
-
Install the Apache with mod_security-dev and cron packages via the webConfigurator via System -> Package Manager -> Available Packages.
-
Via SSH, log into you router (presumably as root, but if you want to use a user + sudo, install the sudo package and go that route), then mount your filesystem read-write.
/etc/rc.conf_mount_rw
- Configure dnsmasq:
3a) Create the dnsmasq configuration directory:
mkdir /usr/local/etc/dnsmasq.d
- Edit /usr/local/etc/dnsmasq.conf:
domain-needed bogus-priv local=/localhost/ domain=localdomain expand-hosts cache-size=10000 dns-forward-max=300 resolv-file=/var/etc/resolv.conf conf-dir=/usr/local/etc/dnsmasq.d
3b) Create the shell script /usr/local/bin/update_adblock_list.sh that will create and udpate the ad blocking list (this will eventually be placed as a cron job). At this point, select an unused IP address from some network you have no intention of using. I chose 10.254.254.254, but feel free to do as you please if you're using the full 10/8 for something else.
#!/bin/sh REDIR_TO='10.254.254.254' ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext' ADBLOCK_CONF='/usr/local/etc/dnsmasq.d/adblock.conf' TMP_FILE='/tmp/adblock.conf' echo Fetching AD Blocking list from $ADBLOCK_URL... /usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" | /usr/bin/sort | /usr/bin/uniq > $TMP_FILE echo Analyzing for changes... if ! cmp -s "$TMP_FILE" "$ADBLOCK_CONF"; then echo Changes detected! echo Mounting filesystem read-write... /etc/rc.conf_mount_rw echo Updating $ADBLOCK_CONF with latest entries... /usr/bin/sort $TMP_FILE $ADBLOCK_CONF | /usr/bin/uniq > $ADBLOCK_CONF echo Mounting filesystem read-only... /etc/rc.conf_mount_ro echo Restarting dnsmasq... /usr/local/bin/dnsmasq_restart.php echo Update completed. else echo No updates required. fi
3c) Create /usr/local/bin/dnsmasq_restart.php, a php-based script used to restart dnsmasq:
#!/usr/local/bin/php -q require_once("functions.inc"); $retval |= services_dnsmasq_configure(); ?>
3d) Make the dnsmasq_restart.php and /usr/local/bin/update_adblock_list.sh scripts executable:
chmod 755 /usr/local/bin/dnsmasq_restart.php /usr/local/bin/update_adblock_list.sh
3e) Execute update_adblock_list.sh, this will create the ad blocking list (adblock.conf) in /usr/local/etc/dnsmasq.d, query the pgl.yoyo.org list, update and restart dnsmasq:
/usr/local/bin/update_adblock_list.sh
- Configure httpd:
4a) Copy the transparent_pixel.gif file from the pfsense_ng theme directory, then remove index.html:
cp /usr/local/www/themes/pfsense_ng/images/transparent_pixel.gif /usr/pbi/proxy_mod_security-i386/www/apache24/data/ rm /usr/pbi/proxy_mod_security-i386/www/apache24/data/index.html
4b) Create a new httpd.conf file with minimal modules, etc.:
cp /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf.bak vi /usr/pbi/proxy_mod_security-i386/etc/apache24/httpd.conf
ServerRoot "/usr/pbi/proxy_mod_security-i386" Listen 127.0.0.1:2010 LoadModule authz_core_module libexec/apache24/mod_authz_core.so LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so LoadModule unixd_module libexec/apache24/mod_unixd.so <ifmodule unixd_module="">User www Group www</ifmodule> <directory>AllowOverride none Require all denied</directory> DocumentRoot "/usr/pbi/proxy_mod_security-i386/www/apache24/data" <directory "="" usr="" pbi="" proxy_mod_security-i386="" www="" apache24="" data"="">AllowOverride None Require all granted</directory> ErrorLog "/var/log/httpd-error.log" LogLevel emerg ErrorDocument 402 "/transparent_pixel.gif" ErrorDocument 403 "/transparent_pixel.gif" ErrorDocument 404 "/transparent_pixel.gif" ErrorDocument 500 "/transparent_pixel.gif"
- alternately, edit /usr/local/pkg/apache.template for the ErrorLog, LogLevel and ErrorDocument functions above (comment out access logs if desired) and bind to 127.0.0.1/2010 via web interface (Services -> Mod_Security+Apache+Proxy). This will persist the modifications across changes made via the web interface.
4c) Start httpd:
httpd
- Configure webConfigurator:
5a) Go to Firewall -> NAT, then add a new NAT rule with the following options:
Interface: LAN Protocol: TCP Destination: Single host or alias Destination address: 10.254.254.254 (same as what was designated above!) Destination Port Range: 80 (http) Redirect target IP: 127.0.0.1 Redirect target Port: 2010
5b) Click Save, then click Apply to reload the ruleset. At this point, your ad filtration should now be operational.
- Cleanup!
6a) To add a cron job that automatically updates the ad blocking list for dnsmasq, go to Services -> cron and add as follows:
0 4 * * 1 root /usr/bin/nice -n20 /usr/local/bin/update_adblock_list.sh
This will call the update_adblock_list.sh script at 04h00 every Monday morning.
6b) Mount filesystem read-only:
/etc/rc.conf_mount_ro
-
-
This is genius. Gonna try this for sure!
-
Beware the httpd package, it looks as if the apache.template file is grotesquely broken/misconfigured for 2.4. I'm working on a fixed version, but the developers should be aware that the web interface, when re-writing the configuration, WILL break the package.
In short, it's pretty much awful - for now, stay with a manual configuration and do NOT touch the config via the webConfigurator.
-
I guess this forum doesn't allow edits after some undefined period of time, or not that I can find - brilliant, absolutely brilliant. ::)
I made a few changes to the adblock update script, got rid of the ugly sort/uniq operators, lots of local logging junk (now logs via syslog to /tmp/resolver.log properly) and no longer requires the assistance of a php script to restart dnsmasq.
#!/usr/local/bin/bash REDIR_TO='10.254.254.254' ADBLOCK_URL='http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext' CONF_DNSMASQ='/usr/local/etc/dnsmasq.conf' CONF_DNSMASQ_DIR='/usr/local/etc/dnsmasq.d' CONF_ADBLOCK='/usr/local/etc/dnsmasq.d/adblock.conf' CONF_ADBLOCK_TEMP='/tmp/adblock.conf' CONF_ADBLOCK_BACKUP='/tmp/adblock.conf.orig' LOCAL_DEBUG=false daemonlog () { logger -p daemon.info -i -t dnsmasq $1 $LOCAL_DEBUG && echo $1 } restart_dnsmasq () { echo '' | php -q } if [ ! -d $CONF_DNSMASQ_DIR ]; then daemonlog "Initializing ad blocking configuration, mounting filesystem read-write" /etc/rc.conf_mount_rw mkdir -p $CONF_DNSMASQ_DIR if [ ! -r $CONF_DNSMASQ ]; then daemonlog "Creating dnsmasq configuration" echo "conf-dir=$CONF_DNSMASQ_DIR" > $CONF_DNSMASQ else daemonlog "dnsmasq configuration exists, adding configuration directory" echo "conf-dir=$CONF_DNSMASQ_DIR" >> $CONF_DNSMASQ fi daemonlog "Initializing ad blocking repository, mounting filesystem read-only" touch $CONF_ADBLOCK /etc/rc.conf_mount_ro fi daemonlog "Fetching ad blocking list from $ADBLOCK_URL" /usr/bin/fetch -qo - $ADBLOCK_URL | /usr/bin/sed "s/127\.0\.0\.1/$REDIR_TO/" > $CONF_ADBLOCK_TEMP daemonlog "Analyzing for changes" if ! /usr/bin/cmp -s "$CONF_ADBLOCK_TEMP" "$CONF_ADBLOCK"; then daemonlog "Changes detected, mounting filesystem read-write" /etc/rc.conf_mount_rw daemonlog "Updating $CONF_ADBLOCK with latest entries" cp $CONF_ADBLOCK $CONF_ADBLOCK_BACKUP cp $CONF_ADBLOCK_TEMP $CONF_ADBLOCK daemonlog "Restarting dnsmasq" restart_dnsmasq if ! pgrep -q dnsmasq; then daemonlog "dnsmasq failed to restart, reverting to previous ad blocking configuration" cp $CONF_ADBLOCK_BACKUP $CONF_ADBLOCK restart_dnsmasq fi daemonlog "Update completed. Re-mounting filesystem read-only" /etc/rc.conf_mount_ro else daemonlog "No ad blocking updates required" fi