Pfsense 2.0 [embedded] + squid + usb stick



  • So, I set this up the other day, and it wasn't entirely straight forward.  I searched around for help, but didn't really find everything in one place, so here's a step by step reproduction in case anybody else is interested…

    In case you're wondering why you would want to use a USB stick for this instead of a disk...

    • Unless you have a crappy USB stick, flash based storage has super low latency.  Way lower than a spinning hunk of metal.

    • USB bus bandwidth is (most likely) way higher that your Internet bandwidth, so the absolute transfer speed from flash drive doesn't matter so much.

    • You're squid cache doesn't need to be very big to be useful.

    • USB sticks are dirt cheap… You probably have them lying around your house, in which case they're free. ;)

    • I did this on an embedded install, so I didn't have a writable disk to start with…

    First off, you need to get your USB stick formatted with a decent filesystem. So, insert the USB stick and login to your box via a terminal.  Your stick should be recognized and appear as /dev/da0, though I suppose that can vary.  'dmesg' would show you for sure…  If it's a normal dos-esque stick, it should have one partition, da0s1.  Assuming it does, you can create the UFS filesystem with  'newfs -U /dev/da0s1'.  The -U option to newfs enable soft-updates on the filesystem.

    Next, we need to get the box to mount this stick every time it boots, otherwise Mr. Squid is going to be very unhappy when the box reboots (cuz' of a power outage, not stability, duh).  To do this, I installed the Shellcmd package.  This package allows you to run commands everytime the system boots.  You could edit the scripts on the box, but this allows you to back up your entire config via the backup/restore option, so it's a winner.

    Anyway, go to the Services --> Shellcmd page and add the following commands as type = shellcmd in this order:

    • mount | grep /tmp/mnt/cf > /dev/null; if [ "$?" -eq "0" ]; then umount /tmp/mnt/cf; fi

    • if ! [ -d /var/squid ]; then mkdir /var/squid; fi

    • mount -o noatime /dev/da0s1 /var/squid

    A couple of notes here…

    • When my box booted, it always had the usb stick (incorrectly) mounted as a msdos filesystem at /tmp/mnt/cf.  That first command unmounts that mount if present.

    • I'm (obviously) using /var/squid as my squid install…  If you don't like that, move it.

    • That 'noatime' mount option is always a good idea for mounts where performance matters.  The man page for mount AND squid say so.

    At this point, your USB stick should get mounted every time the box boots.  Now, you can install the squid package.  Yay.
    This should be pretty straight forward as the squid package comes with sane defaults, however I did make these changes:

    • Move the Log Store Directory to /var/squid/logs.  I've got gigs of space for squid logs now ;D

    • Turn on hard disk caching and set the hard disk cache system type to 'aufs'.  Diskd was unstable for me, but YMMV

    • I also increased the Maximum object size value to 4096 (which is the squid default).  Since my cache is now flash backed, I figure I can store bigger objects that the package default of 4k.

    Lastly, I added one rule to my firewall.  I like to prioritize my web traffic above bittorrent and other background, bandwidth hogs, so I added the following floating rule:

    • Action: Pass

    • Quick: Hells yeah

    • Interface: WAN

    • Direction: Out

    • Protocol: TCP

    • Source: WAN address

    • Desination: ANY, port 80 [http]

    • Ackqueue/Queue: [my ack queue]/[my better than bittorrent but worse that games queue]

    And that's it.  Hopefully, you can now go use the interwebs.  If you want to actually see if your cache is doing something useful, there are lots of threads that tell you how to enable cachemgr.cgi (interface to squid statistics via a web browser) and/or SNMP.


Log in to reply