DNS cache and pfSense on same box?



  • I'm running an experimental Web crawler that makes a very large number of DNS requests, and have been asked if I could cache DNS info in order to reduce some of the load on our primary DNS server.  Rather than set up another box to serve as our local DNS cache, I was wondering if it's possible to do that on the same box that's running pfSense.

    Yes, I know that pfSense has a built-in DNS cache, but it's incredibly small.  I make thousands of DNS requests per minute, and want to do some custom caching of the results.  If I can't do it on the pfSense box, I'll set up a machine with BIND or some other DNS software.  But … if I can combine the two machines, all the better.

    I'm currently running pfSense (1.2-RC2) on a 2.4 GHz Core 2 Duo with 4 gigabytes of RAM.  Solid as a rock.  But the machine is horribly under-used.  How difficult would it be to set up a huge DNS cache on this machine?  Understand, I don't need this thing to act as an authority or anything--just forward requests to the primary DNS server and then cache the results for a while.

    What DNS software would be best?  How do I install it beside pfSense and have both come up?  Would pfSense get confused if I tell it that its primary DNS server is localhost?

    I'm kind of a newb when it comes to fiddling with BSD, so as much detail as possible would be helpful.  Thanks in advance.

    Jim



  • I suppose the first question would be "why aren't you running a cacheing nameserver on the box where you're running the web crawler?"  That would be the logical place for it to live…

    -Rob



  • With a distributed crawler architecture, you still want an upstream cache.



  • Did you try the DNS-server package?  It installs TinyDNS.  You can find instructions to adjust the cache size on this page:

    http://cr.yp.to/djbdns/cachesize.html



  • Thanks for the pointer to dns-server.  Perhaps that will do the job.


Locked