DNS cache and pfSense on same box?
-
I'm running an experimental Web crawler that makes a very large number of DNS requests, and have been asked if I could cache DNS info in order to reduce some of the load on our primary DNS server. Rather than set up another box to serve as our local DNS cache, I was wondering if it's possible to do that on the same box that's running pfSense.
Yes, I know that pfSense has a built-in DNS cache, but it's incredibly small. I make thousands of DNS requests per minute, and want to do some custom caching of the results. If I can't do it on the pfSense box, I'll set up a machine with BIND or some other DNS software. But … if I can combine the two machines, all the better.
I'm currently running pfSense (1.2-RC2) on a 2.4 GHz Core 2 Duo with 4 gigabytes of RAM. Solid as a rock. But the machine is horribly under-used. How difficult would it be to set up a huge DNS cache on this machine? Understand, I don't need this thing to act as an authority or anything--just forward requests to the primary DNS server and then cache the results for a while.
What DNS software would be best? How do I install it beside pfSense and have both come up? Would pfSense get confused if I tell it that its primary DNS server is localhost?
I'm kind of a newb when it comes to fiddling with BSD, so as much detail as possible would be helpful. Thanks in advance.
Jim
-
I suppose the first question would be "why aren't you running a cacheing nameserver on the box where you're running the web crawler?" That would be the logical place for it to live…
-Rob
-
With a distributed crawler architecture, you still want an upstream cache.
-
Did you try the DNS-server package? It installs TinyDNS. You can find instructions to adjust the cache size on this page:
-
Thanks for the pointer to dns-server. Perhaps that will do the job.