Setting up pfsense and squid in my home, and need some specific advice.
Ok so I have ran several pfsense boxes over the years, I love this software. However all of my previous setups were mostly for the the firewall. I had a PIII running at home, and a still have a P4 in a 1U Dell server running at work that performs great. The PIII had 512MB ram, and the P4 has 1GB if I recall.
However this time around I want to do something different. I am wanting to run a squid server too, but I am wanting to accomplish some very specific tasks with it. I also have some hardware related questions too.
99% of the time it is just My wife and I, and on the weekends we may have some friends over for a LAN party. I do run a Synology NAS that we both access very frequently (multiple time a day) over the internet. We do ALLOT of video streaming, and playing of Final Fantasy XIV (online MMO).
1. I want to cache the updates for Final Fantasy XIV (they can range from a few MB to a few GB)
2. I want to cache the updates for Windows
3. I would like to cache the most commonly accessed data from my server to reduce the load on the server it self. (the server is generally accessed via HTTPS and the cache would mostly consist of images and videos)
4. Any thing else that can be cached is a very welcome bonus. (Youtube, netflix, and Hulu, would be a very nice bonus…but I doubt it's possible)
I would like the Final Fantasy and Windows updates to be cached permanently if possible. This way if Windows needs to be reinstalled I don't have to go through hours of updates, and the same goes for FF, on a reinstall of that game it currently takes about 3 hours -on a good day- to re download all the updates.
The system I am looking at for this would be a Core 2 Duo 3.0 GHz, 8GB DDR2, and a 1TB SATA HDD. I would like to keep pfsense, and Squid on two separate HDD's if possible.
So ultimately, Would that system handle my needs? and 2 How can I go about accomplishing what I am after with the squid server?
I think you will find it very hard to reliably cache a lot of those things. Additionally in order to keep the cached file forever you will need to set an infinite expiry time which might have interesting results on other things.
I could be wrong though I've tried that personally.
Using a separate drive for the cache is not supported within the package or within pfSense so you will have to 'hack' it in. That means that if you ever update the firmware you'll have to do it again.
You haven't mentioned the most important thing when deciding on hardware and that's what throughput you need. What is your WAN bandwidth?
I didn't consider WAN bandwidth, I suppose that would be important. I pay for 130Mb/s Down and 10Mb/s Up, however my speeds are generally closer to 140Mb/s Down, and 11-12Mb/s Up.
Also keeping squid on a separate drive is not make or break for me, so if pfsense doesn't support doing that, and it will just cause more headache down the road, then I wont bother with that part.
Ok, that that hardware will easily manage that. It would probably pass >1Gbps.
Running from one drive will significantly reduce complexity so do that. ;)
There are options in squid 3 for caching dynamic content including a preset for Windows updates but you would have toi write your own pattern for Final Fantasy XIV (or find one). It's hard to do though reliably. Do not expect fantastic results.
Thanks for the advice, I will hopefully be setting this server up soon. I'm glad to know that the hardware i have can handle it, I always like to go bigger than I need (within reason).
Is there any advice on how I would go about creating these custom rules for squid? I mean if I wanted to take a stab at caching these FFXIV updates, where would I even begin? They are downloaded straight through the games launcher, so I have no idea where they come from, what format they are in, what protocol is used to download them, etc…
I have never tried it personally, I've just read account of others having a hard time. ;)
There are quite a few Squid help docs to look at for examples.