Atom D510 4GB for Squid/SquidGuard? 500 Users
-
You run a significant risk of killing the CF card prematurely if you install anything other than the embedded version, which has no vga output.
I got it: only embedded version to CF card. Thanks clarknova.
-
I got it: only embedded version to CF card. Thanks clarknova.
You could technically run the full install on a high quality CF card (SLC flash with wear-levelling controller) and plausibly extend the lifespan further by disabling logging functions internally for reduced amount of data being written.
However, seeing as that this would be in a corporate environment of sorts, I'm not sure if this is feasible. Not to mention, a full install writes much more often to the CF card and the chances of corruption upon power loss would be greater.
Having been through 3 harddrive failures in my boxes over the past 2 years or so, I'm slowly migrating them to embedded on solid state media. Not to mention, power consumption is greatly reduced and thermals are improved on the tiny boxes. -
I use full installs on 8GB SLC CF cards and have not had one fail yet. These are what I use.
http://www.transcendusa.com/products/ModDetail.asp?ModNo=203&SpNo=1&LangNo=0
-
You could technically run the full install on a high quality CF card (SLC flash with wear-levelling controller) and plausibly extend the lifespan further by disabling logging functions internally for reduced amount of data being written.
So, log is the drug to HHD drives, right?
However, seeing as that this would be in a corporate environment of sorts, I'm not sure if this is feasible. Not to mention, a full install writes much more often to the CF card and the chances of corruption upon power loss would be greater.
Having been through 3 harddrive failures in my boxes over the past 2 years or so, I'm slowly migrating them to embedded on solid state media. Not to mention, power consumption is greatly reduced and thermals are improved on the tiny boxes.And let me know if I'm wrong:
The best usage in an environment running Proxy/Snort is a large CF card, or better yet, a Sata SSD. -
Wear levelling is probably the most important feature in prolonging the life of flash that is being written to frequently. Almost any modern SSD will have it, while most CF cards will not. If a CF card has it the marketing will tell you so.
SLC flash generally has 10x the write life cycle of MLC flash, and so is another good feature if you're not using embedded pfsense.
-
That is enlightening. Thx clarknova and dreamslacker.
-
So, log is the drug to HHD drives, right?
And let me know if I'm wrong:
The best usage in an environment running Proxy/Snort is a large CF card, or better yet, a Sata SSD.1) Not quite but logging is a form of writing to the drive. Evidently, less logging equates to less data being written. If you intend to attempt a full install on a CF card, make sure it is minimally a SLC NAND flash. Ideally it should also have wear-levelling (will be stated prominently in marketing literature). The Transcend CF200 series of CF cards have both. The regular industrial flash from them uses SLC but doesn't have wear levelling.
2) You will want to probably use a HDD for such usage patterns. Those that are built to run 24/7 are nice to have but not exactly necessary. Western Digital Raid Edition & Velociraptor or Hitachi's CinemaStar are nice but may be a bit pricey for some. I won't touch the Seagates short of their SCSI/ SAS drives so don't expect any recommendations here.
As far as SSDs go, the best bang for buck now would have to be either the Sandforce 1200 series or the Intel X25M. However, both will still require some form of clean-up (TRIM) which you aren't going to get on pfsense. The only controller that does a clean-up without OS filesystem support is the Indilinx but it doesn't quite stack up in performance. So at some point in time, you're still going to have an SSD that bogs down.
I'd say, save the money from buying an SSD. Get more cooling for the HDD and more importantly, get a UPS that is supported by the NUT package so that the pfsense box can gracefully shutdown during a power failure. -
As far as SSDs go, the best bang for buck now would have to be either the Sandforce 1200 series or the Intel X25M. However, both will still require some form of clean-up (TRIM) which you aren't going to get on pfsense.
All true. Without TRIM your SSD's performance will degrade over time. Some models experience this worse than others, and you can look at anandtech.com's SSD section for all the details and benchmarks you could ever wish for.
Besides trim, there are issues that will affect an SSD's performance and life expectancy, the most significant being write amplification, partition alignment, and IO scheduling. Unfortunately pfsense does not currently address these things, as SSDs have not been a focus, and frankly pfsense's storage performance is not even a consideration for most users.
The exception however is squid. In particular, if your squid is going to be doing high volume with many users, a well-tuned storage/IO system should really maximize the performance advantage of squid, which is why I have ultimately decided to handle my SSD in Linux and export the squid cache to pfsense via nfs. I don't know how well this will perform over a standard hdd install, but I have some R&D funding, so I'm going to find out. At worst I will extend the life of my SSD as long as possible and maintain a consistent performance. At best squid will take full advantage of the SSD's superior IO performance to shine under load.
I mention this because you stated from the start that you intend to do a 2-box solution, and I think this is one sensible option for setting it up (particularly if you decide to go SSD for squid cache). Another option would be to move squid entirely to the Linux box and just let pfsense do the firewalling. Either way, squid + SSD + pfsense is not an ideal combination until pfsense includes at least TRIM support, and preferably some install support for saner partition alignment.
On the other hand, there's nothing wrong with a hard drive install. The trail is a little more clearly marked ;)
-
Well, that's awesome. Anyone increases knowledge with you two, guys.
-
clarknova basically nailed it.
I'm not sure how much latencies would be incurred by NFS but it's definitely much better than traversing the internet to retrieve the data. With enough knowledge/ help, I do suppose he could try to implement iscsi or gasp FC-HBA for better performance.
That said, this is out of the realms of us mere mortals without much in-depth knowledge of the OS's or coding and re-compiling them.If you just want to stick to one box, I vaguely recall seeing a guide on the forums for attaching and mounting a separate hdd just for Squid. That could be a way to go for embedded with a separate mechanical drive for caching so your flash drive doesn't get thrashed.
Also, since you have the budget for a SSD, I do suppose you can go for more RAM and a Velociraptor. Whilst the latter won't quite hold its own against the SSDs for small files IOPS, it is remarkably decent for larger files. You can always tune Squid to cache smaller files in memory (hence, the beefing up on ram) and larger files on the Veloci. (reducing the 'penalties' of small I/Os which isn't quite its forte).
Whether you go for SSD or mechanical drives, you'll still want a UPS for full installs to lower the risk of corruption (half written files on SSD is no less corruption than on a mech. drive during power losses).