** Quick update at the bottom **
Thanks for your reply mhab12. I didn't intend make it more difficult. Partly, my pfsense boxes are not the most robust machines and I've noticed that there are limits to what I can have running on one box at a time. For instance I have had to limit what rules are running on snort using one box and compensate the snort on the other box, kind of splitting the load in a sense, so that one box covers certain rules and the other box the rest. There are some rules in snort that cause my service to stop if I have too many selected. I don't have the best boxes with the up to date components, but I wanted to make it still secure enough and not overwhelm one boxes resources.
Thanks for the heads up on the reverse proxy, I may give that a shot since my web box is not that great either and it would be less for it to deal with if one of the pfsense boxes to could handle a little of the load. I think what I was finding is too much on one machine slows things down, but sharing resposibilities between boxes will lower the load on the computer and also give me more security on my home network as a perk. I hope I didn't sound psycho about having two pfsense for security, I'm just better at visualizing things and this made sense for troubleshooting and, for some reason, give me a quick way to get the internet back up if one box goes down.
@mhab12:
Couple of thoughts:
1 - You mention binding squid to WAN. This will not do what you're thinking and cache the outbound data from a 'slow' web server. Doing this will require something called reverse proxy. The squid package in pfSense will do it, yes, but it requires additional configuration beyond the included GUI.
2 - It sounds to me like what you're explaining could be accomplished by just adding an extra NIC to the first pfSense box. By creating an OPT interface (likely OPT1), you can effectively have two LANs, LAN and OPT1, one will be 1.1 and one 2.1 You can setup firewall rules to prevent/limit access between them, setup bridges, anything you need. If you do not trust the firewall rules well enough and chose to have two boxes for that reason, that's another issue.
** Update for my setup **
Just letting everyone know that I now have 1.1 running snort with rules split between it and 1.2 network pfsense boxes. This is the main reason I wanted to set things up in this way, because I don't have the newest boxes and only 512mb ram in each. I guess if I had a nice firewall box then it would be unnecessary for my setup, but I'm using what I've got… my ram usage on 1.1 is at 62% with snort and squid running, and my ram usage on 1.2 is 68% with snort running 2 main rules and 2 empty rules. I may end up swapping rules on the machines and see if I can balance them a little better, but for now I have backdoor and netbios running with the largest rules and then the two empty ones local and experimental. The rest of the rules are running on the 1.1 pfsense box, but since it has a faster processor I may end up squeezing more out of these rules if I swap the rules between the two boxes. We'll see how things go.
Just FYI.
