Firstly I would like to say that PFSense is an excellent product.
We are trying to roll out a pretty juicy installation. Our goal is to have 4 boxes in a 2 active+2 failover mode with each box having 5-6x 1GigE interfaces, 2Gb for Wan, 2Gb for Lan and 1Gb for CARP+PFSync.
We are actually planing to use this on the Intel SR1520ML platform and have a reseller working to preload PFSense on these boxes for an off the shelf redundant solution in a 1U package (note that the power is not redundant and both boards share it so until that is figured out, it makes sense to get 2 for a total of 4 firewalls…and they are cheap even with the donation to this project tacked on).
We have a box configured similarly to 1 side of the SR1520ML setup for testing purposes currently with a late Oct 08 snapshot of 1.2.1. The Wan links currently terminate at the same provider via a switch and in future this may change to multiple providers. Currently there are multiple GB links from the provider going into that switch. As we are unable to bond the links into a single link, the 2 WAN links of this box are setup with separate IP addresses with the same gateway, Wan1 and Wan2.
The Lan is configured similarly as both links are on the same subnet with different IP addresses, Lan1 and Lan2. There are about 50 webnodes on the LAN each with a private and public IP over 2 interfaces.
What we want to do is split the outbound traffic on these links evenly. The idea was to set half the machines to use Lan1 and the other half to use Lan2 as their gateway. We would then proxyarp all the public IPs of the webnodes on Wan1 or Wan2 depending on where their gateway pointed. For example if the gateway was Lan1 then the proxyarp would be done on Wan1.
A test was made and it seemed most of the traffic was going out Wan1 only. This is because there was no balancing of any kind and we have realized we would need to force traffic depending on its incoming interface for our model to work.
We are trying to figure out how to force the traffic in on Lan1 out on Wan1, in on Wan1 out on Lan1, in on Lan2 out on Wan2, in on Wan2 out on Lan2. In straight PF we should be able to do this and we are wondering how/if it could be done on PFSense. Would it be better just to load balance both Lan1/2 and Wan1/2 instead, letting the traffic crossing the interfaces?
There are a few other issues with replacing the existing load balancer by placing one on each pfsense box Wan interface to balance all of its proxyarped address but we need to get over this first hurdle before the next thing is accomplished as well as the CARP functionality.
Any advice at all will be appreciated.