How to use policy routing with multiple WAN and Squid ?

[duanes]

My installation has three WAN connections.  All three are good for internet, but due to cost and capacity, certain types of traffic are preferred over certain links… ie, email is preferred over the intra office private line.

I want to fail over to alternate gateways in case of link failure, so I have setup several gateway groups.  InternetGroup has the preferred NetGate as Tier 1 and OfficeGate as Tier 2.  EmailGroup is just the opposite.

I have created a firewall rules.  For simplicity, anything with a destination of the company email server gets the gateway set as OfficeGate.  Everything else is NetGate.  The problem is that if I don't set one of the Gateways as the Default Route, then the firewall itself and thus pfsense cannot get to the internet.  But, If I set one of the gateways as the default and it goes down, then the firewall still looses it's ability to reach the internet (although any client PC on the LAN works and fails properly).

After spending a few weeks trying to troubleshoot an unrelated issue, I also believe that Squid (in transparent mode) also follows only the default gateway and does not follow the policies.

Any suggestions ?