You have asymmetric routing. Check "Bypass firewall rules for traffic on the same interface" under System > Advanced.

Oh, that's likely the same thing with a different symptom, with PPPoE your WAN is actually ng0 not the physical interface, and when you disconnect you lose "link" on that.

i'm running a similar setup as urs. just that the WAN connection still remains on pfsense 1/2 .. whereas pfsense 3 is doing all the Load Balancing work .. it would be nice to do dual PPPoE load balancing.. one downside to this setup is that u'll need to have port forwarding rules on both pfsense 1/2 in case if you want external users to get in your internal network..

Now that I think about it you are correct.  I created VIPs on PFsense for every address except the 62.114.15.29/28 address that I use on the WiFi.  So based on what your saying.  I guess I don't have an issue.

As an FYI I did end up placing a check on the "suppress ARP msg when interfaces share physical network".

I was thinking I may have been creating a packet storm or whatever its called but I'm guessing I'm ok.

Jason, THANK YOU for your response and assistance.

Thank You!  :)

Yes such a setup should work without problem.
In your diagram in building2 i would set the normal WAN to the fibre link and the DSL link as an OPT.
In your building1 set the microwave link as normal WAN and the fibre link as OPT.

Hi Jerry,

Yeah, I've previously tried all three you've mentioned :)

I'm no expert either, but I do what I can.

In all 3 cases, the response is negative.  My DNS queries are not being handed over to my local DNS servers AND on top of that, my Blue network has no (apparent) access to the outside world.

I do something similar with Vonage in a dual wan environment using failover only (no load balancing).  I have a LAN firewall rule to bind the source IP (or alias) of my Vonage ATA to the WAN connection using the appropriate load balancer pool as the gateway.  When both wan connections are up, Vonage uses WAN and everything else uses OPT1.

You can get these information per SNMP.
Although you need another software or write something to interprete the information you retrieve.

actually figured it out. you have to save it with an interface name, and choose "add to pool"

Chris got back to me today and helped me setup manual NAT (Reverse NAT) to convince the OPT1 network that machines on my network were the IP the OPT1 (Private LAN) interface.  This allows me to communicate with all the computers on the Private LAN properly.

Thanks Chris!

I think that it's all the same problem:
http://forum.pfsense.org/index.php/topic,21991.0.html

You can route to an arbitrary gateway using the Load balancing feature as long as you edit the config file manually as was detailed in the thread I linked to.  The problem is that it doesn't work 100% for me.  It could be something related to my NetGuardian or it could be a pfSense issue, I haven't nailed it down yet.

10.1.1.100 doesn't know where to find 172.16.1.1 unless

a)10.1.1.1 is its default gateway or
b)it has a static route designating 10.1.1.1 as the gateway to 172.16.1.1/(24-32)

If 10.1.1.100 can ping 172.16.1.1 then perhaps you have done one of the above?

i have the same problem with load balance.  how to do that with failover pools  any idea ?

thanks

That TelstraOnly was a test, sorry I should have changed it back before I made the screenshots.

I made a loadbalancer with just Telstra(WAN2) in it (as I read somewhere else this might help)

I wish that was the problem, I really wish it was obvious to someone why this is happening…