Dual WAN with Failover Not Working

[kejianshi]

So, you are saying failover from packet loss doesn't work?

How are you simulating packetloss?

[kathampy]

Failover pased on a packet loss threshold does work. It works by default.

[kejianshi]

If failover isn't working based on simulation of packet loss then:

Either its broken or

The settings are wrong or

Packet loss is not being done effectively to cause a failover.

Thats why I'm asking how are the packets being dropped.

[rober1sf]

On router 1 in my diagram above, I have an outbound firewall rule that blocks all outbound Internet traffic, thus creating Internet packet loss on WAN 1. I know the packet loss is happening too because the pfSense diag ping tool will have 100% loss, but gateway status will still show it pinging.

In a "real" production setting, I would test this by removing the coax cable from the cable modem because simply unplugging the power to the modem would be link state down, which doesn't happen when the Internet goes down.

[kejianshi]

Are you sure those packets you are blocking are being dropped silently and not rejected with a reply?

REJECT
    Prohibit a packet from passing. Send an ICMP destination-unreachable back to the source host [unless the icmp would not normally be permitted, eg. if it is to/from the broadcast address].
DROP (aka DENY, BLACKHOLE)
    Prohibit a packet from passing. Send no response.

[rober1sf]

When I ping from the workstation behind the pfSense I get response timed out, 100% loss. I believe that is correct, right?

[kejianshi]

Do you own a server on the net anywhere? 
What I would do is maybe set up a couple Centos boxes with public IP you can ping.
Use those IPs as your monitor IPs.

If you shut down the Centos box (no blocking anything), then that will be for sure packet loss.
You and a buddy could set up one at your home and one at his if you want to have control over two "gateway" IPs to use.

I know this sounds like unnecessary work, and it may be, but at least you will know its not your method of inducing packet loss that is flawed.
I suppose you could do the same thing entirely in lab environment with no outside internet.

I don't know if pfsense would know the difference in a packet dropped silently, rejected, or an unreachable offline server. It might.
Since you didn't tell me if you are dropping packets silently, I assume you aren't sure.

[rober1sf]

@kejianshi:

DROP (aka DENY, BLACKHOLE)
   Prohibit a packet from passing. Send no response.

I'm doing a DENY rule in the firewall. I don't think that I need to switch to pinging 2 of my own servers on the Internet (unless you are thinking about something that I'm not) because this setup works in the exact same setup with the Cisco RV042, and the RV042 fails over.


[kejianshi]

When it works for others, but not for you, I'd suggest changing your methods or recheck your settings.  That would be a pretty big bug if it really doesn't work.

[rober1sf]

Ok, so I added a laptop on the same LAN as Router 1 (see attached Visio) and changed my monitor IP on WAN1 to that latptop's IP address. Physically unplugging the cable from the laptop keeps the physical link on WAN1 but allowed for packet loss and a failover to WAN2. Reconnecting the cable to the laptop allowed the pings to begin again and the connection to WAN1 was then automatically restore as well.

WORKED like it should! Still confused by the Cisco RV042 worked with the other setup, but all I can say is that maybe the pfSense "sees" the pings differently from the Router 1 firewall block rule than the Cisco??

At any rate, ready to try in a REAL production environment (see attached Visio for the "UNCOMPLICATED" network you all were looking for ;D ).

Thanks to all for helping. If anyone stumbles across this and needs help, I've also created a step-by-step setup from initial boot/configuration to setup a Dual WAN with Failover… just PM me for a copy.

Thanks again!!


[kejianshi]

Yeah - I wasn't trying to waste your time.  I'm glad its working now. 
I hope your actual install goes well also.