NEED INPUT: Port 80 Traffic Times Out on Failover Setup

[caldwell]

Working on a very basic setup:

WAN = Charter with static iP – set as the default gateway
OPT1 = AT&T UVerse with static IP

Have a failover type gateway set up:
  WAN = Tier1
  OPT1 = Tier2
  Trigger on "Member down"

If I unplug the Charter connection, traffic starts to flow out the AT&T link -- except for ANY port 80 traffic to any host.  It times out.  I've done a "tcpdump host hostname" and then done a manual "telnet hostname 80" and typed "GET /", and I can see that the connection TO the server is made, and the server sends back a reply. But everything after that times out.  Again, e-mail traffic, HTTPS traffic, other things work fine.  Just port 80 traffic fails.

If I go in and MANUALLY set the AT&T /OPT1 as the DEFAULT gateway, then traffic to port 80 flows just fine.

Alternately, if AT&T/OPT1 is set as the DEFAULT gateway, and I unplug IT while leaving Charter plugged in, the same thing occurs.  Port 80 traffic simply times out.

The LAN rule simply has:

IPv4 * LAN net * * * FailOver none

as the default allow rule.

I'm not sure what else to debug, and this isn't working as it is supposed to -- at least from what I understand.  The gateway should be marked as down, and whether or not it was assigned as the default gateway, if it's in a failover group, ALL traffic should flow correctly out the failover port/route.

Can anyone offer suggestions, advice, fixes or additional items to troubleshoot?

Thanks.

[caldwell]

Plenty of views, but no responses.  Has anyone experienced this?  Any idea where to look for whatever it is that's blocking port 80 when the failover is NOT the default gateway?

There is no Squid proxy on the pfsense box.

I suppose I can just remove all firewall rules on all interfaces, delete the failover gateway and rebuild.  But there should be a better way to debug this.

[caldwell]

Does anyone with pfsense team have any pertinent replies to this?  I see that someone else has posted a similar issue with FTP timing out.

[luckman212]

Try disabling AON (automatic outbound NAT) and hit apply. Then manually delete all the rules you see there, then re-enable AON and apply. Re-test after that…

[cmb]

Packet capture on the second WAN filtered on 80 and see what the traffic looks like there.

@luckman212:

Try disabling AON (automatic outbound NAT) and hit apply. Then manually delete all the rules you see there, then re-enable AON and apply. Re-test after that…

Don't do that. That'll just break everything as it'll leave you with no NAT at all.

[luckman212]

The NAT rules don't get re-created when turning AON back on? I'm pretty sure I've done this in the past and everything has worked as expected afterwards….  ::)

[Derelict]

I think there might be a misunderstanding since AON isn't Automatic Outbound NAT but Advanced Outbound NAT (aka Manual).

I caught myself making the same misinitialism a few posts ago.