Failover Troubleshooting ideas wanted

planetinse

I got two FW's up (on ESXI) - 2 physical Networks, on the physical LAN side i have LAN+CARP and physical WAN side is only WAN
i have allowed promiscuous etc. on esxi hosts.

My problem is - when I turn fw1 off, fw2 kicks in on the shared CARP IP's (i can access the admin interface from both LAN and WAN side - (on the shared IPs) and it all looks good virtual IP's, NAT rules etc. seems to be there. (so param sync seems to work)

but no traffic is allowed thru the the fw - either way… when fw1 is booted up again - it takes over - and it all works normal.

any ideas what / how to - check / test / troubleshoot?

edit - setup visualized - >

basic_fw_layout.PNG_thumb

planetinse

Should ALL (that I want to failover) WAN side Virtual IP's be; Normal Virtual IP or CARP IP's ?

when I sat them to CARP' ips - and tried to access services behind FW i got stucked at fw with the DNS Rebinding warning…...

viragomann

@planetinse:

Should ALL (that I want to failover) WAN side Virtual IP's be; Normal Virtual IP or CARP IP's ?

when I sat them to CARP' ips - and tried to access services behind FW i got stucked at fw with the DNS Rebinding warning…...

There must be 1 CARP assigned on each fail-over interface, further IPs can be "IP Aliases", however, CARP should work also.
Are your network masks of the interfaces configured correctly on both boxes and same on CARP? IP Aliases could have /32, but CARP should have the correct mask.

planetinse

ok - lets give it another try. mabye i screwed something up first try.

planetinse

so… when swap from Virtual IP to using CARP Virtual IP - the NAT rules are ignored. (instantly stops working)
mabye i need to delete and add them again? (will try that now)

traffic not allowed thru fw (for that WAN ip) (tried reboot) however when IP's handled by HAProxy it seems to work with the CARPS (at least on primary) have not tried failover yet.

planetinse

switched all Virtual IP's to CARP type, everything looks ok, fw2 takes over the IP's as master …

unfortunatly- no traffic goes thru , not haproxy, not nat, not 1:1...

so next question - allow promiscous mode - what networks do I need to enable that on?

all? or just WAN side or WAN LAN or LAN WAN OTH - or any other combo? :-)

cmb

@planetinse:

so next question - allow promiscous mode - what networks do I need to enable that on?

all? or just WAN side or WAN LAN or LAN WAN OTH - or any other combo? :-)

Every interface that has a CARP IP, as that's the only way VMware will allow the CARP MACs to the VM.

planetinse

Partial success !!

isse was Promiscous mode on one of the interfaces on one of the ESXIs had not been enabled.

Now failover works perfectly for all pfsense NAT's - however the stuff handled by HAProxy still fails to failover
what happens is instead of ending up at backend iam getting handled as I would to access
the fw itself

it forwards to https and the port i have set for web access .- and then i get the DNS Rebind error.

if i manually go fw2 - and check HAproxy it looks ok - all settings are there etc.

PiBa

Can you confirm that haproxy is running and listening on the port you are trying to access in diagnostics\sockets ?

Do you use the carp-monitoring feature of haproxy? Which could cause that haproxy is shutdown on the backup machine, it might still be starting the first half second after fail-over..