Failover Troubleshooting ideas wanted



  • I got two FW's up  (on ESXI) - 2 physical Networks, on the physical LAN side i have LAN+CARP and physical WAN side is only WAN
    i have allowed promiscuous etc. on esxi hosts.

    My problem is - when I turn fw1 off, fw2 kicks in on the shared CARP IP's (i can access the admin interface from both LAN and WAN side - (on the shared IPs) and it all looks good virtual IP's, NAT rules etc. seems to be there. (so param sync seems to work)

    but no traffic is allowed thru the the fw - either way… when fw1 is booted up again - it takes over - and it all works normal.

    any ideas what / how to - check / test /  troubleshoot?

    edit - setup visualized - >




  • Should ALL (that I want to failover) WAN side Virtual IP's be;  Normal Virtual IP or CARP IP's ?

    when I sat them to CARP' ips - and tried to access services behind FW i got stucked at fw with the DNS Rebinding warning…...



  • @planetinse:

    Should ALL (that I want to failover) WAN side Virtual IP's be;  Normal Virtual IP or CARP IP's ?

    when I sat them to CARP' ips - and tried to access services behind FW i got stucked at fw with the DNS Rebinding warning…...

    There must be 1 CARP assigned on each fail-over interface, further IPs can be "IP Aliases", however, CARP should work also.
    Are your network masks of the interfaces configured correctly on both boxes and same on CARP? IP Aliases could have /32, but CARP should have the correct mask.



  • ok - lets give it another try. mabye i screwed something up first try.



  • so… when swap from Virtual IP to using CARP Virtual IP - the NAT rules are ignored. (instantly stops working)
    mabye i need to delete and add them again? (will try that now)

    traffic not allowed thru fw (for that WAN ip) (tried reboot) however when IP's handled by HAProxy it seems to work with the CARPS  (at least on primary) have not tried failover yet.



  • switched all Virtual IP's to CARP type, everything looks ok, fw2 takes over the IP's as master …

    unfortunatly- no traffic goes thru , not haproxy, not nat, not 1:1...

    so next question - allow promiscous mode - what networks do I need to enable that on?

    all? or just WAN side or WAN LAN or LAN WAN OTH - or any other combo? :-)



  • @planetinse:

    so next question - allow promiscous mode - what networks do I need to enable that on?

    all? or just WAN side or WAN LAN or LAN WAN OTH - or any other combo? :-)

    Every interface that has a CARP IP, as that's the only way VMware will allow the CARP MACs to the VM.



  • Partial success !!

    isse was Promiscous mode on one of the interfaces on one of the ESXIs had not been enabled.

    Now failover works perfectly for all pfsense NAT's - however the stuff handled by HAProxy still fails to failover
    what happens is instead of ending up at backend iam getting handled as I would to access
    the fw itself

    it forwards to https and the port i have set for web access .- and then i get the DNS Rebind error.

    if i manually go fw2 - and check HAproxy it looks ok - all settings are there etc.



  • Can you confirm that haproxy is running and listening on the port you are trying to access in diagnostics\sockets ?

    Do you use the carp-monitoring feature of haproxy? Which could cause that haproxy is shutdown on the backup machine, it might still be starting the first half second after fail-over..