VIP Changing between physical MAC and VIP MAC
-
Trying to setup an HA with Multi-WAN fail-over. Both firewalls seem to be configured properly, I can confirm that the VIP on the internal interface is working with no problem as well as the HA Sync. I have three external IP addresses per ISP. Each ISP modem is connected to a Layer 2 unmanaged switch, which then one port is connected to one FW the other the other FW. This setup I have consists of 4 NICs total on each PFsense machine. Here is the hardware I am using: https://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm
Each box is identical, except one is Master and the other Backup of course so I know my HA sync is working.
My problem here is when I have one ISP connected the IP address assigned to the VIP never shows up on the modems ARP table. Instead, the IP Address assigned to the physical NIC changes MAC address between it's actual MAC and the VIP MAC. If I force it to fail over to the backup FW, then the IP address assigned to the physical NIC on the backup FW will do the same thing between it's MAC and the VIP MAC. From what I can see through Wireshark and the console logs I didn't notice any of the MAC switching behavior the ISP saw coming across the modem, though I'm not an expert at looking at either of those, I did try to trace each IP address and the VIP never came up in my Wireshark session when I was in promiscuous mode connected to the WAN switch...
That's alot of explaining, but any help is greatly appreciated so please don't hesitate to ask for a log or a diagram or clarification on any of this.
Thank you in advance to anyone who replies.
-
Each ISP modem is connected to a Layer 2 unmanaged switch, which then one port is connected to one FW the other the other FW.
Different switches per WAN correct?
Each box is identical, except one is Master and the other Backup of course so I know my HA sync is working.
The SYNC interface has nothing to do with the CARP VIP status on each interface or which node is master or backup at any given time.
https://forum.netgate.com/post/719523
My problem here is when I have one ISP connected the IP address assigned to the VIP never shows up on the modems ARP table.
The CARP MAC only shows up in the upstream MAC address table due to the CARP advertisements.
When the node holding the CARP MASTER status sees an ARP request for the CARP VIP, it answers with an ARP response. This ARP response is sourced from the interface MAC address but contains the CARP MAC address as the ISAT MAC address.
There is no reason for the modem to contain the CARP VIP in its ARP table unless it needs to route traffic from itself to the CARP address.
That said, MANY ISP devices simply do not do what is necessary for CARP to function correctly. They might only allow one MAC per port or any of a number of silly things.
Some work fine.
