VIP and MAC Addresses
I've used pfSense for a while now, but this is the first time I've ever posted in the forums.
The problem I have is with a VIP on a WAN interface. I have two identical physical appliances running pfSense (old Ironport boxes) each with three NICs as follows:
Onboard 1: WAN
Onboard 2: LAN
Onboard 3: Cluster (heartbeat)
The WAN links go to a gigabit switch then off to the Internet router, the LAN links go to a pair of HP switches. The LAN links also run a number of interfaces (one per VLAN). On top of that, for each VLAN there's a virtual IP (CARP) replicated to the second box. The WAN side also has multiple virtual IPs.
All the CARP stuff seems to be working perfectly on the LAN side as routing works perfectly and I can ping with no issues.
However, the WAN side doesn't seem to work properly. I've configured pfSense for AON and I'm pushing all traffic through the .236 VIP, which seems to be working nicely. The problem is this:
I'm getting ping drops (about 50%), but only when I ping a device that passes through the WAN interface (the router or any external site). Traffic will pass fine for a few seconds, then drop for a few seconds (about 4-5 each time). However, I accidentally set (and don't ask how) the MAC addresses for the WAN interfaces to the same address and the problem largely disappeared - now I only get a single ping drop once every minute or so. What I don't understand is why that seems to have reduced the number of ping drops.
I've checked and only one of the WAN interfaces is hosting the VIP, so it's not that I have two masters.
Any ideas? I'm running out of ideas pretty quickly!
This is what it looks like on my side:
3 MACs with an IP each:
192.168.3.203 08:00:27:fe:07:7d v-pfSense1.home.xxxx.net WAN < "physical" box1
192.168.3.204 08:00:27:68:d9:26 v-pfSense2.home.xxxx.net WAN < "physical" box2
192.168.3.201 00:00:5e:00:01:02 << Virtual IP (as seen from other device on WAN)
you don't need to mess with MAC addresses, it uses the physical ones and creates one for the virtual IP.
Are you doing NAT or classical routing on the WAN?
What is the gateway for the device on the WAN interface in case of classical routing (should be the VIP)
A layout/IP plan would help if further help is needed.