Pfsense : CARP interface reply to arp request with 2 mac address

[chuyengiason]

Hello, I have 2 pfsense boxes.

• LAN and WAN interfaces are configured to use carp and VIP.
• I see that master box, reply to to arp request for LAN VIP with 2 message : 1 claim that the VIP has mac address is physical mac and 1 claim that VIP has mac address as CARP mac.
• So, the PC in LAN got confuse and think that the CARP VIP is duplicate and refused to use it as default GW.

Here is the sysctl configuration of psense

device  carp
net.inet.carp.allow: 1
net.inet.carp.preempt: 1
net.inet.carp.log: 1
net.inet.carp.demotion: 0
net.inet.carp.senderr_demotion_factor: 0
net.inet.carp.ifdown_demotion_factor: 240
net.inet.carp.stats: Format:I Length:128 Dump:0x44866b01000000000000000000000000…
net.link.ether.inet.carp_mac: 1
net.link.ether.inet.log_arp_wrong_iface: 1
net.link.ether.inet.log_arp_movements: 1
net.link.ether.inet.log_arp_permanent_modify: 1
net.link.ether.arp.stats: Format:I Length:96 Dump:0xb2ac0800000000005d9c020000000000...
net.link.bridge.ipfw_arp: 0
net.pfsync.carp_demotion_factor: 0

In Linux, I used keepalived and it has : arp_announce and arp_ignore do the trick. But I can not find any setting like that in pfsense/freebsd
Pls help to fix it
Tks

[Paint]

if you go to the System -> Adavanced -> Networking tab, you should check the last box on the page to enable ARP Handling - This option will suppress ARP log messages when multiple interfaces reside on the same broadcast domain.

This setting will disable these two sysctl settings:

net.link.ether.inet.log_arp_movements: 0
net.link.ether.inet.log_arp_wrong_iface: 0

[dotdash]

I think those sysctl's just suppress error messages about duplicate macs. OP- what OS was the PC running that wouldn't accept the gateway? I've never seen that behavior.

[chuyengiason]

The 2 sysctl options are about logging only.

Btw, the PC running freebsd also.
PC point GW to VIP of LAN interfacen in pfsense.
When I ping to the internet, and do tcpdump in PC. I see that it ask for mac address of VIP and receive 2 reply from pfsense.

I also have carp configured in 2 freebsd servers. The configuration is quite the same and the freebsd server return arp correctly. I'm not sure why problem happen with pfsense :(

[chuyengiason]

1 thing is

• master pfsense running 2.2.1 version
• slave running 2.3.1

[dotdash]

@chuyengiason:

1 thing is

• master pfsense running 2.2.1 version
• slave running 2.3.1

That could be a problem. I would upgrade the master so they are at the same version.

[Derelict]

Code level differences like that should really only be run for a minimum of time. Enough time to know everything's working, then update the other node to match. If not, fail back and restore the secondary to the working version. You will find that the closer the two nodes in the cluster are to each other (hardware, software, etc) the happier your cluster will be.

The interface will respond with the interface MAC for ARP for the interface address. The unit that is CARP master will respond with the CARP MAC for the CARP VIP address. The ARP request will be for one IP address or the other.

When you're looking at the ARP traffic, you see a WHO HAS X.X.X.X IP address. Only the MAC address that has that actual IP address will respond.

Need more details about what you're really seeing, like specific IP addresses, MAC addresses, and probably packet captures showing what you're seeing to be of any sort of assistance. Both nodes please.

And so we all are talking about the same things let's use the same terminology:

Primary - the node that is usually Master and sends its config XMLRPC Sync to the other node.
Secondary - the node that is usually Backup and does not send config XMLRPC sync to the other node.

Master - the node that is currently CARP master
Backup - the node that is currently CARP backup.