pfsense ha both staying master on wan side

[vrajkumar]

Hi I have a pf sense cluster. CLuster is proper on lan side but wan side both showing master. While doing a pcap on both wan master sending a carp advertisement but slave is not seeing it. how to troubleshoot it?

10:41:48.814786 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:49.092960 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:50.144620 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:50.227570 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:51.204953 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:51.662518 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:52.219710 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:53.141266 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:53.283010 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:54.336882 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:54.620101 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:55.370097 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:56.079753 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:56.388804 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:57.452014 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212
10:41:57.489755 IP slave > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=100 authlen=7 counter=5770705708551257802
10:41:58.515019 IP master > 224.0.0.18: CARPv2-advertise 36: vhid=10 advbase=1 advskew=0 authlen=7 counter=9074612109644037212

[Derelict]

If they are both advertising then they are likely not seeing each other's advertisements. Since that is a capture on the primary, which can see the secondary's advertisements, what does a capture on the secondary look like?

This is often due to having the WANs connected to some ISP device that has things like MAC limitations, limitations passing multicast traffic, etc.

The first thing I would do is connect them both to an unmanaged switch (or even just a looped cable between the two WANs) and see if they go MASTER/BACKUP. Then figure out why that is not happening when they are connected to whatever you have them connected to.

[vrajkumar]

@derelict Hi Now I am able to see both advertisements in both pfsense switches!!!. Not sure why backup pfsense sending advertisements... But still on backup pfsense wan side is showing master.

[Derelict]

Each interface will have a physical interface name, such as em0, ix1, igb0, re4, bc2.

You can get this in Status > Interfaces

Then in Diagnostics > Command Prompt execute ifconfig em0 substituting em0 for the correct interface name of your WAN and post the output. Please do not sanitize more than the first couple of octets of any addresses.

Also please post a quick WAN pcap of the CARP traffic seen on both nodes. Please set the level of detail to Full.

[thesurf]

@vrajkumar said in pfsense ha both staying master on wan side:

@derelict Hi Now I am able to see both advertisements in both pfsense switches!!!. Not sure why backup pfsense sending advertisements... But still on backup pfsense wan side is showing master.

Hi,

if you see this on both switches thats fine. Are this Switches connected? the WAN Interfaces sending both announcements and then they decide which one will be backup on the base and skew. (lowest should get master).

[Derelict]

The BACKUP node will not be advertising. Only MASTER VIPs advertise.

@vrajkumar said in pfsense ha both staying master on wan side:

@derelict Hi Now I am able to see both advertisements in both pfsense switches!!!. Not sure why backup pfsense sending advertisements... But still on backup pfsense wan side is showing master.

I have no idea what a pfSense switch is unless you're talking about an XG-7100 or SG-3100.

[thesurf]

I see both sides advertise until one goes to backup mode.

He wrote about a switch. I think he has a wan Port connected to a switch and both pfsense to it.

Can you explain the setup a little bit more?

[Derelict]

Right. But they should only both be master for an instant. They should never both advertise for any length of time.

[thesurf]

Absolutely. That's what I would see.

If as he wrote sees on the switchs advertisement and both send them my assumption is that he has a switch for each wan line. To work with carp as failover there are two options.

A) connect both switches so the advertising packages can be seen by both pfsense.

B) from each pfsense connect a port to each switch and setup two virtual IP with the regarding interfaces.

Hope that is clear. Else please ask. Maybe I can later provide a drawing.