CARP/HA is incompatible with dynamic addressing.

Get a static /29 from them instead and you'll be all set.

More information… it appears that I can successfully telnet to the VIP on port 26 from another LAN. When initiated on the same LAN/subnet as the VIP, the connection never responds. On this subnet there is only one firewall rule that allows all in/out on any protocol for IPv4+IPv6, so I there isn't any possible rule that could be blocking.

Sorry, I have been away for some times.
–> First option: I got vmx0, vmx1, vmx2 and vmx3. After adding the new NIC, it appears as vmx4 and the existing NIC remain unchanged.

Edit: I tried adding a different type of NIC (E1000 instead of VMXNET3) and both scenario worked. So I guess you are right, it is somehow related to the interface naming scheme.

You need three addresses.

Thanks, not sure what that CARP event would mean in practice. If WAN is down I don't really care if pfsense would switch between the units as long as it is restored when WAN is up again.

But I'm getting the feeling that I'm introducing more ways things can fail rather than mitigating actual risks, I'll see if I can get into the network some other way and restore functionality manually when something have happened.

Hello,

I am having similar problems.  IPV4 failover works beautifully.  But not ipv6.

fw1 and fw2 both have the same tunnel broker settings, both firewall GIF connections are tied to the wan carp ip.  when fw2 is master, ipv6 stops.  when fw1 is master again, ipv6 connectivity returns.

I have compared the output of netstat -rn and ifconfig -a to each other.  The only real difference appears to be how ipv4 is mapped to carp whereas ipv6 is NOT.

What I theorize is this: until HE.net re-pings the ipv4 client address, connectivity is lost.

Do I need to create a virtual ip address on the tunnel interface and assign the he.net assigned client ipv6 address to it?
is there a way to convince HE.net to allow me to use more than one client ipv6 addresses ie one for fw1 one for fw2 and one for carp?  the server and client ipv6 addresses both are /64…

is there a mechanism to bump HE.net if a carp changover has occurred?

Thank you in advance for your time...

==jason

The configuration versions of pfSense must be identical on both units.

It can't synchronize from 2.3.4 to 2.3.2 because they have different configuration formats

https://doc.pfsense.org/index.php/Versions_of_pfSense_and_FreeBSD

2.3.4 is revision 15.8, 2.3.2 is 15.5. Attempting to synchronize that could cause a bad configuration to be loaded on the backup unit.

Upgrade both to 2.3.5-p1 (or preferably 2.4.2-p1) and try again.

Hello, again!

I managed to resolve this problem myself, when I found that Snort package, which I had installed and configured on WAN, was dropping the state on the slave (the box that is becoming the master on failover), because of the ongoing download for which the initialization was only seen by the snort on the previous master. Hence Snort thought it was an intrusive packet and denied the connection.
That also explains, why it was all good again on the master box after failing back.

Hope this helps someone, and sorry for the wasted time!

Yes. Being in a virtual environment might cause an unplugged cable to NOT result in an actual interface DOWN to the virtual machines because they are still connected to the vswitch. If your virtual environment supports simulating an unplugged interface there you should try that. In short, it is up to your hypervisor to actually take an interface down from the VM's perspective.

I use XenServer and that is pretty hard to simulate there - at least in the 2 minutes I devoted to trying to figure out how to do it.

You might also try just taking the interface down in software

ifconfig xn0 down

Dec 15 20:21:39 kernel carp: 236@xn0: MASTER -> INIT (hardware interface down)
Dec 15 20:21:39 kernel carp: demoted by 240 to 240 (interface down)
Dec 15 20:21:39 kernel carp: 239@xn0: MASTER -> INIT (hardware interface down)
Dec 15 20:21:39 kernel carp: demoted by 240 to 480 (interface down)
Dec 15 20:21:39 kernel xn0: link state changed to DOWN
Dec 15 20:21:39 kernel carp: 240@xn2: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn2: 3
Dec 15 20:21:39 kernel carp: 237@xn2: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn2: 3
Dec 15 20:21:39 kernel carp: 241@xn4: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn4: 3
Dec 15 20:21:39 kernel carp: 243@xn5: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn5: 3
Dec 15 20:21:39 kernel carp: 238@xn1: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn1: 3
Dec 15 20:21:39 kernel carp: 242@xn5: MASTER -> BACKUP (more frequent advertisement received)
Dec 15 20:21:39 kernel ifa_maintain_loopback_route: deletion failed for interface xn5: 3
Dec 15 20:21:39 kernel carp: 228@xn1: MASTER -> BACKUP (more frequent advertisement received)

Secondary takes over for all VIPS. All VIPs on primary are either INIT (the two on xn0) or BACKUP (everything else.)

ifconfig xn0 up

Dec 15 20:23:44 kernel carp: 236@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44 kernel carp: demoted by -240 to 240 (interface up)
Dec 15 20:23:44 kernel carp: 239@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44 kernel carp: demoted by -240 to 0 (interface up)
Dec 15 20:23:44 kernel xn0: link state changed to UP
Dec 15 20:23:44 kernel carp: 236@xn0: BACKUP -> INIT (hardware interface up)
Dec 15 20:23:44 kernel carp: 236@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44 kernel carp: 239@xn0: BACKUP -> INIT (hardware interface up)
Dec 15 20:23:44 kernel carp: 239@xn0: INIT -> BACKUP (initialization complete)
Dec 15 20:23:44 check_reload_status Linkup starting xn0
Dec 15 20:23:44 kernel carp: 239@xn0: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 236@xn0: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 241@xn4: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 240@xn2: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 237@xn2: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 243@xn5: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 242@xn5: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 238@xn1: BACKUP -> MASTER (preempting a slower master)
Dec 15 20:23:44 kernel carp: 228@xn1: BACKUP -> MASTER (preempting a slower master)

I have the very same issue, but there's no apply button to fix. Need to change the VIP from IP Alias to CARP and then back to get it to work.

They need to be enabled and set to use the CARP VIP.

Hi,

So the simplest question is: in a carp setup how what can I do to be sure that also slave can reach internet?

In your MASTER, Firewall/NAT/Outbound, make your Mappings similar to the attached pic, 2nd entry. In your case all references to WAN1 should just be WAN. If as you say your CARP is properly configured, the MASTER settings should replicate to the SLAVE via the SYNC interfaces. You do have SYNC interfaces, properly configured & connected, right?

What does this 2nd entry do? It ensures that internet access for the pfSense machines (MASTER/SLAVE) themselves (127.0.0.0/8), goes thru their respective WAN IP addresses.

As for your "several" LANs, LAN2 in particular, create rule(s) similar to the 4th entry. This ensures the allowed LAN machines can access the internet via the designated WAN CARP VIP.

Anyway if you followed the CARP setup docs, all those entries should have been more or less taken cared of already.

Cheers.
Edwin

pfs-nat-outbound.jpg_thumb
pfs-nat-outbound.jpg

Like I said, you will probably need to pcap to see what is really going on.

Are you using your own switch or the ISP device?