@derelict Understand thank you

Still not sure what you are asking. There are no loops in an HA setup. Seems like more of a question for Cisco.

I would say don't use '/' for the time being.

Allright, so that NAT's are not performed like firewall rules top to bottom Yes. they are. Your any rule at the bottom catches everything so everything has NAT performed on it. The whole point is that some things should not have NAT to the CARP VIP performed on them. In your example you are STILL natting for sources in 127.0.0.0/8, which is where traffic sourced from the firewall itself might be originating for NAT purposes, which is NOT the default configuration. If I were you I would scrap what you have start over with a simple WAN/LAN setup on a test bench, read the docs again, and get a handle on what is involved here. If NAT to the interface address doesn't work, then your /30 /28 scheme, which I said was broken out-of-the-gate, is incompatible with the upstream gear.

And this: The ISP Layer 2 device will see the CARP MAC as the source MAC in the CARP advertisements. They are sent to the Layer 2 Multicast address 01:00:5e:00:00:12 (all points multicast) to Layer 3 multicast address 224.0.0.18. That MAC address has to be added to the switch port's MAC address table based on those. This MAC address will change ports on a failover event. The ISP device must move the MAC address to the new port as any switch should. The ISP Layer 3 gear will get the CARP MAC in response to ARP "WHO HAS" requests for the CARP VIP address. Their gear needs to do the right thing with it. The ARP reply from the WAN interface that is currently CARP MASTER will contain the CARP MAC in the ARP "IS AT" response. This ARP response will be sourced from the interface IP and MAC address. The ISP Layer 3 gear also needs to honor the interface addresses that will ARP as normal. The ISP device will only ever see the interface MAC address on the port connected to that node. All references to ISP gear there should be interpreted in whatever is upstream of the two pfSense nodes in your environment. The Mikrotik and whatever else that is.

I am not sure if this is the best to go but I will increase the MTU to 9000 and see if all goes well :) That is the last thing I would do if I was in your position. I would get everything working perfectly and leave jumbo frames out of the picture.

[solved] The problem strangely solved by re-configuring System -> Routing values. Also I changed the default gw to Automatic (I doubt if this has been effective!)

It would probably be easier to take a backup of the primary, replace opt9 with opt8 in a copy of the backup, then restore the edited version.

You do not need outbound NAT on LAN at all. That is just silly. You should be able to ping both interface addresses and the CARP VIP of the connected subnet if the rules on that interface allow it. If you can ping the interface addresses but not the CARP VIP, check the ARP table of the device you are testing from to be sure it has all three ARP entries. The interface addresses should have the interface MAC address. The CARP VIP should have the CARP MAC. If that is all in place, be sure the switch connecting everything has the CARP MAC in its MAC address table. It should be on the switch port that is currently connected to the CARP MASTER node.

@derelict Sorry because my HA interface is a VLAN I forgot to added into the switch That was the problem Thank you

Well, you can add another switch between pfSense and the other switches....

@netblues : Thank you again - that's cleared up the IP address confusion (and yes I had read in the book that auto-nat wasn't supported with CARP - forgot that in the confusion with PPPoE). Our usual networking hardware supplier recommended NETGEAR DM200-100EUS ADSL/VDSL Modem to replace the ISP supplied router. Reading the manual shows it appears to have routing features. Does this qualify it as a "router device" even though it's being called a "modem"? The constant interchangeability of the two terms is driving me nuts. Once I've nailed down what actual type of device I need, I can order one and start an actual experiment. Appreciate your replies very much - thank you for your time and patience

Great answer. That is what I was looking for : a limited privilege account. I will try this soon. Best Regards (and Merry Xmas to all)

also the Book: https://www.netgate.com/docs/pfsense/book/highavailability/index.html

If you do - don't use .1 or .254 since those are common default IPs ;) Pretty much the reason pfsense IP on all its vlans is .253...

You cannot use Proxy ARP for IPv6 because IPv6 does not use ARP. You'll have to use IP Alias. Any IPv6 on inside interfaces should be provisioned using an interface network and a routed prefix to you. Like a /48, /56 or /60. It sounds like you are trying to shoehorn a VPS service designed to run something like a cPanel or Plesk system into use with a router. You're going to meet with undesirable results in all likelihood. No. LADVD is something entirely different.

I think I may have found the issue. Both device their selves were in the NAT range tied to the single VIP. I believe the secondary box was communicating out, but any reply went back to the primary box. I found a NAT setting to map "This Firewall" to it's WAN interface address and not the VIP. That seems to have worked on both devices. I did have our upstream provider NAT all to the same public IP: VIP x.x.x.1 Device 1 x.x.x.2 Device 2 x.x.x.3 Thank you all for the help!

what is your point ? my carp setup does work. i have multiple machines in each vlan. no problem there. i can shutdown either firewall and unplug any cable without producing a mess. i had disabled pfsync in some previous tests which is why using the carp address as the gateway is required. am i expected to configure a LAN interface as the first interface and use the LAN address as the failover peer for each of the dhcp server instances ? this would be meaningful indeed. but in that case, it may be worth to drop a line in the documentation and there is little to no point in setting the same address for each dhcp instance.

@netblues Yeah you were on the money. Bound the OpenVPN client service to the CARP WAN VIP and failover/failback operates perfectly - as does all everything else. Perfect, thank you :) (better check my settings more thoroughly next time ;) )

I just updated my test vms to 2.4.4-RELEASE-p1 and from what I can tell the issue has been fixed! I now get the client-hostname on the master and the backup