Hello Derelict. finally i was able to solve the problem due to the misconfigured XMLRPC SYNC section : the two pfsense had different password to admin username . in addition , in one of the servers the ip address in XMLRPC field was blank thanks for your willing to help me with this issue

This is a UK ISP. They will not put /29 static routes on their edge equipment. That is not an option and I am aware that CARP will not work in this situation hence my question is asking about the alternative virtual IP setups. What interested me is the IP Alias alternative to CARP listed on this page: https://www.netgate.com/docs/pfsense/firewall/virtual-ip-address-feature-comparison.html It says it can be used by the firewall to bind/run services and it can be on a different subnet to the real interface IP. So to explain a bit better, I was wanting to leave the /29 and CARP set up as they are. Then I wanted to add an IP Alias with the /30 on it and the ISP default gateway. The /29 default gateway will be the /30 therefore routing all traffic correctly. As far as I can tell, this should work but I don't have the capability to test and I was hoping someone who knew pfsense better than I would be able to confirm if there are any problems with doing this before I go and break everything on a live circuit... Thanks, Colin.

Maybe you can describe your setup a bit and phrase out some detailed question? -Rico

Hi, Didn't you seen the screenshots? Everything is explained. Outgoing traffic was done by the master and incoming (reply from server) was going trough the backup. Finaly after one week of investigation - we've found the problem. In the Virtual IP defined (used after in NAT 1..1) we've specified the "WAN" interface instead of the WAN CARP interface I think it would be a great idea to put this information in the troubleshooting guide.

@derelict Understand thank you

Still not sure what you are asking. There are no loops in an HA setup. Seems like more of a question for Cisco.

I would say don't use '/' for the time being.

Allright, so that NAT's are not performed like firewall rules top to bottom Yes. they are. Your any rule at the bottom catches everything so everything has NAT performed on it. The whole point is that some things should not have NAT to the CARP VIP performed on them. In your example you are STILL natting for sources in 127.0.0.0/8, which is where traffic sourced from the firewall itself might be originating for NAT purposes, which is NOT the default configuration. If I were you I would scrap what you have start over with a simple WAN/LAN setup on a test bench, read the docs again, and get a handle on what is involved here. If NAT to the interface address doesn't work, then your /30 /28 scheme, which I said was broken out-of-the-gate, is incompatible with the upstream gear.

And this: The ISP Layer 2 device will see the CARP MAC as the source MAC in the CARP advertisements. They are sent to the Layer 2 Multicast address 01:00:5e:00:00:12 (all points multicast) to Layer 3 multicast address 224.0.0.18. That MAC address has to be added to the switch port's MAC address table based on those. This MAC address will change ports on a failover event. The ISP device must move the MAC address to the new port as any switch should. The ISP Layer 3 gear will get the CARP MAC in response to ARP "WHO HAS" requests for the CARP VIP address. Their gear needs to do the right thing with it. The ARP reply from the WAN interface that is currently CARP MASTER will contain the CARP MAC in the ARP "IS AT" response. This ARP response will be sourced from the interface IP and MAC address. The ISP Layer 3 gear also needs to honor the interface addresses that will ARP as normal. The ISP device will only ever see the interface MAC address on the port connected to that node. All references to ISP gear there should be interpreted in whatever is upstream of the two pfSense nodes in your environment. The Mikrotik and whatever else that is.

I am not sure if this is the best to go but I will increase the MTU to 9000 and see if all goes well :) That is the last thing I would do if I was in your position. I would get everything working perfectly and leave jumbo frames out of the picture.

[solved] The problem strangely solved by re-configuring System -> Routing values. Also I changed the default gw to Automatic (I doubt if this has been effective!)

It would probably be easier to take a backup of the primary, replace opt9 with opt8 in a copy of the backup, then restore the edited version.

You do not need outbound NAT on LAN at all. That is just silly. You should be able to ping both interface addresses and the CARP VIP of the connected subnet if the rules on that interface allow it. If you can ping the interface addresses but not the CARP VIP, check the ARP table of the device you are testing from to be sure it has all three ARP entries. The interface addresses should have the interface MAC address. The CARP VIP should have the CARP MAC. If that is all in place, be sure the switch connecting everything has the CARP MAC in its MAC address table. It should be on the switch port that is currently connected to the CARP MASTER node.

@derelict Sorry because my HA interface is a VLAN I forgot to added into the switch That was the problem Thank you

Well, you can add another switch between pfSense and the other switches....

@netblues : Thank you again - that's cleared up the IP address confusion (and yes I had read in the book that auto-nat wasn't supported with CARP - forgot that in the confusion with PPPoE). Our usual networking hardware supplier recommended NETGEAR DM200-100EUS ADSL/VDSL Modem to replace the ISP supplied router. Reading the manual shows it appears to have routing features. Does this qualify it as a "router device" even though it's being called a "modem"? The constant interchangeability of the two terms is driving me nuts. Once I've nailed down what actual type of device I need, I can order one and start an actual experiment. Appreciate your replies very much - thank you for your time and patience

Great answer. That is what I was looking for : a limited privilege account. I will try this soon. Best Regards (and Merry Xmas to all)

also the Book: https://www.netgate.com/docs/pfsense/book/highavailability/index.html

If you do - don't use .1 or .254 since those are common default IPs ;) Pretty much the reason pfsense IP on all its vlans is .253...

You cannot use Proxy ARP for IPv6 because IPv6 does not use ARP. You'll have to use IP Alias. Any IPv6 on inside interfaces should be provisioned using an interface network and a routed prefix to you. Like a /48, /56 or /60. It sounds like you are trying to shoehorn a VPS service designed to run something like a cPanel or Plesk system into use with a router. You're going to meet with undesirable results in all likelihood. No. LADVD is something entirely different.