You can certainly use LACP and VLANs to do LAN and WAN in the lab.

Many people (me included) do not like mixing inside and outside traffic on one switch/stack. Many people (me included) do it anyway. I have not seen a recent, credible case of VLAN hopping with the exception maybe of TP-Link's VLAN1 nonsense. Even less of a reason to be concerned in the lab.

But in your case, I would probably do a lag for the outside and a lag for the inside, with two interfaces each even if they are to the same stack, and one of the add-on ports for SYNC.

Glad it's working.

Status > Interfaces is probably the best tool to use for this since it lists all of the interface elements in play in order in one place.

Any time there is an upgrade, especially across operating system versions, there is always a possibility that will happen. It doesn't always affect everyone, but you can never rely on pfsync working during a significant OS update.

Ok, I found out the following:

The described problem only occurs to VM's hosted our 2 ESX 6.5 hypervisors.

All bare metal servers will work completely fine (on-link and directly connected), only VM's are affected.

On Vmware the vSwitches are configured including the following settings:

Promiscuous mode enabled;
MAC Address changes enabled;
Forged transmits enabled;

However, I don't think this is strictly needed since the firewalls are physical devices.

Is someone aware of a required setting that is required in VMware / pfSense to get this correctly working?

Thanks anyway :)

Got an answer from OVH that CARP is not possible for their hardware dedicated servers due to network design.

I've solved this using OVH Control Panel API - https://api.ovh.com

buy some OVH failover IP's (one or subnet block ) and assign them to "master" firewall in OVH Control Panel

create identical "IP alias(es)" for OVH failover IP's attached to WAN interfaces on both "master" and "backup" firewalls.
    Yes, create identical IP Aliases - no IP conflict will ever happen.

wrote a Python script that moves above OVH failover IP's to "backup" server in case "master" firewall stops responding for let's say 10 seconds
    Script can work on backup server on any other Linux/Windows server anywhere.

Works just fine - API failover IP move takes about 50-55 seconds to finish.
So, if scripts timeout for your "master" firewall is set to 10 seconds - you are looking at max 60-65 seconds outage for your services.

Boom.

Got an answer from OVH that CARP is not possible for their hardware dedicated servers due to network design.

I've solved this using OVH Control Panel API - https://api.ovh.com

buy some OVH failover IP's (one or subnet block ) and assign them to "master" firewall in OVH Control Panel

create identical "IP alias(es)" for OVH failover IP's attached to WAN interfaces on both "master" and "backup" firewalls.
    Yes, create identical IP Aliases - no IP conflict will ever happen.

wrote a Python script that moves above OVH failover IP's to "backup" server in case "master" firewall stops responding for let's say 10 seconds
    Script can work on backup server on any other Linux/Windows server anywhere.

Works just fine - API failover IP move takes about 50-55 seconds to finish.
So, if scripts timeout for your "master" firewall is set to 10 seconds - you are looking at max 60-65 seconds outage for your services.

Boom.

No, the drives don't have to match. But you really ought to get on a somewhat recent version…

Solved! Everything else was correct except the NAT Outbound, now all the servers read the correct IP and are back on the server list.

Thank you for your answers. Everything worked using viragomann rule, in source I used "This firewall" instead of 127.0.0.0/8 and it worked anyway.

Give the servers a private address and do a 1:1 NAT ?

https://doc.pfsense.org/index.php/1:1_NAT

Firewall -> NAT -> 1:1

I, myself, was trying to do the same exact thing.  I found this:

https://b3n.org/pfsense-firewall-ha-failover-cluster/

It worked for me.

I did it with my first machine being hardware and my second being virtual with a managed switch.

Just trying to help here.

You can manipulate the skew to achieve this,
pretty sure you will have to disable Virtual IPs from the HA config sync, other wise the sync will make sure the secondary is larger then the primary.

also: I am not advocating you do this, I have never tested this sort of config. Not sure if any strangeness might occur.

What is the nature of the failure? Is it an interface down, as in no carrier, or something else?

Failover is all or nothing. If an interface fails on the primary, it demotes ALL CARP on that node and the secondary takes over.

There is no "this is active on one node and this is active on the other."

Excellent.

In that configuration the server is running on both nodes all the time. Whichever holds the CARP VIP gets the traffic from the clients.

You can also bind the openvpn server to the CARP VIP (select that instead of WAN in the server config). That makes the server die on the BACKUP node and start on the MASTER node.

I like the port forward technique because it results in fewer things that have to happen on a failover event. Especially as the number of server processes goes up.

Thanks for your answer.

I've added a line in /etc/rc.carpmaster to execute my script and it's working well but I notice that /etc/rc.carpmaster was overwritten during last update.

The second method could be more stable but I'm not familiar with pfSense package, is there any doc about package structure?