Hi,

since pfSense 2.2 3 IPs in one subnet are no longer necessary:
https://doc.pfsense.org/index.php/2.2_New_Features_and_Changes#CARP
https://doc.pfsense.org/index.php/High_Availability#Common_Requirements

However, I've never tried and there are some limitations:
https://forum.pfsense.org/index.php?topic=87546.0

What is the gateway for this /29

.104 would be the wire or network, so you would .105 through .110 as viable hosts with .111 being broadcast.  So they gave you 5 of the six viable address is .110 the gateway?

Changed the Interfaces under ESX into promisious mode.
I left NAT still disabled and no changes into firewall rules.
After a reboot the tunnel came up from the CARP address.
Now syncing the tunnel configuration to the second node, thanks for the hint wikidd :)

i can continue testing and look how stable it will be.

IP aliases respond to ARP.

Again: https://doc.pfsense.org/index.php/What_are_Virtual_IP_Addresses

I have a public IP assigned to a server in the LAN (other IPv4 range / subnet as the WAN).

Is this a routed subnet?

In that case the HA comes from your ISP routing the subnet to the CARP address on WAN.

Then you need three of the public addresses on the inside (but publically addressed) interface.  One for each HA unit and one for CARP.  Then the other hosts on the publicly-addressed segment use the CARP IP as their default gateway.

I don't see any need for VIPs other than the CARP VIPs.

Maybe draw up a diagram if I'm misunderstanding what you're trying to do.

Reboot of server did the job.
ISP did not release the subnet untill a restart of WAN via PPPoE.

Sometimes you need a second set of virtual eyes :)

I changed the VHID (still waiting for the Data Centre to assign/confirm a VHID I can use) and so far it seems stable.

You would think I would remember this from the last time we had a similar unstable connection which turned out to be the same problem.

Thanks for the assistance.

Is that IP address only configured as a CARP VIP on both nodes?

Is the CARP VIP status correct on both (Primary shows MASTER, secondary shows BACKUP)?

I did try just bringing the interfaces up/down with ifconfig, but this didn't seem to work correctly…
I also tried bringing down the physical interface rather than the ppp interface, but that just caused pppoe to stall and never reconnect.

There's options in the webui to connect and disconnect a ppp interface, is there some way to trigger this from the cli?

Sorted… Had to set the VIPs interface to Localhost for whatever reason.

There are several problems with that:

HA nodes with CARP must have identical interface setups. You can't have three different ISPs across two nodes and have it work properly. Failover signaling happens via CARP VIPs not the sync interface and those VIPs decide to fail over based on multicast heartbeats on each segment with a CARP VIP (e.g. LAN) Using HA for "Multi-WAN" is not viable. There is no way to signal node failover based on a WAN failure.

For proper HA, all nodes must be connected to all the same ISPs, though that isn't always possible, without that you can't have a setup that will cover both HA and WAN failover.

I have migrated to IKEv2 because of the strongswan's L2TP implementation does not work for the clients behind their firewall.

IKEv2 works with CARP without any problem!

Best regards
yarick123

In fact the behaviour is totally erratic, sometimes I can get full MASTER on master node and BACKUP on slave nodes, but then some of my servers are unable to reach the network…

Other time i delete a Virtual IP and recreate it and i can reach my server again but the interface is MASTER on both nodes...

My last option is to change vSwitchs options as said in troubleshooting (https://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting) but I can't do it right away.

Here is the ifconfig output with CARP re-configured. We'll see how long it lasts before the LAN interface hangs…

sk0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500         options=8000b <rxcsum,txcsum,vlan_mtu,linkstate>ether 00:90:7f:3e:38:29         inet6 fe80::290:7fff:fe3e:3829%sk0 prefixlen 64 scopeid 0x1         inet a.b.c.243 netmask 0xffffff00 broadcast a.b.c.255         inet a.b.c.241 netmask 0xffffff00 broadcast a.b.c.255 vhid 2         inet a.b.c.242 netmask 0xffffff00 broadcast a.b.c.255 vhid 3         inet a.b.c.240 netmask 0xffffff00 broadcast a.b.c.255 vhid 1         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)         status: active         carp: MASTER vhid 2 advbase 1 advskew 0         carp: MASTER vhid 3 advbase 1 advskew 0         carp: MASTER vhid 1 advbase 1 advskew 0 sk1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500         options=80009 <rxcsum,vlan_mtu,linkstate>ether 00:90:7f:3e:38:28         inet6 fe80::290:7fff:fe3e:3828%sk1 prefixlen 64 scopeid 0x2         inet e.f.g.4 netmask 0xffffff00 broadcast e.f.g.255         inet6 2001:5a8:4:70a0::4 prefixlen 64         inet e.f.g.1 netmask 0xffffff00 broadcast e.f.g.255 vhid 10         inet6 2001:5a8:4:70a0::1 prefixlen 64         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)         status: active         carp: MASTER vhid 10 advbase 1 advskew 0 sk2: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500         options=8000b <rxcsum,txcsum,vlan_mtu,linkstate>ether 00:90:7f:3e:38:27         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (none)         status: no carrier sk3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500         options=8000b <rxcsum,txcsum,vlan_mtu,linkstate>ether 00:90:7f:3e:38:26         inet6 fe80::290:7fff:fe3e:3826%sk3 prefixlen 64 scopeid 0x4         inet 172.16.4.1 netmask 0xffffff00 broadcast 172.16.4.255         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (none)         status: no carrier pflog0: flags=100 <promisc>metric 0 mtu 33172 pfsync0: flags=0<> metric 0 mtu 1500         syncpeer: 224.0.0.240 maxupd: 128 defer: on         syncok: 1 enc0: flags=0<> metric 0 mtu 1536         nd6 options=21 <performnud,auto_linklocal>lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384         options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000         inet6 ::1 prefixlen 128         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8         nd6 options=21 <performnud,auto_linklocal>sk1_vlan192: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500         ether 00:90:7f:3e:38:28         inet6 fe80::290:7fff:fe3e:3828%sk1_vlan192 prefixlen 64 scopeid 0x9         inet 192.168.42.4 netmask 0xffffff00 broadcast 192.168.42.255         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)         status: active         vlan: 192 vlanpcp: 0 parent interface: sk1 gif0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280         tunnel inet a.b.c.240 --> 208.201.234.221         inet6 2001:5a8:0:1::e15 --> 2001:5a8:0:1::e14 prefixlen 128         inet6 fe80::290:7fff:fe3e:3829%gif0 prefixlen 64 scopeid 0xa         nd6 options=21 <performnud,auto_linklocal>bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500         ether 02:08:2c:92:bd:00         nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15         maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0         member: ovpns2 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 13 priority 128 path cost 2000000         member: sk1 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 2 priority 128 path cost 55 ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500         options=80000 <linkstate>inet6 fe80::290:7fff:fe3e:3829%ovpns1 prefixlen 64 scopeid 0xc         inet i.j.k.17 --> i.j.k.18 netmask 0xfffffff0         inet6 2001:5a8:4:70a0::43:1 prefixlen 64         nd6 options=21 <performnud,auto_linklocal>Opened by PID 34443 ovpns2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500         options=80000 <linkstate>ether 00:bd:2f:e2:00:02         inet6 fe80::2bd:2fff:fee2:2%ovpns2 prefixlen 64 scopeid 0xd         nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect         status: active         Opened by PID 20343</performnud,auto_linklocal></linkstate></up,broadcast,running,promisc,simplex,multicast></performnud,auto_linklocal></linkstate></up,pointopoint,running,multicast></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></up,pointopoint,running,multicast></full-duplex></performnud,auto_linklocal></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></performnud,auto_linklocal></promisc></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,linkstate></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,linkstate></broadcast,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,vlan_mtu,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,linkstate></up,broadcast,running,promisc,simplex,multicast>

No. There is no way to change that.

I just noticed this and put a fix in for it yesterday: https://redmine.pfsense.org/issues/4903

Select Network and type in the alias in the address field below.