Hello,

I'd recommend the following setup:

Virtual Network Adapter with a vMAC connected to pfSense WAN

Set in OVH Control Panel the same vMAC for the 4 IPs

Assuming the block purchased was 198.51.100.4/30 (198.51.100.4 - 198.51.100.7), you'd configure pfSense WAN statically with the following settings:
IP: 198.51.100.4
Mask bits: 32 (equivalent to 255.255.255.255)
Gateway: Not set

Configure LAN as suits your better, example:
IP: 10.10.10.1
Mask bits: 24 (equivalent to 255.255.255.0)
Gateway: Not set

Then add a gateway manually for the WAN (If your dedicated server is at 203.0.113.X, you'd use 203.0.113.254 as the gateway) and set the advanced option "Use non-local gateway through interface specific route" to allow gateway outside subnet.

Add the virtual IPs to your WAN: 198.51.100.5/32, 198.51.100.6/32, 198.51.100.7/32

In the past this used to be much more complicated (I've followed those tuts to a certain extent on earlier pfSense versions):
http://blog.magiksys.net/pfsense-firewall-default-gateway-different-subnet
http://magiksys.blogspot.pt/2012/12/pfsense-bridge-gateway-vmware-ovh-ip.html

Regards,
Jorge M. Oliveira

Because you didn't have one on the primary at the time. Add the rule(s) on the primary, add again on the secondary so the primary can sync to it, then sync.

Try to disable Virtual IP sync in XMLRPC (I had some issue with it in the past)
Align Advertising Frequency for VIP's in on both nodes, and manually set skew to 1 on Master and 100 on Backup.

If still not stable, the pfSense doc's recommend to increase Adv.Freq. with 1 until stable situation has achieved….

Nevermind… found my the issue.
As a workaround for a previous issue (carp pre-2.3), I had a difference in master/backup VIP's advertising frequency. And it looks v2.3.x does not like it. I aligned these again, and now it is behaving as it should. Upgrading as I write this update  8)

Hi cmb, I've had the chance to test this and it works as expected :-)

Thank you

Outside Switch.

ONT <-> Switch <-> HA WAN ports.

You don't want them doing it anyway. They'll just screw up your CARP multicasts and blame your gear (pfSense).

The only place I was ever offered a true HA solution from the get-go was a colo at www.supernap.com.

Stacking switches with a Multi-WAN on each stack member is about as good as you can get at the typical endpoint.

Almost certainly an IP conflict. Check your system logs for "xx is using my IP …", if that's the case, you may see exactly which device there.

Thanks for the clarification .

When there's no text there, that means the IP can't be found configured on the OS. Maybe it already has that 7.1 IP on it elsewhere? In which case it'd fail when trying to add that as a CARP IP, leaving you in that situation. That's just one reason that comes to mind as maybe the most likely cause. Any ifconfig errors or anything relevant in the system log?

Yeah it's probably time to post your Firewall > Virtual IPs, Firewall > NAT, Outbound screens.

And you don't have to power down the primary to test. Just temporarily disable CARP on Status > CARP for basic functionality testing.

It is technically possible to have a CARP VIP and no other IP addresses in that subnet on an interface, but it's not ideal. Only the master node has outbound connectivity so it's difficult to manage packages or updates on the secondary without some extra hoop-jumping.

Hi Adam,

Could try binding the public VIP ip's to a localhost interface.?
https://redmine.pfsense.org/issues/4026#note-1

Regards,
PiBa-NL

And because you are using VPN server on 127.0.0.1 / any listening port you want ( TCP ) it is no problem to come on wan on 443 TCP ( on any Virtual IP ) and forward to 127.0.0.1 / listening port. ( set on NAT )

This how I configured my server and depend of country/users IP I also come on other ports (443, 4343, 43434… ) that are forwarded to the same server on 127.0.0.1/43434 TCP

Can you try a 2.3.1 snapshot on one of those HA pairs? There was a fix or two for XMLRPC last week or so, would be worth trying out.

@Derelict:

So for the 1:1 NAT entry Single host is selected for Internal IP?

All of the netmasks on all the CARP VIPs on your L3 circuit should be /28. Not that it's causing this problem.

Enabling that 1:1 NAT should not stop any traffic.

How about a screen shot of the 1:1 NAT edit screen?

Ah, good eye on the /28 CARP IPs. Although, I'm attempting to reach an IP on the WAN_COX circuit (WAN_L3 isn't connected yet).

Initially I was thinking that it was an incorrect outbound NAT rule, however without the 1:1 rule enabled, the device at 192.168.4.225 has no problem reaching the internet.

Screenshot of the 1:1 edit is below.

pfsense-1-1edit.png
pfsense-1-1edit.png_thumb