High Avail secondary node IPs - How to find it

SteveITS

@michmoor I was going to say, disconnect the primary and log in using the shared IP. :)

Package installs do not automatically install on the secondary. Package config may sync if the package supports it.

michmoor

@SteveITS Full disclousre. This is my second time working on a HA configuration on pfsense so the nuances I'm not familiar with namely package sync.

So on the standby node i have to manual install the same set of packages? Thats no problem i just wanted to be clear.

And get this....The secondary had the pfSync interface using WAN while the primary had the pfSync interface using the dedicated interface labeled HA

So clearly this can work but its so weird.

SteveITS

@michmoor yeah…agree with your label.

https://docs.netgate.com/pfsense/en/latest/highavailability/settings.html

There is state sync and one-way config sync. The latter I’d think could be allowed on WAN by firewall rules, but why???

Yes install packages yourself.

michmoor

@SteveITS
One last thing. Im using IPsec here and I'm testing failover. So far it does failover to the Backup node but i noticed the firewall rules are not syncing to my VTI interface.

According to the documentation i must create the interface manually which i understand but shouldn't the firewall rules come over as well?

viragomann

@michmoor
Yes, presumed you have configured the VTI interface on the backup node as well.

michmoor

@viragomann yep. Same interface assignment

michmoor

Im really not understanding why my VTI interfaces are not syncing up.
The interface assignments are the exact same

Master

Backup

Yet the only interface with firewall rules not syncing is the VTI. Very strange behavior.

stephenw10

Is it ipsec1 on the primary too or ipsec0?

Yeah sync can work between any interfaces but it's much better to use a dedicated sync interface if you can.

michmoor

@stephenw10 ipsec1 on both. I shared pictures above

stephenw10

Is the IPSec filter option set to VTI mode on both nodes?

michmoor

@stephenw10 Yep it sure is.

stephenw10

Hmm, well it's possible to set individual rules to not sync. That might have been done for some reason.

michmoor

@stephenw10

I thought the same as well but the "No XMLRPC Sync" checkbox is unchecked.

Logs show no errors. I even tried re-creating the interface and still no luck

stephenw10

No error on the Secondary either?

michmoor

@stephenw10

Figured it out.
The primary has OPT3 - IPsec1
The secondary has OPT4 - IPsec1

I was just looking at the IPsec1 identifier but clearly the OPT has to match. So i deleted the interface on the secondary and made sure that when it got recreated the OPT identifier matched. Once that's done Rules synced across.

The last thing I'm dealing with now is IPsec failover is very slow. An outage of over 3-4 minutes. CARP WAN address is being used but i do notice in my logs that the firewall keeps trying to initiate a connection using the WAN address and not the WAN CARP address. Is this normal?

Eventually, the firewalls speak and all is well.

stephenw10

What are they connected to?

If the IPSec is on the CARP VIP at this end the secondary should start trying to connect as soon as it fails over but the remote side will have to time-out before starts trying.

michmoor

@stephenw10 Remote side is me. How do i speed up the cutover? Is there a way?

stephenw10

So remote is a mobile IPSec client?

You can change the dpd settings for the client so it times out faster.

michmoor

@stephenw10 This is an IPsec site2site

stephenw10

Ok well if the other side is pfSense the default values are 10s and 5 failure so you can change that to, say, 5s and 3 failures.