Huh? No a router can not have the same networks or overlapping networks on multiple interfaces, ie its wan and lan..

But if the /29 is routed to you this would never be the case since your wan would be the transit network and wouldn't overlap with your routed /29

This has zero to do specific with pfsense - and is just basic 101 routing.

Here lets do an example...

isp .1 --- 1.2.3.0/30 --- .2 wan pfsense opt .1 --- 4.5.6.0/29 --- devices .2, .3, .4 etc..

And sure pfsense could also have lan network in 192.168.1.0/24

Now your isp routes 4.5.6.0/29 to your 1.2.3.2 address.

Okay. Noted. Thank you.

I spoke too soon, it's still not working 100% of the time.

When the backup-pfsense is entering MASTER-status, not all of the time Radius gets started correctly, sometimes I see the following in the Radius-logfile, right afster started it gets stopped again:

Mon Apr 15 14:16:21 2019 : Info: Ready to process requests
Mon Apr 15 14:16:21 2019 : Info: Signalled to terminate
Mon Apr 15 14:16:21 2019 : Info: Exiting normally

Sorry. I have no idea what you are even asking.

The basic things that need to be changed to run pfSense HA in VMware ESXi are described here:

https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-high-availability-clusters.html?highlight=esxi#hypervisor-users-especially-vmware-esx-esxi

So now everything worked fine for a little while.

But then I saw this:
ntp_status_unreachable.png

Using a shell on pfsense I can ping each NTP server and I can also use ntpdate to set the clock. So basically it's not a firewall or routing problem I think.

NTP log doesn't show anything unusual or different compared to the master firewall.

I'll keep looking.

You can probably get away with having only one proper WAN with two single-address WANs as long as:

All addresses are static (not DHCP, PPPoE, etc)

You ensure that the default gateway in the routing table is always the interface where the secondary can get out (has its own routeable interface address)

The main issue is that the secondary can access the internet (get updates, resolve DNS, etc) when it is CARP BACKUP.

Is there a quick way to copy the config from the secondary over to the replaced device?

We had a similar failure here.

Thanks, I will continue my investigation, if I have any further information or questions I will get back to this topic.

Thanks

OK, I found the problem.

https://github.com/pfsense/pfsense/blob/master/src/etc/rc.filter_synchronize

A bunch of these lines:

$config_copy['nat']['outbound']['rule'][$x]['descr'] = remove_special_characters($config_copy['nat']['outbound']['rule'][$x]['descr']);

The function remove_special_characters strips out everything but a-z, A-Z, 0-9 _ - +

Should have maybe used the function htmlspecialchars instead to get special characters encoded instead of stripping them. Also having the xml in UTF-8 allows you to put a lot of international characters in the xml file.

Anyway, it's looks like it's been working like this for years.

I was able to reproduce both of those bugs. We've hit similar things in the past.

I created new issues for them:
Limiters: https://redmine.pfsense.org/issues/9468
ALTQ Shaper: https://redmine.pfsense.org/issues/9469

All changes that are synced.

check the clocks on both nodes

A 255.255.255.252 netmask is only a /30.

Please send the actual addresses in a chat. That makes zero sense and it's impossible to help you without knowing what they actually are.

As posted in other thread...

@Derelict Thanks for the tips. I got it to work. I didn't really understand what you meant, but I agreed it seemed like a NAT issue. I found a separate thread where you said the 'NAT Addresss" should be the VIP address. So, I made sure to change all the WAN1 and WAN2 mappings to the VIP addresses. (I tried this once in the past, but I didn't think it worked. I must have not refreshed it or something)

https://forum.netgate.com/topic/119782/solved-setup-manual-outbound-nat-section-in-pfsense-docs-unclear-to-me/4

Anyway, after using the VIP addresses in the NAT mappings, it fixed the WAN1 to be online at all times.

Thanks!

@Derelict Thanks for the tips. I got it to work. I didn't really understand what you meant, but I agreed it seemed like a NAT issue. I found a separate thread where you said the 'NAT Addresss" should be the VIP address. So, I made sure to change all the WAN1 and WAN2 mappings to the VIP addresses. (I tried this once in the past, but I didn't think it worked. I must have not refreshed it or something)

https://forum.netgate.com/topic/119782/solved-setup-manual-outbound-nat-section-in-pfsense-docs-unclear-to-me/4

Anyway, after using the VIP addresses in the NAT mappings, it fixed the WAN1 to be online at all times.

Thanks!

@Derelict thank you so much, have a nice day!

Success!

Tried it out live now.
Shutdown machine #1. Branch Office lost its connection to the Main Office for about 10 seconds. This is OpenVPN reconnecting.

After OpenVPN peer-to-peer reconnected.
Resolving from AD DNS works! TIL: So, multiple domain overrides is the way to go for internal name resolution in this scenario with AD DNSs.

Goal achieved with HA, AD DNS and OpenVPN peer-to-peer.

Branch Office is not a sitting duck when Main Office is not available. Normal DNS function maintained. Branch Office AD DNSs reach-ability is kept when HA is failed over.

Kudos to all!

Brgs,