@spectre-988 The clients use for DNS, what you tell them to use. Enter the CARP IP as DNS server, and the will send request to it. If they are configured by DHCP, tell the DHCP to send the CARP IP for DNS. In pfSense DHCP server you can enter it at "DNS servers".

Well, it can be done, with minimal changes. You need to change local ip's and make ha ones as vip Not a big thing But, do keep in mind that all interfaces have to be created in the same order in both ha instances. You will need some experience with the ha setup. Many things can go wrong if you don't know what you are doing. (as is usually the case too) I strongly suggest to setup a lab and experiment with ha setup. When you will feel confident, you can proceed with the real thing. Doing such chores on a live system without prior experience will probably cause significant downtime.

@dara said in pfSense CARP + Cisco N5k vPC: @philippe-richard Hi Philippe, Thanks a lot. This is more complete and interesting than our setup. I wonder how you configured the connection between the routers and switches? In my setup, each router has a single connection to a single switch configured as an Orphan port. For now it is working perfectly. I am not sure however how it will handle different link and device failure scenarios but I will test it sometime soon and post my findings here. Hello, have you made progress on your configuration? Have a good day

Could someone post an example for the necessary NAT rule(s), please? EDIT: got it already, at least I think so

@neilewing When an interface with a CARP VIP loses carrier, all VIPs on that host are demoted. This makes the VIPs on the other node "better" and the rest of the VIPs on the first node swing to BACKUP status (because they see the "better" advertisements) and the ones on the backup node assume MASTER (because they see that they are the "best" VIP status).

@viragomann We indeed had very strange routing issues on the location the pfSense instances are deployed. It's really nothing wrong with them but we had a strange situation in combination with our WAN Switches and the LACP upstream to the provider. OpenVPN to the CARP Address is now running stable.

@derelict IT WORKS! Thank you

@brunoroza If that is really the case then your switch is likely not properly passing the CARP advertisements. They are multicast to 224.0.0.18. 20:17:32.490656 IP 172.25.228.18 > 224.0.0.18: CARPv2-advertise 36: vhid=228 advbase=1 advskew=0 authlen=7 counter=2770184658337638700 If those are not received by the secondary node, it will also become MASTER and begin advertising its CARP VIP.

Update: after turning the whole infrastructure from left to right we found the solution. It's the limiter bug that is already known. After removing the limiter from the firewall rule (it was a just one catch all rule for the whole NAT traffic), it works as before. Which also means: the same setting worked perfectly fine before the upgrade. I am some much hoping for a soon fix of the limiters in an official update or release!

@m4rek11 After applying the patches, I did not notice that the routers changed the roles of Master-> Backup, Backup-> Master. All the problems went with those when I made any changes to the rules, dns or DHCP. I found my configuration error early. For unknown reason, for 2 different networks I sent the same vhid for Virtual IP. But the problems were still there. After applying the patches, the problem was gone.

@digger30 Perfect! Glad I could be of assistance.

The maintenance mode switch is in the config and persists across reboots.

@chrullrich I replaced the pfSense 2.6 "local router/firewall"s in my test setup with OPNsense 22.1 (this is FreeBSD 13.0 instead of pfSense 2.6's 12.3) to get a second opinion. The behavior is the same: As soon as the CARP failover happens, everything sent towards the "Internet" goes out the default route with the NATed source address appropriate for the policy route. When I tried it the first time today I thought I saw ping (and only ping) work correctly, but now I cannot reproduce it. I probably just saw what I wanted to see.