@steveits Yes I need Snort on both interfaces, I like to know for example who from my family wants to download torrents.
I have Zabbix and FreeRADIUS package install along with Snort.

@joezyz
think about A Records later, first make the network work
configure it step by step
the gateway to the network will be the shared IP
it's easy only after you understand it

@bp81
Exactly. You configure the secondary WANs as private network, so that they can talk together. Then you hook up the CARP VIP on this interface on the master and add the WAN gateway in System > Routing > Gateways to this interface.

Internet access over the secondary WAN has only the router which has the master role. I.e. in case of failover to the secondary box it takes over the WAN2 CARP and gets access to the internet over WAN2.

In normal state when the secondary box is backup it can access the internet over WAN1. So WAN1 GW has to be set as default gateway.

There is also a workaround to get internet on the backup router over a single WAN connection and a single IP over the master, but that makes no sense in a Multi-WAN setup.

@binary_bandit I went with a solution roughly as explained by Mike here.. the advice came from elsewhere, but the comments were basically the same.

Have two sites both routable always, each with its own carp cluster (no pfsync across sites, not necessary for me), but only one is routed to at a time. I allow my upstream provider to route for me, but could do this myself later by enabling/disabling an IP at either site and have them route to that instead. Each site is completely independent and although they advertise 3 public ranges they both have their own native/local range of public ips too.

Really the concensus from everyone I've spoken to is to do this with switches and bgp not pfsense, which is a huge bottleneck - but it does work.

@ronrn18 I have a domain with cloudflare, that points to my wan IP. And I use haproxy to do ssl offloading of this service because its just a docker and https is not really supported.

I am not having any issues with this. I use a acme cert..

I can bounce off the proxy both internally, and externally my users are able to access it. I even share the outside 443 port being used with openvpn and have not problems.

@magnus-maximus said in VIP addresses stop working:

https://datatracker.ietf.org/doc/html/rfc3768#section-8.2

That seems to indicate what is included in the ARP IS AT response in the ARP protocol itself. It is silent about the source MAC address of the frame containing the ARP response.

8.2 pretty much describes what CARP does. The MAC address in the ARP response for a CARP VIP is always the virtual CARP MAC address.

What, exactly, is the ISP doing that is breaking things? Why are they not issuing another ARP request when they have traffic for an IP address after the ARP cache has expired?

@viktor_g

Thank you very much for your help and all details @viktor_g . I have successfully implemented the changes :)

Regards

@iptvcld
Interestingly. Didn't know that. Was assuming only the master is handing out DHCP leases and only the lease state is synced to the other node.

@iptvcld said in HA/CARP DHCP Lease Hand Out:

showing the DHCP server IP of my backup node interface IP

Normal:
https://forum.netgate.com/topic/166542/pfsense-ha-lan-interfaces-only/27

@bp81 To sync states (for a transparent changeover) you need identical interfaces, see this. Otherwise the hardware shouldn't matter so much. Presumably the secondary would be using the same memory and packages as the primary.