Understood Thanks for taking time to respond.

@derelict nope, the only warning which i got was one of the pfblockerng package. There was a faulty url in one ASN rule i have installed for another purpose...

The number of CARP VIPs and the number of interfaces are completely unrelated problems, but thanks for the note.

Sounds like whatever is between those two nodes on those interfaces is not passing the CARP advertisements properly.

Apologize for that. I have seen so many examples of setting up carp with VM's it didn't cross my mind about promiscuous mode.

Hey @jimp, thanks for your answer! In the secondary I'm injecting what I believe are the only required (counterpart/secondary) HA settings: [image: 1549965184784-screenshot-2019-02-12-at-10.51.54.png] Are these sufficient? The only changes I can see after hitting sync on the webUI are: Replacing the bcrypt-hashes for those in the master. This might be important? Removing ipsec, aliases, wol and openvpn, empty fields. Adding all the vips as they're defined in the master. Also tried rebooting them both (it's actually one of the steps I defined for the deploy process to catch up with the synthetic config) but no luck. However, as you point out, running rc.filter_synchronize did work for me - I can just include it as an additional action over SSH for the master node. Nice! Was looking for something just like that.

All that matters is that they are syncing and are mostly identical.

Well if the identifier doesn't match the address used it will fail to match in IPsec. It needs to be built with everything referencing the CARP VIP (or some other common identifier, like an FQDN that resolves to the CARP VIP.) If this is IKEv2 with a server certificate then the CN and SAN there needs to match whatever the client thinks it is connecting to or server certificate validation will fail.

@viragomann said in Slow IPsec / internet when using CARP: Have you configured the Outbound NAT to use the WAN CARP VIP? YEP

Why would sync interface traffic ever have to go out the WAN? Yes, outbound NAT with source any is almost never right - especially to a CARP VIP. Traffic from Localhost should NAT to the interface address Traffic from inside hosts should: Use the local interface CARP VIP as their default gateway Have outbound NAT to the WAN CARP VIP set. Traffic from the sync interface should never need internet access.

I will try the setup in that page. In addition, i have no idea whether my original setup is possible or not, may i have suggestion on it? Thank you for your assistance.

So, in short, the 'seamless, state-table sync' functionality of CARP simply isn't going to work with computers using a VPN service? I'm willing to accept that. I just wanted to be sure. I suppose one solution is to use the VPN apps (installed on the computers) as an alternative for those computers which must not loose connection on a fail over. I can't think of any that would fall into that category at the moment, but I might test it to know if it's an option. (Only drawback is that you use up a connection for a single computer, instead of many. Not a big deal, now that they give you 6 or so for $4 a month.

Two things I see: Upstream is not responding at all when sourced from .164. Did you filter that packet capture on icmp? I would expect to see ARP or something there if not. The replies to pings sourced from .163 should be destined to the CARP MAC address, not the interface MAC address. It looks like something upstream does not like moving MAC addresses around like CARP does but just a guess at this point. The ISP Layer 2 device will see the CARP MAC as the source MAC in the CARP advertisements. They are sent to the Layer 2 Multicast address 01:00:5e:00:00:12 (all points multicast) to Layer 3 multicast address 224.0.0.18. That MAC address has to be added to the switch port's MAC address table based on those. This MAC address will change ports on a failover event. The ISP device must move the MAC address to the new port as any switch should. The ISP Layer 3 gear will get the CARP MAC in the "IS AT" response to ARP "WHO HAS" requests for the CARP VIP address. Their gear needs to do the right thing with it. The ARP reply from the WAN interface that is currently CARP MASTER will contain the CARP MAC in the ARP "IS AT" response. This ARP response will be sourced from the interface IP and MAC address. The ISP Layer 3 gear also needs to honor the interface addresses that will ARP as normal. The ISP device will only ever see the interface MAC address on the port connected to that node.

No. DNS will return every A record for the fqdn but you need a specific one. You could do it if your DNS was off the firewall using something like BIND views. Queries for firewall-b.example.com return 192.168.1.3 if received from 192.168.1.0/24 Queries for firewall-b.example.com return 192.168.2.3 if received from 192.168.2.0/24 Etc. But that seems like a lot of work when this is why people manage these things from a specific network.

@jimp Killing me softly with these words :)

Hello Derelict. finally i was able to solve the problem due to the misconfigured XMLRPC SYNC section : the two pfsense had different password to admin username . in addition , in one of the servers the ip address in XMLRPC field was blank thanks for your willing to help me with this issue

This is a UK ISP. They will not put /29 static routes on their edge equipment. That is not an option and I am aware that CARP will not work in this situation hence my question is asking about the alternative virtual IP setups. What interested me is the IP Alias alternative to CARP listed on this page: https://www.netgate.com/docs/pfsense/firewall/virtual-ip-address-feature-comparison.html It says it can be used by the firewall to bind/run services and it can be on a different subnet to the real interface IP. So to explain a bit better, I was wanting to leave the /29 and CARP set up as they are. Then I wanted to add an IP Alias with the /30 on it and the ISP default gateway. The /29 default gateway will be the /30 therefore routing all traffic correctly. As far as I can tell, this should work but I don't have the capability to test and I was hoping someone who knew pfsense better than I would be able to confirm if there are any problems with doing this before I go and break everything on a live circuit... Thanks, Colin.

Maybe you can describe your setup a bit and phrase out some detailed question? -Rico

Hi, Didn't you seen the screenshots? Everything is explained. Outgoing traffic was done by the master and incoming (reply from server) was going trough the backup. Finaly after one week of investigation - we've found the problem. In the Virtual IP defined (used after in NAT 1..1) we've specified the "WAN" interface instead of the WAN CARP interface I think it would be a great idea to put this information in the troubleshooting guide.