CARP Setup with Multiple WAN IP's

zatco · Feb 6, 2019, 9:03 PM

Hello,

I currently have PFSense WAN interface setup with static ip from ISP and Gateway.

I would like to setup a second PFSense with CARP for fail-over purposes. WAN connection on second IP will come from internet provider Fiber box with a second ISP issued Ip address.

Questions are

Do I need to setup a WAN CARP virtual IP
If I do need to set this up, would I need to receive a third IP address issued by my ISP to use for the carp Virtual IP or can a random IP in the same network be used to allow communication?

Thanks

Derelict · Feb 6, 2019, 11:39 PM

I'd start here:

https://docs.netgate.com/pfsense/en/latest/book/highavailability/index.html

zatco · Feb 12, 2019, 2:20 PM

Thanks, I went through the steps and everything from CARP heartbeats to Syncing is working correctly.

Issue I am running into now is clients do not route traffic through the LAN VIP as the default gateway. NAT Rules have been set Source is 10.40.0.0/16 to NAT CARP 10.2.1.70.

If I go back to automatic NAT rules, I can get network connectivity using the ip on Firewall1 as the gateway.

I am not using DHCP, and have tried the DNS as the LANVIP, 8.8.8.8, and secondary firewall IP.

Any direction you can point me to what the issue may be would be appreciated. Im stumped now.

Setup is:

VIP WAN: 10.2.1.70
VIP LAN: 10.40.20.3

Firewall1
WAN: 10.2.1.71
LAN: 10.40.20.1
SYNC: 10.41.20.1

Firewall2
WAN: 10.2.1.72
LAN: 10.40.20.2
SYNC: 10.41.20.2

Derelict · Feb 12, 2019, 4:07 PM

You have to tell the clients to use the CARP VIP as the default gateway. this is just like any other default gateway setting on the client. It can be static or come from DHCP.

You have to change outbound NAT so traffic from the LAN subnet is NAT to the CARP VIP instead of the WAN address.

zatco · Feb 12, 2019, 4:24 PM

Thanks for the reply. I currently have NAT rules setup to forward to the WAN CARP IP.

I have set a static IP with the LAN VIP as the default gateway. I am unable to ping the VIP on the LAN and WAN, but have ICMP traffic allowed.

Is there something I need to do in addition to make the VIP's reachable?

Derelict · Feb 12, 2019, 4:25 PM

No. Not unless your switch is simply not allowing the traffic. CARP issues are almost always the switch/Layer 2.

zatco · Feb 12, 2019, 8:19 PM

Thanks for the help. I figured it out, I had to enable Promiscuous Mode on the VSwitch. Once I did that everything worked great.

Under Hypervisor Users (VMWare ESX/ESXi)
https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-high-availability-clusters.html

Derelict · Feb 12, 2019, 8:21 PM

Yeah. Telling us it was virtual in the beginning would have helped.

zatco · Feb 12, 2019, 8:25 PM

Apologize for that. I have seen so many examples of setting up carp with VM's it didn't cross my mind about promiscuous mode.