CARP Backup pfSense : no internet for LAN computers

[rouggy]

Hi all,

I have the same problem which was exposed here : https://forum.pfsense.org/index.php?topic=51610.0
Despite following the answer i still do have the same issue which is that when my master pfsense goes down the second is going in master state but i have no internet access from any of the clients on the LAN while if the master pfsense goes back online everything comes back to normal.

My setup is two pfsense 1 & 2 on VM on two different ESXi servers

PFS1
WAN=192.168.0.12
LAN=192.168.1.1
SYNC=192.168.2.1

PFS2
WAN=192.168.0.13
LAN=192.168.1.2
SYNC=192.168.2.2

Virtual
WAN=192.168.0.2
LAN=192.168.1.7

My DHCP server is on on the pfSense with the gateway as 192.168.1.7 and DNS server 192.168.1.7

Even after renewing a lease on client still no chance to get internet and an ipconfig /all gives me :

• Gateway: 192.168.1.7
• DHCP server: 192.168.1.1 (strange cause this IP is the ip of the pfSense1 which is down for testing purpose)
• DNS 1: 192.168.1.7
• DNS 2: 192.168.1.11 (which is windows 2012 server with DNS server)

Any help would be greately appreciated

Thanks a lot

[Derelict]

Did you adjust outbound NAT so users are mapped to the WAN CARP VIP (192.168.0.2) when they make connections outbound?

when my master pfsense goes down the second is going in master state but i have no internet access from any of the clients on the LAN

"no internet access" is not very descriptive. What exactly fails?
Can they not resolve names?
Can they not ping/access 192.168.1.7
Can they not ping 8.8.8.8?

[rouggy]

Thanks for your reply !
Sorry for not giving enough information.

When i turn off the master pfsense, clients :

• Cannot resolve names
• Can ping 192.168.1.7
• Cannot ping 8.8.8.8

I confirm i have my outbound nat rule changed and mapped correctly to 192.168.0.2

Thanks for your help

[Derelict]

You are positive the clients' default gateways are set to 192.168.1.7? Check on the client, regardless of the DHCP setting.

This generally just works. What you have wrong is really anyone's guess. Sounds like there might not be proper XMLRPC sync to the secondary and it doesn't have all the necessary firewall rules, NAT, etc.

Or the client default gateway is wrong as above.

[rouggy]

Dear Derelict,

I am positive the client is using the correct gateway 192.168.1.7, this is also the first DNS ip.

Sync is working ok between firewalls.
I will make some screenshot tonight from home.

I have been looking to solve this for weeks and unfortunately could not find the issue :(

[Derelict]

Perhaps the upstream has a problem with the MAC switching? What's upstream? A switch? Some cheesy ISP device?

[rouggy]

I am using Intel dual ethernet in the machine and the switch is a Cisco SG300 10 ports.
I may try to start from scratch with two new pfsense and see if it works.

[Derelict]

What I am really talking about is if your WAN IP address is 192.168.0.2, there is something upstream that is actually doing the internet access and NAT for your HA cluster.

Going to probably take a manual failover and some packet captures to see where the traffic flow is actually failing.

[rouggy]

Oh yes sorry it is from my internet provider a Humax HG100RE DOCSIS modem/router.
I cant put it in bridge mode so i have configured DMZ to 192.168.0.2 for inbound and no DHCP.
Not sure how i can see if it is the problem or not but i can ping 8.8.8.8 from the slave pfsense while the master is down.

I have just sent you PM with the screenshots.

[rouggy]

One thing, my HUMAX modem router shows:

DHCP Clients

MAC Address IP Address Duration Expires
xx:xx:xx:xx:xx:xx 192.168.0.2 D:– H:-- M:-- S:-- STATIC IP
xx:xx:xx:xx:xx:xx 192.168.0.13 D:-- H:-- M:-- S:-- STATIC IP

I do see the Virtual CARP WAN IP
Also the backup pfsense wan IP but not the master pfsense.
Which is strange but anyway...not sure this has anything to do.

[Derelict]

Why are you obfuscating MAC addresses?

MAC addresses can be important to CARP troubleshooting.

You're hindering the help we can provide.

[rouggy]

Here we go

00:50:56:8E:26:D6 192.168.0.2 D:– H:-- M:-- S:-- STATIC IP
00:50:56:8E:9A:AC 192.168.0.13 D:-- H:-- M:-- S:-- STATIC IP

[rouggy]

Ok i found out the following maybe it can help ?!?

If i turn off my pfsense1 (master) and then change the outgoing NAT rule in pfsense2 from translation address 192.168.0.2 to Interface address, internet will be back on client and if then i changed it back to 192.168.0.2 i still have internet as well.

Turning on pfsense1 at this point will work as well and failover is working as pfsense1 becomes again the master.

If i turn off again pfsense1 i am back to square 1 and lose internet connectivity :(

[Derelict]

You really need to look at the WAN side and be sure there isn't something weird going on, like the switch not moving the CARP VIP from one switchport to another.  The CARP MAC address (00-00-5E-00-01-VHID) needs to be able to move from primary WAN to secondary WAN and back freely.

[rouggy]

Thanks for your help !
I finally got it to work but honestly not really sure what was the issue.

On my pfsense2 i changed the LAN ip and the WAN IP.

In the NAT rule i changed several times back and forth the translation address from interface address to 192.168.0.2, rebooted the Humax modem and it worked finally.
When i turn off the pfsense1 i will keep having internet with pfsense2, when pfsense 1 is back online it is still working as well…

I think this is solved.

Thanks a lot for your help and sorry i bothered you with this !