Internal Firewall traffic issues after CARP Failover

[dbennett]

Greetings,

To start, I inherited this, not exactly how I would do it.

I attached a pic that depicts a conceptual layout of the infrastructure I am working with.  Now, I will be using words like Red & Green simply to simplify the post so please don't laugh (as you read this you might.  I did  ;D ).

INFO:
Like color firewalls have CARP connection
Red firewalls provide 1:1 NAT (Public IP's –-> 172 NET)
LOAD BALANCERS provide 1:1 NAT (172 NET ---> 10 NET)
Green firewalls DO NOT provide 1:1 NAT (172 NET –-> 192 NET)
Blue firewalls provide VPN connection to 192 NET.  Virt IP's are unique to the network.
Red firewall 02 is a new addition.  Up till a month ago there was only ONE Red firewall.

The Situation:
1.  When Red Firewall 01 (MASTER) is online, traffic works fine.  No issues
2.  When Red Firewall 01 (MASTER) is put in Maintenance mode and backup takes over MASTER role, traffic works fine.  No issues.
3.  When Red Firewall 01 is rebooted/powered down, existing connections are fine but new connections from 192 NET do not have internet access.  10 NET internet access is not affected.
4.  When Red Firewall 01 powered on (still in Maintenance mode), Red Firewall 02 is still MASTER, traffic returns to normal.
5.  When a VPN connection is made, browsing to Red firewall (MASTER or Backup), log in and dashboard displayed is good, but connection times out after that.

Now, I have another infrastructure with just Red Firewalls ---> LOAD BALANCERS and failover isn't an issue.  I cross compared the configs from the site shown here to the other site and there really isn't much difference (except for the lack of the Green Firewalls).

I am assuming this has to do with the Double NAT'ing that is going on.  But I need help after that...

Please and thank you for any assistance you can give

Dino

[dbennett]

If there is the need for more info please ask.  I've posted and removed at least 4 posts because of zero comments feeling that I wasn't describing it correctly.  My hope is that the drawing helps clarify it a bit.  I'm honestly having a hard time believing that having an internal firewall without 1:1 NAT setup is so rare or impossible to setup with CARP to work correctly.

Question I keep asking myself is 'What is so different between the MASTER and the Backup firewall that the clients behind the firewall can't receive replies back from the internet?  Could it be something as simple as a firewall rule?

Thoughts:
1. Other then moving the internal firewall to the WAN (last resort), would setting up a port forward for the WAN IP associated to the internal firewall take care of this issue?  If so why does it work now?
2. Would simply setting value in System > Advanced > Firewall / NAT > Network Address Translation "NAT Reflection mode for port forwards" = NAT + proxy be the answer?
3. Is setting up DHCP values on the firewall MANDATORY for CARP to work correctly even though I will not be using pfSense DHCP?

Trace routes done during the rebooting of the MASTER shows that the ping replies stop at our gateway IP.  It's like the world only knows of one ip for our site.  I've checked with the ISP and apparently our edge router only has network segments pointing at us and not specific addresses.

Please, would appreciate any thoughts or questions.

[dotdash]

I would guess that the reason you're not getting replies is because this is the kind of thing people get paid to diagnose.
I think you'd be better off buying an incident from support than trying to get someone from the forum to invest the time and energy to dig through your problem.
My quick thought is that you have traffic passing through the Interface address on the primary firewall where is should be pointing to one of the CARP addresses. That's about as much as I can devote to this.

[dbennett]

I appreciate the honesty and I totally see your point.