Using NAT on BACKUP CARP

rwfitzy · Apr 4, 2025, 2:57 PM

I would like to use a NAT rule on my BACKUP CARP router to allow trusted sources to connect to an iSCSI drive. This works on the MASTER, but can't connect through the BACKUP router. I did a packet capture and found the responses to the request are going through the MASTER.

They both use the same network gateway of the data center so I'm assuming this is due to CARP? Or my Outbound NAT setup. I do not use a LAN per se, I have two public subnets with a hybrid NAT with NO NAT for those subnets with 4 interfaces...

VIP -> 1.x.x.100/29
WAN MASTER -> 1.x.x.101/29
WAN BACKUP -> 1.x.x.102/29
SYNC MASTER -> 2.x.x.251/24
SYNC BACKUP -> 2.x.x.252/24
SUBNET1 VIP -> 3.x.x.1/26
SUBNET1 MASTER -> 3.x.x.2/26
SUBNET1 BACKUP -> 3.x.x.3/26
SUBNET2 VIP -> 4.x.x.1/27
SUBNET2 MASTER -> 4.x.x.2/27
SUBNET2 BACKUP -> 4.x.x.3/27

My NAT rule is port forwarding 3260 to 1.x.x.20/26. How could I possibly send traffic using my NAT rule to respond directly back from the BACKUP router?

viragomann · Apr 4, 2025, 4:17 PM

@rwfitzy
Since you don't nat the upstream traffic from the inside networks, I assume, that these subnets are routed to one of your WAN IPs from outside, most probably to the CARP IP, which is occupied by the master, of course.
So traffic to them logically go to the master.

However, I don't understand, why your internal devices direct upstream traffic to the backup, but not to the CARP VIP and hence to the master.

SteveITS · Apr 4, 2025, 5:02 PM

@rwfitzy You're asking about incoming NAT rules/port forwards? Set the Dest. Address of the rule to the shared IP. Then it will work on both routers, whichever one is holding the shared IP.

rwfitzy · Apr 4, 2025, 5:59 PM

Yes, incoming NAT rule/port forward. When you say set to the shared IP, you mean the WAN CARP Virtual IP address? That doesn't seem to work either, same result.

To be clear, I want this to work through the BACKUP router even when in CARP BACKUP mode. I had the NAT rule destination set to the WAN address thinking that is what's needed instead of the WAN CARP Virtual IP. When set to the WAN address, I see the traffic going back in the packet capture on the MASTER router...

17:46:24.672688 IP 1.x.x.102.3260 > <remote ip>.56203: tcp 0

On the BACKUP router, I see the traffic coming in ...

17:46:24.885546 IP <remote ip>.56203 > 1.x.x.102.3260: tcp 0

If I set the NAT rule destination to the WAN CARP Virtual IP address, I see the same incoming on the BACKUP packet capture with nothing on the MASTER packet capture.

NOTE: I did make a mistake in my original post, the NAT port forward rule has a redirect target IP address of to 3.x.x.20/26 to SUBNET1. Also, the request is being done to 1.x.x.102, the WAN BACKUP real IP.

SteveITS · Apr 4, 2025, 6:20 PM

@rwfitzy I think you'd need to use the backup router's WAN IP .102 in the NAT rule, and then set the backup router's SUBNET1 IP as the gateway on the .20 device.

rwfitzy · Apr 4, 2025, 7:26 PM

Thanks, yeah, makes sense, but using the gateway of SUBNET1 IP 3.x.x.3/26 on the .20 server disables access to the server all together. The route to the public SUBNET1 from the data center flows to the WAN Virtual IP address. Perhaps for these reasons, I am not able to do what I'd like. I'm looking for a way to pfSense to handle the responses when the server itself is always going to do that. The server is a TrueNAS server, perhaps I'll need see if there is a way to route back to the two different routers depending on which one it came from? There is a layer 3 switch between the two as well, maybe something there could help.