1:1 NAT through Tier-2 gateway
-
Hi Team,
I have set up two WANs as a routing group [Tier-1 + Tier-2]. I have two PFsense machines set up this way, with CARP addresses shared between them.
If I set up a CARP address terminating on the Tier-1 network, then set up 1:1 NAT to a machine on that network, everything works fine, inbound and outbound. All NAT transactions happen via the CARP address, as expected. For the sake of simplicity in this experiment, the NATed machine is allowed outbound everywhere and inbound traffic is allowed inbound via ICMP and SSH.
If I set up the same machine, but terminate 1:1 NAT on the Tier-2 network, outbound NAT works fine, again via the CARP address on Tier-2, as expected. Firewalls include policy based route via the Tier-2 gateway. I can ping the machine from the outside, but I can't ssh in via the Tier-2 address.
When I run TCPDUMP on the machine, I can see packets coming in on port-22 and being returned to the client:
The state table seems to contain correct connections:
However, we see no return packets on the client machine and the SSH connection never completes. Same is true of TELNET [etc].
If I force a failover to the Tier-2 connection, everything starts working as it does for Tier-1, so it looks like whatever I'm missing is related to the NAT setup on secondary gateway ...
It 'feels' like the Policy based routing is being ignore for inbound NAT returns, even though its being honoured for outbound connections.
I must be missing something obvious here, otherwise there would be loads of questions like this here, but the only thing I see vaguely related is someone suggestion that you set outbound rule 'state type' [under 'advances' from 'Keep' to 'None'. I tried that, but no dice.
Any advice most appreciated.
Thanks ChIP.
-
@fartypants said in 1:1 NAT through Tier-2 gateway:
It 'feels' like the Policy based routing is being ignore for inbound NAT returns
It is in fact. You might have policy-routed outbound traffic, but you cannot policy-route responses on inbound traffic. It either follows the routing table or the reply-to if applicable.
So you have to rely on the reply-to for proper routing here.
To ensure, this takes effect, the incoming traffic on tier 2 interface must be allowed by a firewall rule, which is defined on this interface.
Note that there must no pass rule on an interface group or a floating rule match this traffic, since these rules have precedence over interface rules. -
Many thanks for your advice. Much appreciated.
I have a rule on the Tier-2 interface as follows:
As you can see, we're getting hits on there. Traffic is also being logged.
Reply-to is enabled, as default:
I have no floating rules or interface groups.
Sorry - I may have confused things here. In my orignial post I said 'inbound NAT', while I actually meant 'outbound'.
To be clear - I see traffic entering the firewall just fine. I also see the server machine on 192.168.92.101 receiving and returning the traffic to the correct client. Please see the tcpdump.
My problem is, the traffic never gets there, and I can't see the PFsense machine trying to send it.
Thanks again for your time.
ChIP.
-
Sorry for any confusion in my original post.
Maybe it would have been clearer if I'd said:
It 'feels' like the Policy based routing is being ignored for returns to inbound NAT, even though its being honoured for outbound connections.
ChIP.
-
@fartypants
The packet capture on the machine shows a sequence of SYN packets hitting it. So probably the SYN-ACK doesn't reach the client. A reply-to issue might be a probable reason for this.To investigate if the reply-to works probably sniff the traffic on the WAN interfaces.
I guess, the response packets are missing on the tier 2 WAN.
But do you see them on the tier 1? If so the reply-to doesn't work here. -
You are absolutely correct ... we had a reply-to issue.
The issue was cause by there not being a default gateway set on the Tier-2 interface, so it wasn't spotted as a WAN interface, so reply-to wasn't enabled.
Heaven only knows how long it had been that way, but now its set, everything works as advertised.
Thanks again for your time and effort ... much appreciated.
May the force be with you.
ChIP.