1.2.3 - 1:1 NAT outgoing reply uses internal IP

tata_tulen · Mar 10, 2011, 3:24 PM

Hi all,

at first, I'm sorry for long description of the problem, I'm trying to be as accurate and clear as I can… :)

I've just met very strange problem with pfSense 1.2.3 - we have several pfSense boxes on network and all fo them suffer by the following problem:

We have multiple both WAN and LAN ifaces (in separate VLANs) on all pfSense boxes. When I try to access some server NATed on pfSense from host in one of the WAN subnets, the reply from that server DOES NOT come from external NATed address as expected, but it DOES come from the internal address of that server. This problem occures ONLY IF the communication is between the pfSense and hosts in WAN subnets - from other public hosts on Internet the communication works perfectly (this is the most stranges part of this issue…)

Let's show it:

Below is the tcpdump from one of the pfSense boxes (with WAN IP 93.99.x.y assigned to vlan0 iface) - let say it's PF#1. This dump shows the packets just after the 'ftp 93.99.a.b' command on client behind this PF#1, where '93.99.a.b' is the CARP VIP on another pfSense (let say PF#2). This VIP is 1:1 NATed to internal server 10.0.1.6 - this server is not on default LAN subnet but on additional internal subnet of that second PF#2.


# tcpdump -n -i vlan0 'tcp port 21'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan0, link-type EN10MB (Ethernet), capture size 96 bytes
15:04:11.299899 IP 93.99.x.y.10224 > 93.99.a.b.ftp: S 1812921410:1812921410(0) win 65228 <mss 0="" 1460,nop,wscale="" 4,sackok,time9="">15:04:11.300733 IP 10.0.1.6.ftp > 93.99.x.y.10224: S 2910489148:2910489148(0) ack 1812921411 win 5792 <mss 5="" 1460,sackok,timestamp="" 103,nop,wscale="">15:04:14.290744 IP 93.99.x.y.10224 > 93.99.a.b.ftp: S 1812921410:1812921410(0) win 65228 <mss 0="" 1460,nop,wscale="" 4,sackok,time9="">15:04:14.291089 IP 10.0.1.6.ftp > 93.99.x.y.10224: S 2910489148:2910489148(0) ack 1812921411 win 5792 <mss 5="" 1460,sackok,timestamp="" 103,nop,wscale="">15:04:15.561378 IP 10.0.1.6.ftp > 93.99.x.y.10224: S 2910489148:2910489148(0) ack 1812921411 win 5792 <mss 5="" 1460,sackok,timestamp="" 103,nop,wscale="">15:04:22.069709 IP 10.0.1.6.ftp > 93.99.x.y.10224: S 2910489148:2910489148(0) ack 1812921411 win 5792 <mss 5="" 1460,sackok,timestamp="" 103,nop,wscale="">15:04:23.269744 IP 10.0.1.6.ftp > 93.99.x.y.37474: S 2374844344:2374844344(0) ack 468714409 win 5792 <mss 5="" 1460,sackok,timestamp="" 1038nop,wscale="">15:04:34.870294 IP 10.0.1.6.ftp > 93.99.x.y.10224: S 2910489148:2910489148(0) ack 1812921411 win 5792</mss></mss></mss></mss></mss></mss></mss>

As you can see, the initial packet is sent to the external VIP address on the second PF#2, however the reply comes from the internal IP of server NATed behind PF#2… Why ???

I'm not able to understand it as I'm not so network-skilled. Is here anybody that can explain this strange NAT behaviour of pfSense?

Thanks!

Update: the outgoing traffic from that NATed server is correct, the issue matches just reply packets...

jimp · Mar 10, 2011, 5:12 PM

That's a known issue on 1.2.3. It was fixed somewhat recently on 2.0.

http://redmine.pfsense.org/issues/958

tata_tulen · Mar 11, 2011, 7:34 AM

I see, that's silly :(

Thank you, jimp!