Port forward egress packets not being rewritten
Struggling a little with some port forwarding. Previously, we were running pfSense 2.1.5 on a desktop PC, and all was good. We upgraded to 18.104.22.168, then migrated (via backup then restore of config) to a PCEngines board running the same.
Network looks like:
Public interwebs <–> ADSL router (which I don't have control over, so can't put into bridge mode) 192.168.0.1 <--> 192.168.0.2 PFSense 10.105.56.250 <--> 10.105.56.1 Server in question
There's a bunch of other 10.105.x.0/24 subnets on the LAN side of PFSense as well, each of which (including the .56.0/24) is a tagged VLAN on igb0; 192.168.0.2 is untagged on igb2. We're not running multi-WAN. The only gateway is 192.168.0.1
The ADSL router is set up to DMZ everything through to PFSense. This hasn't changed since everything was working pre-upgrade.
Outbound NAT from 10.105.56.1 to the internet works fine.
However, we have a port forwarding rule pointing inwards:
rdr on igb2 inet proto tcp from any to 192.168.0.2 port = 1050 -> <int_server>round-robin
And associated filter rule:
pass in log quick on igb2 reply-to (igb2 192.168.0.1) inet proto tcp from any to <int_server>port = 1050 flags S/SA keep state label "USER_RULE: NAT Test" dnpipe(2, 1)
If I mirror both igb0 and igb2 on the switch to a machine running Wireshark so I can see packets on both interfaces, I see:
a.b.c.d:56748 –> 192.168.0.2:1050 SYN - from ADSL router's MAC to igb2's MAC
a.b.c.d:56748 --> 10.105.56.1:1050 SYN - from igb0's MAC to the internal server's MAC
10.105.56.1:1050 -> a.b.c.d:56748 SYN ACK - from the server's MAC to igb0's MAC
10.105.56.1:1050 -> a.b.c.d:56748 SYN ACK - from igb2's MAC to the ADSL router's MAC
The fourth packet is where the problem shows up - pf seems to be failing to do the address translation of the outbound packet, but obviously does do the right thing on the inbound packet.
If I look at the state table at this point, it's in the half-established state:
igb2 tcp 10.105.56.1:1050 (192.168.0.2:1050) <- a.b.c.d:56748 SYN_SENT:ESTABLISHED
igb0_vlan56 tcp a.b.c.d:56748 -> 10.105.56.1:1050 ESTABLISHED:SYN_SENT
I don't think I'm doing anything crazy, but obviously it isn't wanting to play :-). I've tried rebooting, disabling the rules and re-enabling, disabling and enabling NAT Reflection (though I believe it should not apply here as I'm coming in from a WAN IP).
Any thoughts very much appreciated…
[Edited to add: I've just gone back to the original hardware, post-upgrade to 2.3.2 (not p1), and the behavior is the same, so I don't believe it's hardware-related.]
Ahh… a clue: this only seems to apply when I have a pipe selected on the filter rule.
Dropping the pipe off so the rule becomes:
pass in log quick on igb2 reply-to (igb2 192.168.0.1) inet proto tcp from any to <int_server>port = 1050 flags S/SA keep state label "USER_RULE: NAT Test"
…everything works correctly.
The limiter is configured to allow a fixed amount of bandwidth, with no schedule, no mask, and no advanced options set.
Limiters with NAT are broken everywhere but on 2.4 devel snapshots.
Thanks - just found the bug having established the connection! https://redmine.pfsense.org/issues/4326
That explains why it broke after the upgrade from 2.1.5 :-(.