Port forward egress packets not being rewritten

  • Hi folks,

    Struggling a little with some port forwarding.  Previously, we were running pfSense 2.1.5 on a desktop PC, and all was good. We upgraded to, then migrated (via backup then restore of config) to a PCEngines board running the same.

    Network looks like:

    Public interwebs <–> ADSL router (which I don't have control over, so can't put into bridge mode) <--> PFSense <--> Server in question

    There's a bunch of other 10.105.x.0/24 subnets on the LAN side of PFSense as well, each of which (including the .56.0/24) is a tagged VLAN on igb0; is untagged on igb2. We're not running multi-WAN. The only gateway is

    The ADSL router is set up to DMZ everything through to PFSense. This hasn't changed since everything was working pre-upgrade.

    Outbound NAT from to the internet works fine.

    However, we have a port forwarding rule pointing inwards:

    pfctl -sn | grep 1050

    rdr on igb2 inet proto tcp from any to port = 1050 -> <int_server>round-robin

    And associated filter rule:

    pfctl -sr | grep 1050

    pass in log quick on igb2 reply-to (igb2 inet proto tcp from any to <int_server>port = 1050 flags S/SA keep state label "USER_RULE: NAT Test" dnpipe(2, 1)

    If I mirror both igb0 and igb2 on the switch to a machine running Wireshark so I can see packets on both interfaces, I see:

    a.b.c.d:56748 –> SYN - from ADSL router's MAC to igb2's MAC
    a.b.c.d:56748 --> SYN - from igb0's MAC to the internal server's MAC -> a.b.c.d:56748 SYN ACK - from the server's MAC to igb0's MAC -> a.b.c.d:56748 SYN ACK - from igb2's MAC to the ADSL router's MAC

    The fourth packet is where the problem shows up - pf seems to be failing to do the address translation of the outbound packet, but obviously does do the right thing on the inbound packet.

    If I look at the state table at this point, it's in the half-established state:

    pfctl -ss | grep 1050

    igb2 tcp ( <- a.b.c.d:56748      SYN_SENT:ESTABLISHED
    igb0_vlan56 tcp a.b.c.d:56748 ->      ESTABLISHED:SYN_SENT

    I don't think I'm doing anything crazy, but obviously it isn't wanting to play :-). I've tried rebooting, disabling the rules and re-enabling, disabling and enabling NAT Reflection (though I believe it should not apply here as I'm coming in from a WAN IP).

    Any thoughts very much appreciated…

    [Edited to add: I've just gone back to the original hardware, post-upgrade to 2.3.2 (not p1), and the behavior is the same, so I don't believe it's hardware-related.]


  • Ahh… a clue: this only seems to apply when I have a pipe selected on the filter rule.

    Dropping the pipe off so the rule becomes:

    pass in log quick on igb2 reply-to (igb2 inet proto tcp from any to <int_server>port = 1050 flags S/SA keep state label "USER_RULE: NAT Test"

    …everything works correctly.

    The limiter is configured to allow a fixed amount of bandwidth, with no schedule, no mask, and no advanced options set.


  • Banned

    Limiters with NAT are broken everywhere but on 2.4 devel snapshots.

  • Thanks - just found the bug having established the connection! https://redmine.pfsense.org/issues/4326

    That explains why it broke after the upgrade from 2.1.5 :-(.