Outbound NAT for Public Static IP's Behind PPPoE

kcoombs · Jul 15, 2018, 2:23 PM

Hello, I have a /29 block of public static IPs from Centurylink (e.g., x.x.x.46/29), and I'm having trouble with outbound NAT for these addresses under pfsense (2.4.3-RELEASE-p1).

First, some background for what I'm trying to accomplish. Previous to installing my pfSense firewall/router (i.e., when using the ISP modem/router) I could choose to configure a "Block of Static IP Addresses". This would result in the modem obtaining the .46 address from the block as its WAN address, the modem assigning an x.x.x.x.40/8 address to its LAN port, and the modem allowing for the routable .41-.45 addresses to be assigned to devices on the modem's LAN ports.[1] These public addresses could be mixed with the "private" addresses (e.g., 192.168.x.x addresses) on the LAN. For example, I could statically assign x.x.x.x.41 a device (or jail/VM) within the LAN, and it would be externally reachable at x.x.x.x.41, with the outbound packets appearing to come from x.x.x.x.41. This is what it might look like, with a single LAN machine being configured with all the static addresses (e.g., as jails):

I'd like to re-create this with pfSense serving as my router. I have set the ISP modem to transparent bridge mode, connected it to the pfSense WAN port, and configured pfSense to use PPPoE on the WAN interface. When doing so, the ISP hands out the .46 address as the WAN address. Similar to the modem, I've assigned a x.x.x.x.40/28 address to the pfSense LAN port as a virtual IP (I've tightened the netmask vs. the ISP modem as far as a could; pfSense won't let me assign x.x.x.x.40/29 as a VIP). This lets me statically assign the routable public addresses (i.e., the .41-.45 addresses) within my LAN, similar to when using the ISP modem exclusively. I've also created a couple other LANs on the OPT1 and OPT2 interfaces for other uses (e.g., to isolate IoT devices). This is what it looks like:

When doing this, the .41-.45 addresses are reachable from inbound packets originating from the WAN side of my router (i.e., from the Internet). However, any outbound packets from those addresses appear to come from the .46 address. This clearly seems to be a NAT issue. It's been a while since I actively troubleshooted this (so I don't remember everything I tried), but it seems to me like the solution should lie in configuring some outbound NAT rules. I've tried creating such rules (under both a hybrid and manual outbound NAT scheme), but every time I do these public addresses become unreachable. I could see it being a NAT issue, a routing issue, and/or a firewall issue. I'm a bit out of my element here, and have tried everything I can think of. I'd like any suggestions for how to get the outbound packets to appear to come from the proper 41-.45 address, rather than the .46 address. I believe a complicating factor here is that the .46 address is assigned to pppeo0, and not the actual WAN port (i.e., em0).

[1] For a thread discussing this setup, see https://forum.netgate.com/topic/86505/centurylink-pppoe-a-static-29-subnet-without-1-1-nat

johnpoz · Jul 15, 2018, 8:26 PM

You wouldn't assign the vip to your lan, you would assign it to your wan interface.. Since that is the interface that would get hit from the public side.

You then just port forward the traffic you want inbound to your rfc1918 host, or 1:1 nat. And sure your outbound nat would use the public vip address as your source IP when talking to the internet.

Not sure where you would of gotten the idea you would put that on your lan.

kcoombs · Jul 16, 2018, 11:51 PM

@johnpoz Thanks for the reply.

I put the public IPs on the LAN side because I was trying to recreate how things operated with the ISP modem, and because it "worked" for me in the sense that the public IPs were reachable from the outside and they could initiate outbound traffic (albeit with the WAN IP).

I seem to recall previously trying to assign the VIP to the WAN interface and use 1:1 NAT. I just tried again. It works for inbound traffic (and replies), but it doesn't work for traffic initiated by the internal machine. Here is what I've tried:

Remove the .40 IP alias from the LAN interface
Remove the 42 public address assignment from the internal machine, and add a 192.168.1.100 address assignment to it
In pfsense, assign an IP Alias for the .42 public address to the WAN interface
In pfsense, create a 1:1 NAT rule on the WAN interface using the .42 address for the "External subnet IP" and a 192.168.1.100 address for the "Internal IP." I left the "Destination" as any.
To avoid firewall issues in testing, in pfsense create blanket allow rules on the WAN and LAN interfaces (i.e., allow any protocol, from any address to any address with any port) and move them to the top of the respective list.

Doing this, I can reach the 192.168.1.100 machine externally using the .42 address (yay!). However, the 192.168.1.100 machine cannot initiate connections to machines outside of the LAN network (e.g., it cannot ping google.com, browse the web, etc.). Running a "traceroute google.com" from the 192.168.1.100 machine shows the packets reaching the LAN interface (192.168.1.1) and going no further.

I tried to remedy this by enabling "Hybrid Outbound NAT" (rather than automatic outbound NAT) and adding a manual rule on the WAN interface with the 192.168.1.100 address as the source network, "any" as the destination network, and the .42 address as the translation address. This doesn't seem to help. I've also tried configuring the same rule on the LAN interface. No dice.

Any suggestions on how to use 1:1 NAT, as you've suggested, and have outbound traffic work?

SteveITS · Jul 20, 2018, 3:36 PM

To get outbound traffic from a server using 1:1 NAT, we use Manual Outbound NAT.

There will be a default mapping for "192.168.1.1/24" which is for any PC on the LAN, with its NAT address set to "WAN Address."

Above that one, add a row for 192.168.1.100/32 with a NAT address of the .42 public IP for this server. This will force outbound traffic, from that server only, to use the .42 address.