NAT with WAN, LAN, and DMZ
I'm a pfSense newbe, but I do know networking. That probably makes me dangerous as I think I know what I want, but then maybe pfSense doesn't do it that way.
Currently I have one connection provider, one pfSense box, a 3-bit subnet of static IPs, one aging server that's multi-homed, and two new servers that aren't fully functional yet. I've been connecting through an ancient SonicWall, which the new pfSense box is to replace.
I watched this video by PC Pickup on YouTube and it was hugely helpful, but I'm not fully there yet. Part of the problem is that I want a 3-way split of my networks–I want a LAN running simple NAT to/from the WAN with no port forwarding, and a DMZ that could be bridged, 1-1 NATed, or port forwarded to/from the WAN.
I swapped out the pfSense box for my SonicWall today. The world servers found the remote uniserver and came back on line, but I couldn't get out to the WAN from the LAN, and so I couldn't test anything else. I gave up after a bit and swapped the SonicWall back in.
To start with, traffic between LAN and DMZ is pretty light, so I set NAT reflection to enabled (pure NAT).
I set virtual IPs as shown:
As a point of reference the firewall address ends in 117 and gateway (connection provider's modem) ends in 118. vrt1 and web1 are in use, while rails1 and pbo1 are (near) future projects. Because they are future projects I don't have any legit traffic on those addresses yet.
The ancient SonicWall only has WAN and LAN (DMZ) and so I'm using a cheap wifi/router to do NAT to my LAN, which has the public address ending in 116. I'd like that to go away when the pfSense box is up, and so NAT between LAN and WAN will be through the address ending in 117 (firewall's address).
It's not easy to cleanly switch between pfSense and SonicWall because the SonicWall is set to bridging to the DMZ, but I've decided I wanted to use server load balancing via the pfSense box and so I'm doing NAT (port forwarding currently, but I could change that to 1-to-1 forwarding, maybe). Thinking… Maybe I should set the SonicWall to 1-to-1 NAT so swapping firewalls isn't such a pain.
As suggested by PC Pickup in the video, I'm using port forwarding to equate various public addresses to specific addresses in my DMZ. I'm using RFC 1918 class-B addresses on my DMZ, and class-C addresses on my LAN. Here are my port forwarding rules:
Port aliases are either for a custom service, or logical groups of well-known ports. IP aliases are for a single DMZ address, but I want to switch that to a server-load-balanced pool (later, gotta get this working first).
The WAN rules set themselves up just fine, but I think pfSense was confused by my third network (DMZ). There were no firewall rules on the DMZ interface, just a warning that without rules there was no connectivity. So I manually created these:
The first rule is a copy of the first outgoing rule on the LAN. I thought I might need it. Also, this seems to be working, it's my NAT between LAN and WAN that's dead. But I do have a question: Destination "DMZ1 net" and "DMZ1 address" seem to do the same thing. I'm confused because there doesn't seem to be a way to select a specific address in the DMZ (not that I need to).
Now let's get into the NAT settings. I'm not using 1-to-1. My outbound settings are:
Purely auto-generated, but I cannot see the Internet from my LAN. I can ping all the ports on the pfSense box (including all the virtual IPs), but I can't resolve DNS entries in the wild, and so cannot see anything out there. I'm using my LAN server as a caching DNS server, not the SonicWall or the pfSense box.
As for outbound NAT from the DMZ. I'd like a particular address on the DMZ to always exit my network via a particular virtual IP on the WAN. Manually generating firewall rules isn't an issue, so should I dump port forwarding on those virtual IPs and just do 1-to-1 NAT? Ultimately (like next week) I'd like to turn my single DMZ addresses into pools. I have two new servers and I'll run jails on them to get different functional groups (vrt1, web1, rails1, etc.) then mirror between the two physical machines.
Okay, it's messed up. What have I done wrong?
Thanks a million for your help.
Read it, learn it, live it.
Your rules on DMZ1 have nothing to do with inbound connections on WAN to your port forwards. Those rules go on WAN (and you have selected to have the NAT rules automatically generate them - a wise move for now). The rules on DMZ1 govern what connections into DMZ1 hosts on DMZ1 can make (outbound connections from DMZ1 hosts into pfSense and potentially onward to somewhere else.)
And, for good measure:
On your last Port forward rule, FTP to PBO restricted, it is highly unlikely that you will be receiving packets inbound on WAN with a source address of LAN net. And if you did receive such packets inbound on WAN, you would probably want to drop them, not pass them. You probably don't need NAT at all and the rule for that would be placed on LAN with a source of LAN net and a destination of dmz_pbo1 (whatever that is).
Thanks. I'll study those pages.
Removed port forwards and rules for future projects, just to keep things simple for now. Success!
Tried to set 1:1 NAT on the SonicWall to make swapping between it and the pfSense box easier. Oops, complex setup and I might lose control of the SonicWall if I try. Guess it's back to manually swapping settings on the old server for now.
Tried to move the router's public interface to outside the SonicWall (it's WAN side) to make swapping between SonicWall and the pfSense box easier. Oh dang… the stupid SonicWall thinks the address belongs to it and fights with the router. I guess pfSense's virtual IPs aren't an unnecessary complication.
Still no go.
For awhile it seemed as if the connection to the DMZ was working, but not the LAN (so I could see neither the WAN nor the DMZ from the LAN). Then I got the LAN working, but the DMZ was definitely broken. Part of the problem is that I don't want any DMZ server disconnected from the WAN for more than a couple of minutes at a time. Then to make things interesting, I'm running several instances of the ActiveWorlds world server–which acts like both client and server depending on who it's talking to.
I tried a bunch of different physical topologies to get a temporary test setup going. This is what I currently have:
I put the pfSense box behind the SonicWall because the SonicWall thinks it owns all the public static IPs except the Actiontec's IP (which it sees as the gateway). Once this is working, all the pink stuff goes into the round file, the blue wires vanish, and the pfSense box gets the orange wire directly from to Actiontec. There are three networks: WAN which is split into two segments shown in orange and blue. DMZ which is shown in red. LAN which is shown in gray. The full IPv4 network address is shown, with the numbers next to each device's Ethernet port being the last octet.
Currently the LAN is hooked up to the Internet, and both the LAN and the Internet can see the paleolithic public everything server (let's call it the PPE server). More importantly, the PPE server can see AW's unisever, which is a metaserver for the world servers. Humans running the AW VR client log in to the uniserver, and world servers log in as well (acting as a client). The uniserver gets the IP address and port number of the world servers, then it builds a list of running worlds. Humans then click on a world's name and are transferred to the world server, which must now act like a server and not a client (following the client/server model). It's hyper-critical that whatever sits between the world server and the Internet doesn't change the IP address or port number of the world server in unexpected ways. 1-to-1 NAT or port forwarding is okay–as long as outbound packets (DMZ -> WAN) have the same VIP address used for the inbound NAT or port forward. The list of running worlds presented by the uniserver to the VR client bypasses DNS, so the only way to see a world server from the LAN is if NAT reflection is working.
I've been thrashing about with Outgoing NAT. I read https://doc.pfsense.org/index.php/Outbound_NAT, but it doesn't fully make sense. For one thing, what are those port 500 entries in the auto-generated rules? I tried equating a single DMZ address to a VIP address on the WAN, but I couldn't get the world servers to see the unisever. I even enabled and disabled that port 500 rule on the DMZ, with the same result. This is my setup for Outbound NAT:
When I look at the system logs for the firewall, I see a lot of broken TCP connections from the uniserver. Seems they are hitting the WAN address rather than the VIP address belonging to the PPE server. I rebooted both the pfSense box and the PPE server, started and stopped services, etc, all multiple times. About the only thing I didn't do is beat it with a rake.
The pfSense box currently has the WAN address ending in 116, and a VIP ending in 115 (which doesn't interfere with anything else). My thought is to leave the PPE sever where it is, then get one of the new servers going. I can port forward one service at a time to the new box (diddle DNS to point to the new VIP), or in the case of the problematic world servers, I can move my world to the new server for experimentation, and not take down the worlds of paying customers for hours at a time.
So this is probably a NAT problem and not an incoming rules problem. What am I doing wrong?
Thanks a million.
The port 500 outbound NAT rules are to help outbound IPsec pass-through. They won't hurt anything.
Rules are matched top down. If you want outbound NAT to have a different behavior for 172.16.1.16/32 you have to move that rule above the rule for 172.16.1.0/24 (which will match 172.16.1.16).
This is covered in the firewall rule processing order document linked above. You might want to review it again.
Thanks. Let's look at these rules.
Rules 1 and 2 are for the loopback address subnet. How/why does anyone expect/want/need a network device to pass any of these addresses? Why do these rules exist?
Rules 3 and 4 look fantastic. And I totally get the verbiage about address pools on this page: https://doc.pfsense.org/index.php/Outbound_NAT, and how that relates to groups of LAN users using different WAN IP addresses (so WAN address doesn't run out of resources). I also suspect I can use address pools for my DMZ. E.g. 8 - 9 pool outbound to VIP 113, 16 - 17 pool outbound to VIP 114, etc (but I could be completely wrong about that too).
Rules 5 and 6 are the problem. I thought the more specific rule (port 500) wouldn't affect the broader rule I constructed. It was my guess that if I put my rule before the port 500 rule, then the port 500 rule would never come into play. Is this wrong?
In general, it's my understanding that rules for other networks (e.g. loopback and LAN) have no effect on DMZ rules. Is that also wrong?
I do understand how the order of rules is significant, but I didn't see an issue with these rules. Apparently pf and pfSense make assumptions. For example on this page: https://doc.pfsense.org/index.php/Firewall_Rule_Basics it says "Where no user-configured firewall rules match, traffic is denied." (2nd paragraph)–which is why there's no explicit block everything rule at the bottom of the list. Are there other built-in assumptions I've missed?
Thanks again for your help.