NAT Reflection (Pure NAT) not working for same subnet (v2.2.2)
-
I am trying to get NAT Reflection (Pure NAT) completely working on pfSense 2.2.2. I cannot use Split DNS (some NATs change the destination port, and there are access restrictions between internal subnets). "NAT + Proxy" didn't work either, and I don't want that anyhow.
The short summary is that I've tried both to enable "Pure NAT" at a single NAT rule and at the system level (System::Advanced::Firewall/NAT::Network Address Translation). The best I can get is to have it work on every internal subnet except the target one. E.g. I can't be on a machine and have it reflect back to itself via the public IP:port.
I know the technical reason why it's not working. The rules that pfSense is setting up are only half the necessary solution. ( http://www.openbsd.org/faq/pf/rdr.html#reflect )
pfSense is not creating the "pass out on … nat-to" rule to change the source IP. I have confirmed this is exactly what's happening.(See Edit 1.a down below)At first I used netstat -n on each machine to watch the connections as I tried to hit a web site via reflection. Finally, I used "Diagnostics::Execute Command" to run pfctl -s nat and used BeyondCompare to look at how things changed. The output from pfctl confirms that when I turn on Pure NAT (either at a NAT rule or globally), pfSense doesn't create all of the needed rules to make it completely work.
Does anyone know what's going on and how to fix it?
For a more concrete example: Here's one of my NAT rules to my "betaweb" alias, from the WAN ("re0"), as reported by pfctl. I masked the IP address.
rdr pass on re0 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin</betaweb>When I turn on Pure NAT, I get some added rules. "betaweb" is in the subnet served by "re2_vlan2" (But I've confirmed that the behavior is identical from "re1", so please don't waste time on the Vlan idea).
rdr pass on re0 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on re1 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on re2_vlan2 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on openvpn inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin</betaweb></betaweb></betaweb></betaweb>Considering what I've read on the Internet about
OpenBSD's ipfilter(See Edit 1.b down below), I would have expected another rule for "re2_vlan2" that rewrites the source-address on same-subnet redirected packets so that the target machine would send them back to the gateway instead of trying to directly contact the source machine. But there's nothing added, as reported by pfctl.Edit 1.a: As of FreeBSD 10.1, its implementation of pf does not yet support "nat-to" and "rdr-to". ( https://www.freebsd.org/cgi/man.cgi?query=pf.conf&sektion=5 )
Edit 1.b: pfSense is based on FreeBSD, not OpenBSD. Additionally, it uses "pf", not "ipfilter". -
"I cannot use Split DNS"
And why is that exactly?? You have something hard coded to use a public IP and doesn't use dns? Fix the something! ;)
-
I can't use it for the 2 reasons mentioned in my post. Split DNS is a niche solution.
[1] The external port is different than the internal one; I can't change this requirement. Externally it's "https://some.domain/…" but internally, the URL would have to be "https://some.domain:1234/..." and DNS doesn't address this situation.
[2] Some subnets are not allowed access to others. I could work around this, but it would require adding (essentially) duplicate firewall rules on most of the interfaces pfSense controls. I do not want this extra complication as it makes maintenance more difficult.
NAT Reflection is a much more elegant solution across the board for configuration and maintenance, both within pfSense and the various applications within the networks it supports.
-
This NAT Reflection packet ping-pong is a nasty hack, not a solution to anything.
-
So… you have no idea.
Should I file a bug report then?
As for the "nasty hack" comment, that's your opinion. I'm of the opinion that "Split DNS" is its own "nasty hack". Both for the above-mentioned port-related shortcoming as well as the fact that you can't serve different IPs based on the source subnet. NAT Reflection is standard, basic stuff for a NAT'ing router. It's just an IP-rewrite operation that rewrites both IPs instead of one. It relies on the same stateful packet inspection (SPI) functionality that outbound NAT'ed connections rely on.
-
So your using nat to change dest ports – that is the reason you can not use split dns?? Resolving the ADDRESS closest to you and on your own local network, to use that address is not a niche anything.. That would be the correct thing to do..
NAT has been and always be an abomination!! ;) Be it your reflecting it or not..
-
So your using nat to change dest ports – that is the reason you can not use split dns?? ...
Yes. That was answered twice already.
-
Since no one has anything constructive to offer here, I'll proceed with the bug report. Hopefully, the process won't take so long that I forget to post an update here for other Googlers that pass by.
-
So your problem is your design then.. And still haven't answered the question with a valid response..
I have a server listening on port X.. Why would I want to hit pubic IP on port Y to be reflected back to port X.. Why not just hit the private network address that my box is on already with port X..
You seem stuck on wanting to NAT more than is required.. Yes you have public IP and you want to get to private IP this must be natted. And hey you want public IP port X to go to private IP Y.. Ok that is fine.. But why does another machine same private network have to hit public IP on X just to be sent back to private Y??
Why not just go to private Y directly??
You saying you are doing port redirection is not an answer to WHY you would need or want to do such a nonsense thing..
-
I know you are dead set in your opinion that NAT reflection should work like you want it to but I think you are going to be disappointed.
Your real problem is you expect something to perform NAT but the client and servers are on the same subnet. Anything that will make that happen will be firmly in the "ugly hack" category.
Put your servers behind an OPTX interface and NAT to them from WAN
Then since your ports have to be translated, use split DNS to point your LAN hosts directly at the inside (real) IP address of the servers and translate the ports (not the addresses) with rules on LAN similar to those on WAN.
Set it up that way and it'll just work.
-
…
Then since your ports have to be translated, use split DNS to point your LAN hosts directly at the inside (real) IP address of the servers and translate the ports (not the addresses) with rules on LAN similar to those on WAN.
Set it up that way and it'll just work.
I already have this functionality. And it appears that you don't realize that your proposal requires the functionality of the "ugly hack" you eschew. Translating the ports is the same functionality as translating the IP: it's all Network Address Translation.
The problem explained in the initial post is that the pfSense-supported feature of NAT Reflection is not completely working, because the necessary pf rule is not being set. It's a single checkbox. I'm asking for people to offer help as to why the functionality offered by this checkbox wouldn't be working. Perhaps there's a related setting or feature-set that is documented to interfere with it? I don't know; that's why I'm asking. If there was a supported way for me to manually add in a persistent pf rule as a workaround, that would be a possibility.
It's like I'm asking about why my backup-camera isn't working on my new car. But instead of trying to help, people decide that time is better spent pontificating about how stupid backup-cameras are, and that I just shouldn't use them.
-
You need to enable "Enable automatic outbound NAT for Reflection", which adds the "nat on …" rules to accommodate routing traffic back out the same interface it came in on.
-
Sounds like you have that box checked. You're not looking for nat-to on a pass out, first make sure you're actually looking for the right thing.
Example for a port forward to WAN IP 192.0.2.127 port 1234 to internal host 192.168.111.100, where vmx0 is WAN, vmx1 is LAN.
# grep "rdr on" /tmp/rules.debug rdr on vmx0 proto tcp from any to 192.0.2.127 port 1234 -> 192.168.111.100 rdr on vmx1 proto tcp from any to 192.0.2.127 port 1234 -> 192.168.111.100 # grep "nat on" /tmp/rules.debug no nat on vmx1 proto tcp from vmx1 to 192.168.111.100 port 1234 nat on vmx1 proto tcp from 192.168.111.0/24 to 192.168.111.100 port 1234 -> 192.168.111.1 port 1024:65535If you don't have such rules, most likely it's because you have a gateway chosen on the interface in question under Interfaces>LAN or whatever the internal interface is. That's strictly for Internet connections, and outbound NAT for reflection is only for non-Internet connections. If that's not the case, we'll need some more details about your config. This definitely works in general.
-
@cmb:
Sounds like you have that box checked. You're not looking for nat-to on a pass out, first make sure you're actually looking for the right thing.
I see my mistake: I conflated OpenBSD with FreeBSD. FreeBSD doesn't currently implement the "nat-to" stuff, so yes, I looked for the wrong thing in filter.inc.
@cmb:
Example for a port forward to WAN IP 192.0.2.127 port 1234 to internal host 192.168.111.100, where vmx0 is WAN, vmx1 is LAN.
# grep "rdr on" /tmp/rules.debug rdr on vmx0 proto tcp from any to 192.0.2.127 port 1234 -> 192.168.111.100 rdr on vmx1 proto tcp from any to 192.0.2.127 port 1234 -> 192.168.111.100 # grep "nat on" /tmp/rules.debug no nat on vmx1 proto tcp from vmx1 to 192.168.111.100 port 1234 nat on vmx1 proto tcp from 192.168.111.0/24 to 192.168.111.100 port 1234 -> 192.168.111.1 port 1024:65535If you don't have such rules, most likely it's because you have a gateway chosen on the interface in question under Interfaces>LAN or whatever the internal interface is. That's strictly for Internet connections, and outbound NAT for reflection is only for non-Internet connections. If that's not the case, we'll need some more details about your config. This definitely works in general.
I do not have a gateway set on my (internal) "BETA" interface. The config I have in /tmp/rules.debug does not have anything like the 2 "nat on" rules you listed.
However, I manually recreated the necessary "nat on" rule on the "Firewall: NAT: Outbound" page, and now NAT Reflection is completely working as expected. So, at least I have a workaround for now. Thanks for the example. When I first attempted a workaround by manually creating an outbound NAT rule, I didn't know that "rdr" applies before "nat", I mistakenly put the WAN IP:port for the destination.
If ever I get time, I expect that I'll have to rebuild my config on a test platform to discover what's interfering with this functionality. In the mean time, this is the relevant part of /tmp/rules.debug now. ( betaweb resolves to 10.121.12.131, and re2_vlan2 serves 10.121.12.0/24 ):
... BETA = "{ re2_vlan2 }" ... betaweb = "<betaweb>" ... # Outbound NAT rules (manual) ... nat on $BETA proto tcp from 10.121.12.0/24 to 10.121.12.131/32 port 443 -> 10.121.12.1/32 port 1024:65535 ... rdr pass on re0 proto tcp from any to x.x.x.198 port 443 -> $betaweb # Reflection redirect rdr pass on { re1 re2_vlan2 openvpn } proto tcp from any to x.x.x.198 port 443 -> $betaweb ...</betaweb>…and the same thing, as displayed by pfctl:
... nat on re2_vlan2 inet proto tcp from 10.121.12.0/24 to 10.121.12.131 port = https -> 10.121.12.1 port 1024:65535 ... rdr pass on re0 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on re1 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on re2_vlan2 inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin rdr pass on openvpn inet proto tcp from any to x.x.x.198 port = https -> <betaweb>round-robin ...</betaweb></betaweb></betaweb></betaweb> -
I suspect it's something to do with the round robin you're doing. Would you mind sharing a copy of your config with me? If you copy/paste it from status.php it'll sanitize it to remove certs, passwords, etc. Can email to cmb at pfsense dot org or attach to a PM here and I'll delete it after testing.
-
I'm interested in a fully integrated solution to this as well. I've been using pfsense as an enterprise firewall for a long time, and freebsd for even longer; roughly 20 years now. There are use cases (like mine) where split DNS is simply not an option.
For example, a current project of mine has me using pfsense in a multi-wan environment protecting a corporate internal LAN as well as separate LAN subnets for machines (VMs) the company is hosting for customers. These segments are all in their own VLAN and intentionally disallowed from accessing other customer subnets as well as the corporate LAN subnet.
For example, imagine the following five networks:
WAN1 : 1.0.0.0/28 (VLAN 1)
WAN2 : 2.0.0.0/28 (VLAN 2)
LAN: 3.0.0.0/24 (VLAN 10)
CUST1 : 4.0.0.0/24 (VLAN 100)
CUST2 : 4.0.1.0/24 (VLAN 101)This is very similar to a setup I'm currently working with. All of the public IPs are assigned as virtual IPs on a single pfsense install. LAN and CUST networks are allowed nearly unrestricted outbound traffic to the internet. LAN traffic is allowed into each CUST subnet for management. CUST networks are not allowed to initiate traffic to the LAN or to other CUST networks.
Desired operation looks basically like this:
WANx -> LAN : Restricted. Some explicit port forwards.
WANx -> CUST* : Restricted. Some explicit port forwards.
LAN -> CUST* : Unrestricted.
CUST1 -> CUST1 : Unrestricted
CUST2 -> CUST2 : Unrestricted
CUST -> LAN : Rejected
CUST1 -> CUST2 : Rejected
CUST2 -> CUST1 : RejectedThere are other ways to do it, but "NAT reflection" is the simplest, requiring the fewest changes to existing infrastructure.
-
Not working for me too.
Version 2.2.4 (amd64)
Old instalation 2.1.5 (amd64) is work.
Update 2.1.5 to 2.2.4 too workNew instalation 2.2.4 not work
-
Still not working :o
-
Pure NAT for same subnet works fine. You also need to enable the source NAT option, "Enable automatic outbound NAT for Reflection".
-
I know this is a bit of thread necromancy on my part, but I had the same issue as the OP, and was able to use the discussion here to put together a fix for the bug. (Yes, I really do think it is an actual bug).
I figured I would put my findings here for future people googling the issue.
HOWEVER.. when I try to submit, I am being told that I am spam. "Post content was flagged as spam by Akismet.com" Grr.
So here is my analysis:
https://docs.google.com/document/d/1DCtqI2q3RlaK6HkTgp_xFw6poxWg6wiJlzw0V7lDtGU/edit?usp=sharing
