Please help me oh mighty gods of pfsense and nat reflection

fagoti

Hi there, im having a hard time trying to achieve some things with pfsense here at work. please aid me with some good directions

My scenario is this (ips and names changed):

pfsense local ip - 172.16.100.2
pfsense wan1 - X.X.X.X
pfsense wan2 - Y.Y.Y.Y
domain mypfsense.com

active directoty /dns - 172.16.100.1
main local network 172.16.0.0 / 16
routers 172.16.200.0 / 24
hosts behind routers 192.16.0.0 / 24

i have lots of nats here,
such as
pfsense (port 1111) goes to 172.16.100.11 (1111)
pfsense (port 2222) goes to 172.16.100.12 (2222)
pfsense (port 3333) goes to 172.16.100.13 (3333)
pfsense (port 4444) goes to 172.16.100.14 (4444)
.... and so on.. to 80 records.

i have non transparent proxy and squidguard all set.

host are configured as follows:
IP 172.16.X.X
MASK 255.255.0.0
GATEWAY 172.16.100.2 (PFSENSE)
DNS1 172.16.100.1 (DNS/AD)
DNS2 172.16.100.2 (PFSENSE)
PROXY ENABLED, and pfsensedomain + local address on bypass.

and host behind routers (172.16.200.x) are on dhcp

with nat reflection configured as (nat + proxy) this happends:
OK - external host (world wide) to access all NATs through domainname
OK - internal host with proxy enabled to access all NATs through domainname (wan ips)
OK - internal host with proxy disabled to access all NATs through domainname (wan ips)
OK - internal host behind routers (192.168.x.x proxy enabled, behind router 172.16.200.x) to access all NATs through domainname (wan ips)
OK - internal host behind routers (192.168.x.x proxy disabled, behind router 172.16.200.x) to access all NATs through domainname (wan ips)

but there is a catch ...

all those internal redirections behave abnormally.. as if part of the requests is getting lost while youre navigating...
example1: when using a php website, clicking on a button to submit... it hangs.... you click it again, and then it goes.. but this occurs only part of the times, at given moments.
example2: there is a specific service on a specific port, developed in asp, that is a little heavy on loading. and in this scenario, it wont even complete the nat.

i then change (nat+proxy) to pure nat.. and all that behavior is fixed (even the heavy asp system). but then i break some of the internal access of that checklist (those behind routers).

obs: i dont need any of those nat to be intercepted by proxy

so my questions are:

What exactly is happening with that strange behavior on internal access through (nat + proxy)?
How can i avoid that behavior and keep all those internal access working?
Is there a way to make pure nat work on those host behind routers? cause if so, that is already a solution for me.

viragomann

@fagoti said in Please help me oh mighty gods of pfsense and nat reflection:

How can i avoid that behavior and keep all those internal access working?

Set up split DNS.

fagoti

@viragomann hi viragomann! thanks for your quick response!

i did read lots of suggestions on that matter in my research.. but i count figure out how to do it effectively

can you give me a step by step on that? if is not too much trouble

thanks in advance

viragomann

@fagoti

DNS1 172.16.100.1 (DNS/AD)
DNS2 172.16.100.2 (PFSENSE)

Windows AD, I guess?
As far as I remember, you have to add your domain to the DNS as a local one. Then add the host name to this domain (for the root domain leave the host name blank) and state its local IP.

On pfSense in the DNS Resolver settings (I assume you're running it) add a domain override for your domain and point it to the Windows server.
Alternatively you can simply add host overrides for all your internal host name. So the local clients can even resolve host names if the Windows server is offline.

E.g. a host override for mypfsense.com:
domain: com
host name: mypfsense

fagoti

@viragomann That was quick!

thanks! im heading for lunch break now, but soon i can try that then.

i let you know if that worked out well..

but thanks a lot for the help!

fagoti

@viragomann

here is an update

the pfsense part was quite straightforward, so im positive i managed to do exactly as you suggested.
but the ad part is a little bit off for me..
in ad there is two zones on dns, the direct and reverse, right?... i assuming you told to add in the direct,
but my domain (Active directory domain) is different from the domain used for reaching my firewall from outside and into the nats.
my active directory domain name is only used locally for my network user management. should it be the same?
in that example of mine, my outside domain is mypfsense.com but my active directory domain is something.else

dont really know how to handle that

but lets imagine i make it work at some some point.. then i should use pure nat?

viragomann

@fagoti said in Please help me oh mighty gods of pfsense and nat reflection:

in ad there is two zones on dns, the direct and reverse, right?... i assuming you told to add in the direct,
but my domain (Active directory domain) is different from the domain used for reaching my firewall from outside and into the nats.

So you have to add a new zone for your public domain, according your example mypfsense.com.
Then add a host in this zone. For mypfsense.com itself the name should be blank or a dot, not sure.

fagoti

@viragomann

oh that i could understand, ok, sucessfully did that, new zone , new host.

but no success on that doing the trick.

i did some testing here ...

i took one of this hosts behind router (192.168 behind a 172.16.200.x) , and assigned to it only my AD dns, and no gateway, and no escape to internet.
i does translate mypfsense.com to the local ip correctly, wich means my ad/dns is doing its job. but it cant use ports with pure nat for some reason, it can only use the (nat + proxy)

besides, even if i use the actual ip of the pfsense in the addressbar, with the ports.. it doesnt work with the port set in pure nat, only on the ports with nat + proxy.
and that is a case only for hosts behind routers (using the common 192.168)

fagoti

@fagoti i can only think proxy is interfering in some way, and blocking pure nats when source is from different ip class other then his own. because hosts not behind routers, sharing same network manage to access it alright..

viragomann

@fagoti
Ah yes, that's an issue. DNS does not consider your port translations.

You can only resolve a single host name to a single IP.
E.g.
host1.mypfsense.com > 172.16.100.11
host2.mypfsense.com > 172.16.100.12

So you would need to call host2.mypfsense.com:2222 to get to 172.16.100.12:2222.

i can only think proxy is interfering in some way, and blocking pure nats when source is from different ip class other then his own.

The NAT proxy is out of play with DNS overrides.

fagoti

@viragomann

thats a bummer :(
well, i guess thats thats a way to solve it .

maybe i should create and redo all links in that fashion

dont know if im really happy with it, but i think i will start doing it right away, so that really becomes an option.

its sad cause.. it would be fantastic if a single domainlink could work inside and outside my network, always reaching for the pfsense and always getting translated to according port.

but for now i think i gonna stop the efforts.

now im just eager to know why pure nat over proxy still works on the same network but not in the subnetworks routed.....

viragomann

@fagoti
The issue with the NAT reflection, which you described in the first post, may possibly indicate an asymmetric routing issue. But you would have to investigate this.
You can use packet capture on pfSense to check if request and response packets passes pfSense properly.

fagoti

@viragomann i will try that, im afraid im not exactly great on that , but im try my best.

as for your suggestion , i just did a sample here, and it works well.. but yes... im gonna have to notify every client, change every link in every station.
and will be exactly like this
host1.mypfsense.com > 172.16.100.11
host2.mypfsense.com > 172.16.100.12

the ultimate bummer, is that, one might think it would be really nice to use that oportunity to have outside domainnames that can be typed without the ports and redirected apropriatedly,...
but, if i do that, i will have to use a link for for external (with no ports) and a link for internal (with ports)... and to have 2 links will drive people nuts...

so unfortunatelly i will be creating those subdomains with the redirects without ports,, and use the ports in every link so the external and internal links match.

thanks for all your time and attention

greating from brazil!

viragomann

@fagoti
Yes, when using certain ports with NAT reflection, it cannot be simply replaced with DNS overrides.

However, there is a simpler method to detect asymmetric routing, assuming the issue concerns to TCP connections:
In Status > System Logs > Settings ensure that "Log packets matched from the default block rules in the ruleset" is enabled.

After facing an issue check the firewall log. If you have asymmetric routing you would see block of TCP packets with other than SYN flags, e.g. RA, PA.

johnpoz

@fagoti said in Please help me oh mighty gods of pfsense and nat reflection:

active directoty /dns - 172.16.100.1
main local network 172.16.0.0 / 16
routers 172.16.200.0 / 24

Ok this problematic - so your lan is 172.16/16 but you have some other router a 172.16.200/24 - that overlaps.. Can you do a drawing.. If you have a downstream router that really should be on a transit, unless this downstream router is doing nat? if that is the case why would it be on some 172.16.200/24 that overlaps your /16?

fagoti

@johnpoz

thanks for trying to help too john,

my lan is indeed /16 and the subclass 200.X is reserved to routers
then , this routers are all 200.x on wan and have dhcp 192.168.x.x for their lan.

client 192.168.x.x access without problems servers on 172.16.x.x

but those routers are not using nat of any kind.

johnpoz

@fagoti so its really just 172.16/16 your just putting all your routers on 172.16.200/16 range of IPs

but those routers are not using nat of any kind.

That is going to be asymmetrical, if your downstream routers are not doing nat, then that network from your upstream to downstream should be a transit network, no hosts on it..

If I am understanding yoru setup this is asymmetrical.

either direction ends up with asymmetrical flow, unless you were doing host routing on the clients on your 172.16/16 network

When you use downstream routers that do not nat, you should be using a transit network.

fagoti

@johnpoz oh my, you even made diagrams... :) , thanks again, but now it got a little overwhelmed with that many networks and information

i wont be asking for more until i manage to study this with the same care and fully understand it

but it seems like a lot of changes for my already huge network.

johnpoz

@fagoti said in Please help me oh mighty gods of pfsense and nat reflection:

but it seems like a lot of changes for my already huge network.

Creating a transit is not a lot of work.. But running downstream routers that don't nat are going to cause you all kinds of grief without a transit network.. Or doing host routing on ever host..

Transit network is the correct solution..

fagoti

im trying to reply but i keep receiving akismet spam message :/