Backend servers cant see Real users IP's
-
After switching to pfSense, backend server receiving IP of Nginx proxy which is 192.168.2.2 so every web user is coming from same IP. Before I have simple consumer router with same simple port forwards. Using default pfSettings just added some port forward.
Updating with more info
Ok, so it seams problem is with subdomains not domains. Situation now:
If client go to domain.com - everything is fine backend server can see real clinet IP
If client go to subdomain.domain.com - backend server see proxy server IP
All domains A records points to external IP, then pfSense forward 80 port to proxy, then proxy depending on domain forward to corresponding internal server.
I have 2 physical servers, 1 - pfSense router and another with virtualbox running many VM's in this example 4 VM's
Solved
Ok so I managed it, partially you all right but problem is not in proxy server, I needed extra step in backend server (litespeed web server) to complete my task, there is option "Use Client IP in Header" this must be accidentally disabled, I know about this option but was sure that it's set to yes for that machine for all other servers it set to yes, that why some servers working as expected and this one, not. So it was backend server misc-configuration, not pfSense and not proxy server.
-
Bump. Updated question.
-
Well seems you should be on the nginx forums then.. What does this have to do with pfsense? All it does is forward traffic it sees from internet to your proxy IP..
You do understand you don't need nginx in the mix, pfsense can be a reverse proxy.
-
Well seems you should be on the nginx forums then.. What does this have to do with pfsense? All it does is forward traffic it sees from internet to your proxy IP..
You do understand you don't need nginx in the mix, pfsense can be a reverse proxy.
I do understand this, but as I wrote: I used another consumer router before, and I dont want to change current infrastructure.
Another interesting thing, when I try to access subdomain.domain1.com from inside I getting this:
And again, no problems with domain1.com domain2.com and so on…
And how come this related to Nginx if before everything worked perfectly. I just changed routers...This is definitely pfSense setup problem.
-
dude your trying to access your public IP from internally.. Did you setup nat reflection?
What does subdomain.domain1.com resolve too? Test you access from outside not inside coming from where your lan.. Look at your drawing that would be a nightmare not only nat reflection but then asymmetrical route.. And yeah proxy would see your lan IP in that sort of test.
-
dude your trying to access your public IP from internally.. Did you setup nat reflection?
What does subdomain.domain1.com resolve too? Test you access from outside not inside coming from where your lan.. Look at your drawing that would be a nightmare not only nat reflection but then asymmetrical route.. And yeah proxy would see your lan IP in that sort of test.
NAT reflection is checked in global config and it's pure nat, tried to change other reflection settings, nothing helped. Tried to set NAT reflection in port forward rule, same result.
I always testing from outside, just once tried to access from local network and get this security warning, anyway I don't usualy access any of domains from that network. By the way if client comes from inside, reported IP is gateway as expected (domain1.com and domain2.com)
I'm not network specialist I'm programmer, just want to set it and forget.
And subdomain1.domain1.com resolve my external IP ofcourse. Din't tried ping localy bet I guest from that error that it will resolve to external IP too.
Sorry for typos, using mobile now and can't check spelling.
-
"I always testing from outside"
Tnen you have NO use of nat reflection and it should be OFF!! Setup up our internal name resolution to resolve your fqdn to the internal IPs of either the server directly or your proxy internal IP.
-
"I always testing from outside"
Tnen you have NO use of nat reflection and it should be OFF!! Setup up our internal name resolution to resolve your fqdn to the internal IPs of either the server directly or your proxy internal IP.
By default ir was off, when I first time setup pfSense and saw problem, then started to digg, then started changing reflection settings. This doesn't make sense to me right now, what is different between domain1.com and subdomain.domain1.com they both pointing to same external IP and every request to port 80 forwarded to proxy.
-
So your saying this does not work from external.. PM me the actual fqdn or post it here.
-
So your saying this does not work from external.. PM me the actual fqdn or post it here.
Ok so here is 2 links who will show you IP, it's simple php remote addr:
1. hdd.lt/php.php
2. dev.prodev.lt -
I get this..
Sure looks like different pages are presented to me
getting back a 404 for your favicon
GET /favicon.ico HTTP/1.1
Host: dev.prodev.lt
HTTP/1.1 404 Not Found
Server: nginx
Date: Sat, 27 Aug 2016 12:41:39 GMT
-
I get this..
Sure looks like different pages are presented to me
They all point to same server, I can give you another excample
https://prodev.lt/php.php vs http://dev.prodev.lt
They both pointing to same external IP, same server, but dev.prodev.lt reporting proxy IP instead of my external.
-
And what IP is 192.168.2.2 That is not pfsense IP is it?? Your just doing a simple port forward.. pfsense doesn't give 2 shits what about what is in the traffic only that it is to port 80.. Why don't you sniff on pfsense lan IP and validate for yourself. You have a proxy issue.
Is your proxy resolving these internal fqdn??
Maybe your old router was doing nat reflection and your proxy was sending traffic to your wan IP and then getting redirected in, etc.. Which should never happen anyway in a correct setup.
Pfsense doesn't care what the fqdn it sees traffic to 80, it looks in forward and says oh that goes to IPaddress123.. It doesn't look in the packets, it doesn't care if they are looking for domain.tld or host.domain.tld it just sends it to your proxy.. What your proxy does with it is up to the proxy..
Sniff on your wan and see what is happening.. My guess is your proxy is trying to resolve a name and is getting public vs internal and then sending it to your wan and your trying to do nat reflection so yeah traffic looks like it comes from your own proxy IP which is that 2.2 address I assume.
-
And what IP is 192.168.2.2 That is not pfsense IP is it?? Your just doing a simple port forward.. pfsense doesn't give 2 shits what about what is in the traffic only that it is to port 80.. Why don't you sniff on pfsense lan IP and validate for yourself. You have a proxy issue.
Is your proxy resolving these internal fqdn??
192.168.2.2 is a proxy, and proxy upstreaming traffic to internal IP depending on FQDN… very simple config. I will double check latter today.
-
And what IP is 192.168.2.2 That is not pfsense IP is it?? Your just doing a simple port forward.. pfsense doesn't give 2 shits what about what is in the traffic only that it is to port 80.. Why don't you sniff on pfsense lan IP and validate for yourself. You have a proxy issue.
Is your proxy resolving these internal fqdn??
Maybe your old router was doing nat reflection and your proxy was sending traffic to your wan IP and then getting redirected in, etc.. Which should never happen anyway in a correct setup.
Pfsense doesn't care what the fqdn it sees traffic to 80, it looks in forward and says oh that goes to IPaddress123.. It doesn't look in the packets, it doesn't care if they are looking for domain.tld or host.domain.tld it just sends it to your proxy.. What your proxy does with it is up to the proxy..
Sniff on your wan and see what is happening.. My guess is your proxy is trying to resolve a name and is getting public vs internal and then sending it to your wan and your trying to do nat reflection so yeah traffic looks like it comes from your own proxy IP which is that 2.2 address I assume.
Ok this partially make sense, I will get back home and double check nginx (proxy) and post results here.
-
Your web servers are going to see connections coming in from the IP address of the reverse-proxy - no two ways about it.
What you need to do is to configure nginx to add "X-forwarder-for" headers to the headers of incoming connections, and tell your web servers to log the "X-forwarded-for" IP address instead of the actual incoming IP address.
-
^ this makes sense.. Which again as been saying has nothing to do with pfsense at all..
-
Your web servers are going to see connections coming in from the IP address of the reverse-proxy - no two ways about it.
What you need to do is to configure nginx to add "X-forwarder-for" headers to the headers of incoming connections, and tell your web servers to log the "X-forwarded-for" IP address instead of the actual incoming IP address.
You wrong, I'm using nginx real ip module and sending required headers, you don't realise, that IP is already forwarded as shows in my excamples, just this stuff doesnt work in specific subdomain, for all other domains working as expected. Going to check specific vhost maybe there is something wrong or typo or something.
-
If you are forwarding a WAN address to your nginx proxy and have example.com, subdomain1.example.com, and subdomain2.example.com resolving to WAN address, pfSense has zero idea what domain or subdomain your clients are connecting to. It is only looking at IP addresses and ports. It will be behaving exactly the same for all of them.
If that is the case your problem absolutely lies in your proxy.
-
Ok so I managed it, partially you all right but problem is not in proxy server, I needed extra step in backend server (litespeed web server) to complete my task, there is option "Use Client IP in Header" this must be accidentally disabled, I know about this option but was sure that it's set to yes, so it was backend server misc-configuration, not pfSense and not proxy server.
