[SOLVED] HaProxy forward client IP

manjotsc

Solved

Guide : https://www.digitalocean.com/community/questions/get-client-public-ip-on-apache-server-used-behind-load-balancer

First you need to set option forwardfor in each backend to have in HaProxy, under Advanced Settings then in Backend pass thru box put option forwardfor and hit save, Don't set forwardfor in frontend, then you need to enable "sudo a2enmod remoteip" in your apache2 server and edit config /etc/apache2/apache2.conf and add the following line at bottom of config, RemoteIPHeader X-Forwarded-For and in that same config locate the log file line : LogFormat "%h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined and change %h to %a to see the ip in logs, like this LogFormat "%a %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined and restart your apache2 server and you are done and check your apache access logs and you should see public ip of clients.

Screenshot 2021-01-02 114223.png

[root@cPanel ~]# a2enmod remoteip
Enabling module remoteip.
To activate the new configuration, you need to run:
  systemctl restart apache2
[root@cPanel ~]# systemctl restart apache2
[root@cPanel ~]#
[root@cPanel ~]# nano /etc/apache2/apache2.conf 

LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

# HaProxy Forward for Enable

RemoteIPHeader X-Forwarded-For

[root@cPanel ~]# systemctl restart apache2

Screenshot 2021-01-02 114051.png

------------Before--------------- 

192.168.80.1 - - [02/Jan/2021:11:10:00 -0500] "GET / HTTP/1.1" 200 623 "-" "Mozilla/5.0 (Linux; Android 10; SM-G975W) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36"
192.168.80.1 - - [02/Jan/2021:11:10:00 -0500] "GET /style.css HTTP/1.1" 200 277 "https://speed.manjot.net/" "Mozilla/5.0 (Linux; Android 10; SM-G975W) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36"
192.168.80.1 - - [02/Jan/2021:11:10:02 -0500] "GET /favicon.ico HTTP/1.1" 200 112448 "https://speed.manjot.net/" "Mozilla/5.0 (Linux; Android 10; SM-G975W) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36"

---------- After --------------  

204.48.93.246 - - [02/Jan/2021:11:11:05 -0500] "GET / HTTP/1.1" 200 623 "-" "Mozilla/5.0 (Linux; Android 10; SM-G975W) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36"
204.48.93.246 - - [02/Jan/2021:11:11:06 -0500] "GET /favicon.ico HTTP/1.1" 200 112448 "https://speed.manjot.net/" "Mozilla/5.0 (Linux; Android 10; SM-G975W) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36"

GreenAnt

@manjotsc

some further notes on this:

the forwardfor option in the HAProxy front-end config works fine for forwarding the client address
you can use the backend option but I don't think it's necessary.

For users using nginx. A similar logging can be achieved by:

add this to /etc/nginx/nginx.conf in the http { block:

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    log_format combined_realip '$http_x_forwarded_for - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

then in your virtualhost config:

access_log      /var/log/nginx/access.log combined;
access_log      /var/log/nginx/access_realip.log combined_realip;

the access_realip.log will then include the client address. You can then use that to verify the HAProxy is passing through the variable properly.

see this guide for reference: https://djangocas.dev/blog/nginx/nginx-access-log-with-real-x-forwarded-for-ip-instead-of-proxy-ip/

GreenAnt

@greenant

if you have the http_realip_module available in nginx, then the config is even simpler, just add to nginx.conf:

real_ip_header X-Forwarded-For;
real_ip_recursive on;
set_real_ip_from <your proxy/CDN IP CIDR here>;

braunerroei

Hello,

Didn't work for me (both ways [frontend / backend]).

I have Synology NAS behind my pfSense/HaProxy, and the Synology shows the pfSense IP and not the real client IP.

-Roei

braunerroei

for synology:
just configure "Trusted Proxies" - Control Panel> Security > Trusted Proxies

bigtfromaz

@braunerroei I curious, how did you get HAProxy in front of Synology to work? Is there an article somewhere that lays it out? Frontend-backend relation seems to be correct. It's working for our private Docker registries but when I set up https offload to http backend Synology, I just get 503s. "Use forwardfor" is checked and the front-end and backend are green in the stats.

braunerroei

@bigtfromaz
Hi

I am not pfSense / Haproxy expert, but it seems to be very easy.

Why https offload to http backend?

I configured backend with port 5001, and 2 frontends, one (shared) with 443 port and one with 5001 port, both use the same backend.

-Roei

bigtfromaz

@braunerroei

I assume you are storing your certificate on the Synology's box. If not, how are you securing your connection?

I have the HAProxy frontend handling SSL negotiation using a certificate stored on the pfSense router. The certificate is acquired using the ACME package, configured to use Let's Encrypt DNS-01 protocol. This way we don't have to expose any ports when renewing our certificate. In this scenario, SSL offload is not from pfSense to Synology. It's the other way around. Our HAProxy frontend is listening on a custom port, negotiating the encrypted connection and forward traffic to the Synology backend which listening, unsecured, on a 5xxx port.

I have the exact same configuration working (using different ports) for our Docker repository, without checking the forward for box. It works fine. In that case the frontend port is 443 and the backend port is in the 5xxx range.

It appears I need to open a ticket with Synology to see if they can help me find logs. I'm also having trouble getting detailed logs from HAProxy on pfSense.

braunerroei

@bigtfromaz

In the matter af fact, I use the certificate stored on the pfSense, that was created by ACME.

-Roei

bigtfromaz

@braunerroei Then your frontend config looks like this?

That's the SSL Offloading I was talking about. If you don't check that box, then pfSense won't negotiate SSL. I was worried that you might be processing unencrypted.

In any event, I resolved my 503 problems. I'm not using the default port 443 for this new connection. Therefore, the value of the "Host Matches" ACL entry needed to be my.host.com:6407. I had used my.host.com with no port.

I had assumed that HAProxy would tack the port number on to the value because the port number is specified in the external address table. I see now it can't do that. The External Address table may contain multiple entries. It follows that the ACL match routine has no way to know your intentions unless you specify the port number in the ACL.

Thanks for the help. Your answers got me questioning my own configuration which turned out to be in error.