Transparent reverse proxy by HAProxy in 3-Legs scheme

IB

Hi,
I have one trouble with Haproxy. Situation:

1. pfSense server with 3 interfaces - LAN, WAN and DMZ.
2. LAN and DMZ segments in private IP addresses ranges, with routing rules between them. WAN has ONE white (real) IP-address.
3. Two servers with names S1 and S2 (for example) locates in DMZ. From LAN we must have access to them by many protocols - HTTP, SSH, SMB and other. So I want allow full access from LAN to S1 and S2. From Internet only HTTP access allowed.
4. Servers S1 and S2 must see real client IP (NOT for logging only).

What I do:
1. Create two records in local DNS for S1 and S2, pointed to local addresses in DMZ. Create two records in external DNS pointed to WAN address.
2. On pfsense allow full access from LAN to S1/S2
3. Setup HAProxy with listeners on WAN address port 80 and backends for S1 and S2 with name filtering.

All work fine except S1 and S2 see real client IP only for LAN users. For internet users - pfsense DMZ interface address.

When I turn on "transparent proxy" on backend settings - we can't connect from LAN by HTTP.

How I can provide transparence for internet users without problem for local users?

PiBa

Hi IB,

Thanks for asking in English makes it a bit easier to understand the topic you previously posted in the Russian channel :) ( https://forum.pfsense.org/index.php?topic=97907.0 ).

For getting the real client ip to the webserver behind a proxy some trickery is needed.. With haproxy this can be done in several ways:
1- X-Forwarded-For (header inserted into http traffic)
2- proxy-protocol
3- TransparentProxying (haproxy spoofs client-ip in the backend connection..)

Each has its own drawbacks..
1- webserver needs to be configured to read back this header and process it further in logging / authentication / other things.. its not for other software outside the webserver its not possible to easily determine the real client-ip and for example create a firewall rule on the webserver to block a certain client..
2- again the (web)server needs to be configured to accept this protocol, in theory other services could implement this protocol as well so it would not be limited to 'http'..
3- its the rather 'universal' solution and even works for example for smtp, it does require the (web)server to use haproxy as its default-gateway as haproxy will be 'spoofing' the clientip in its forwarded requests but must be in the path of the reply traffic..

As you have seen option 3 comes with a problem as warned in the webgui that it needs IPFW to 'capture' reply traffic from the destination server&port.. This ipfw rule affect all traffic from that destination also if the request never went through haproxy in the first place.. This causes the server to be unreachable directly from the local lan..

workarounds

1- configure the server to listen on 2 seperate ports or ipadresses.. this will allow haproxy to use the 'second' port so it can access the webserver over port 81 while the internal dns / clients will use the webserver port 80. This could have the drawback that the webserver reply will contain links that point to the port 81 while haproxy on the outside listens on 80.. A second ip address on the webserver with the normal port 80 could 'solve' that.. So that one ip is used for the proxyed traffic and the other ip is used for the internal traffic, which will not be affected by the ipfw rule.

2- configure a second frontend on the LAN of pfSense (on a ipalias ip) that also uses the wanted backend and let local dns point to that ip, so that traffic from the lan is also proxyed by haproxy.. This comes again with a problem that traffic from the dmz network itself trying to reach the server will likely also point to haproxy, and the solution of transparent proxying simply cannot ever work where client and server are on a single network..
–----------------------

I'm hoping divert-reply option in 'pf' would eventually remove this limitation, but it has not yet been committed into FreeBSD http://www.freebsd.org/cgi/query-pr.cgi?pr=188511

If anyone has another way to make this work properly that would be really nice, and i will if possible implement it into the haproxy pfsense package or suggest to the end-users where currently only the 'warning' is shown..

Best regards,
PiBa-NL

Marvho

Hi IB,

for what are you using HAProxy?

A bit off-topic, I'm just interested.

IB

Hi, PiBa

Solution from rubic work correctly: write NAT rule to redirect connection "from LAN to DMZ-server" to loopback, and add loopback to HAProxy listener.

PiBa

Hi IB,

Yes that should work to, it does make the connection go through haproxy kinda like it is with my #2 workaround. Only that dns can still point to the correct destination server ip, so in that regard it the workaround of Rubic works even better.

Thanks for reporting it back :D.

Regards,
PiBa-NL

IB

@PiBa:

Hi IB,

Yes that should work to, it does make the connection go through haproxy kinda like it is with my #2 workaround. Only that dns can still point to the correct destination server ip, so in that regard it the workaround of Rubic works even better.

Thanks for reporting it back :D.

Regards,
PiBa-NL

I have two DNS - internal with dmz-address for lan-users and external with external address for internet-users.