HAProxy TCP mode has problem with option Transparent ClientIP enabled

starless_boi

Hi every one,
I have pfsense installed and configured to use HAProxy
In a simple architecture, I have one public IP as WAN address, and a local network configured with LAN address that are in the same network with two servers which traffic should be balanced between them.
this model works when the option Transparent ClientIP is not enabled, I mean incoming traffic from WAN is balanced between two servers in LAN network and I can see results, but when I enable above feature, nothing works, and incoming traffic doesn't reach to internal servers (I need this feature!).
What should I do to route traffic to it's correct way. I did everything I could, but didn't make it!
BTW I use pfsense version 2.2.1, and HAProxy-devel 1.5.11 pkg v 0.23
thanks

PiBa

You are testing from "outside" ?

As a workstation on the lan trying to connect to a webserver on that same lan will not work because the webserver reply would be sent directly to the client, which rejects that traffic..

Edit:
Also those webservers do have pfsense set as their default-route?

starless_boi

first thanks for your answer

yeah testing from internet, trying to use IP that has been set on my pfsense as WAN address
pfsense installed on a ESXi as virtual machine, with one network adapter added and set as wan network, with a public IP, and another network adapter as Lan address (this adapter is in the same network of two servers that will handle requests coming from internet)
this is scenario, clients send request to my WAN address (set as ip address my HAProxy listens on) with a specific port that has been opened on my pfsense firewall. this request should be forwarded to my servers that will handle the request and send back the result to client
when feature Transparent ClientIP is disabled, everything works and requests are answered correctly,but when Transparent ClientIP is enabled and Interface set to LAN, no traffic reaches my two servers that handle requests

how can I see what is default route on Windows server? and if this is problem,why everything works when Transparent ClientIP feature is disabled?

PiBa

default-route and default-gateway are basically the same kind of thing.. Anyway running "route print -4" on a cmd screen will show you configured routes.. There should be a route for 0.0.0.0 with netmask 0.0.0.0 that uses pfsense-ip as the gateway..

As for that it works without the transparent-client-ip setting then the traffic originates from the lan-ip of pfsense where haproxy builds the connection from. That way the traffic seems 'local' to the network, which makes it easy for the servers to know where to send the reply to..

Another option is that the windows-firewall is not allowing traffic from 'client-ips' / edge-traversal and is blocking traffic.

The windows server machines only have 1 network-connection right?

Perhaps you could run a the following on pfsense to see if traffic is send out on lan and if a reply comes back if any..

tcpdump -en -i em1 "host 192.168.10.20 && port 80"
```(adjust interface ip and port to your environment)

starless_boi

I changed the default route to what you said

I checked, ports were open on firewall, also turned Off windows firewall, but no luck

Yeah,there are just one network adapter added to windows servers

this is output of command you gave:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
capability mode sandbox enabled
19:15:37.525261 00:50:56:b9:3b:8e > 00:50:56:b9:78:c6, ethertype IPv4 (0x0800), length 74: 212.16.95.133.6202 > 192.168.128.143.5222: Flags , seq 2309272852, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1424972 ecr 0], length 0
19:15:40.718878 00:50:56:b9:3b:8e > 00:50:56:b9:78:c6, ethertype IPv4 (0x0800), length 74: 212.16.95.133.6202 > 192.168.128.143.5222: Flags , seq 2309272852, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1428172 ecr 0], length 0
19:15:41.198832 00:50:56:b9:3b:8e > 00:50:56:b9:78:c6, ethertype IPv4 (0x0800), length 74: 212.16.95.133.29009 > 192.168.128.143.5222: Flags ~~, seq 561606561, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1428652 ecr 0], length 0~~

PiBa

The tcpdump output is a bit mangled, you should probably have put it in a 'code' # block using the edit buttons above..

Anyway i assume all 3 packets send to the windows server are using tcp flags [ S ] right? That would mean haproxy is properly sending a connection request 'S' syn to the server. To which it should reply with 'S.' syn-ack.

Perhaps you could test, if you dont use haproxy but use a simple portforward you do you see the same issue.?. I do expect so..

Anyway it seems the windows server is not responding.. Can you show output from 'route print -4' on the server?

starless_boi

Hi
Thank you PiBa, you were right, the problem was windows servers, that did not return the traffic, I talked to our windows and network administrator, and they corrected the routing.
now everything is working.
thanks