[SOLVED] Problem with haproxy and Transparent-Client-IP
-
Hi,
i have enabled the Transparent-Client-IP option in the haproxy backend section. What i can see in the logs of the app is that the src ip of the client is set correctly, but there seems to be a problem with the routing of the traffic back to the client. I see a lot of TCP:SA messages in the firewall log of the pfsense.
Setup:
clientip (curl) -> pfsense (WAN Vip) -> haproxy (Listen on WAN Vip) -> haproxy Backends (Lan) -> app
Tcpdump on the the app server:
20:07:26.657444 IP clientip.36930 > appip.https: Flags [s], seq 924252362, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 3426871208 ecr 0], length 0[/[/code] Traffic arrived with the right ip on the app. Log of the pfsense: [code]Jan 10 19:03:24 ► WAN WANIP:28357 clientip:32809 TCP:SA[/code] So for the traffic back to the client the firewall use the wrong wan address. Do i have to insert or delete nat rule ? With the tcp or http mode without the Transparent-Client-IP setting everything work as expected. Thanks in advance Marc [/s] -
But tcpdump on the appserver doesnt show the SynAck ? Does the appserver have multiple nic's and perhaps replying out the wrong one?
-
I forgot to add the syn/ack line:
10:56:00.679643 IP clientip.48180 > appip.https: Flags [s], seq 4080321817, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 3480185230 ecr 0], length 0 10:56:00.679715 IP appip.https > clientip.48180: Flags [S.], seq 2993922115, ack 4080321818, win 28960, options [mss 1460,sackOK,TS val 90865184 ecr 3480185230,nop,wscale 7], length 0 also the appserver has only one interface and as default gw the pfsense. what i don't see is the ack from the client. What i can see on the pfsense: [code]10:06:13.138605 IP clientip.33344 > pfsenseip.443: Flags [s], seq 1444608814, win 29200, options [mss 1460,sackOK,TS val 63209798 ecr 0,nop,wscale 7], length 0 10:06:13.138651 IP pfsenseip.443 > clientip.33344: Flags [S.], seq 1101463738, ack 1444608815, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2978566388 ecr 63209798], length 0 10:06:13.142968 IP clientip.33344 > pfsenseip.443: Flags [.], ack 1, win 229, options [nop,nop,TS val 63209803 ecr 2978566388], length 0 10:06:13.143666 IP clientip.33344 > pfsenseip.443: Flags [P.], seq 1:297, ack 1, win 229, options [nop,nop,TS val 63209803 ecr 2978566388], length 296 10:06:13.143680 IP pfsenseip.443 > clientip.33344: Flags [.], ack 297, win 518, options [nop,nop,TS val 2978566393 ecr 63209803], length 0 10:06:15.144976 IP clientip.33344 > pfsenseip.443: Flags [F.], seq 297, ack 1, win 229, options [nop,nop,TS val 63211805 ecr 2978566393], length 0 10:06:15.145013 IP pfsenseip.443 > clientip.33344: Flags [.], ack 298, win 520, options [nop,nop,TS val 2978568394 ecr 63211805], length 0 Also there i can see no ack. Routing on the app server: default via internal_vip_ip_on_pfsense dev bond Haproxy is listen on a wan ip and connect to the backends via internal network. [/s][/code][/s] -
Ok it was my own fault. >:(
I used dns entries in the backend section and that don't work with the ipfw module in the pfsense, so there wasn't a catch all rule.
-
Ah that explains it indeed, i should probably check and prevent that combination. Or try and 'fix' the ipfw rule by resolving the backserver ips.. Still tricky if it changes though which haproxy itself could do with the dnsresolvers setting.