Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HAProxy TCP/use client ip and carp cluster problem

    Cache/Proxy
    2
    16
    2.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      PiBa @brlamnr
      last edited by

      @brlamnr
      You do have the vSwitch of ESX configured to allow spoofing and promiscuous mode? Im not even sure packets should ever be using the mac of vmware host itself.. Or do you have the real hardware nic's passed through to the pfSense VM? Ive never done that..

      B 1 Reply Last reply Reply Quote 0
      • B
        brlamnr @PiBa
        last edited by

        @piba said in HAProxy TCP/use client ip and carp cluster problem:

        @brlamnr
        You do have the vSwitch of ESX configured to allow spoofing and promiscuous mode? Im not even sure packets should ever be using the mac of vmware host itself.. Or do you have the real hardware nic's passed through to the pfSense VM? Ive never done that..

        That's a good point, while preparing to deploy the virtual pfsense that will eventually replace the ones above, I read about the vswitch requirements, I didn't think about it since the ones with the problem are physical appliances connected to a physical switch, but, you never know. I will ask to have those settings enabled in the vmswitch (or port group) where the servers connect to. Will report back.

        Thanks.

        P 1 Reply Last reply Reply Quote 0
        • P
          PiBa @brlamnr
          last edited by

          @brlamnr
          Only the pfSense VM that is using the carp ip would need to have such special vSwitch 'permissions'.. The webservers should not need it.. If they are currently still on hardware that should not be required.. I'm kinda running low on ideas though..

          B 1 Reply Last reply Reply Quote 0
          • B
            brlamnr @PiBa
            last edited by

            @piba said in HAProxy TCP/use client ip and carp cluster problem:

            @brlamnr
            Only the pfSense VM that is using the carp ip would need to have such special vSwitch 'permissions'.. The webservers should not need it.. If they are currently still on hardware that should not be required.. I'm kinda running low on ideas though..

            The servers are actually virtual servers, the pfsense are physical appliances, I'll give it a try tomorrow anyway, nothing to lose.

            B 1 Reply Last reply Reply Quote 0
            • B
              brlamnr @brlamnr
              last edited by

              @brlamnr said in HAProxy TCP/use client ip and carp cluster problem:

              @piba said in HAProxy TCP/use client ip and carp cluster problem:

              @brlamnr
              Only the pfSense VM that is using the carp ip would need to have such special vSwitch 'permissions'.. The webservers should not need it.. If they are currently still on hardware that should not be required.. I'm kinda running low on ideas though..

              The servers are actually virtual servers, the pfsense are physical appliances, I'll give it a try tomorrow anyway, nothing to lose.

              The changes on the vswitch didn't make any difference. As soon as client-ip is turned on, the client stops seeing the server.

              1 Reply Last reply Reply Quote 0
              • P
                PiBa
                last edited by

                the 'real' clients connect to haproxy, and that connection is likely still working properly.. as nothing changes on that side when enabling use-client-ip on haproxy for the backend connection.
                Haproxy however probably nolonger sees the reply from the server.. which is strange.. as you see them in the packet captures...

                Do you have any plugins like suricata/snort running? Or do you use the captive-portal which also uses ipfw for some 'low level' firewall tasks..

                B 1 Reply Last reply Reply Quote 0
                • B
                  brlamnr @PiBa
                  last edited by

                  @piba said in HAProxy TCP/use client ip and carp cluster problem:

                  the 'real' clients connect to haproxy, and that connection is likely still working properly.. as nothing changes on that side when enabling use-client-ip on haproxy for the backend connection.
                  Haproxy however probably nolonger sees the reply from the server.. which is strange.. as you see them in the packet captures...

                  Do you have any plugins like suricata/snort running? Or do you use the captive-portal which also uses ipfw for some 'low level' firewall tasks..

                  No, there are no plugins nor captive portals. The appliance was configured to do load balancing only.

                  P 1 Reply Last reply Reply Quote 0
                  • P
                    PiBa @brlamnr
                    last edited by

                    @brlamnr
                    Can you check result of command 'ipfw show' ?

                    B 1 Reply Last reply Reply Quote 0
                    • B
                      brlamnr @PiBa
                      last edited by

                      @piba said in HAProxy TCP/use client ip and carp cluster problem:

                      @brlamnr
                      Can you check result of command 'ipfw show' ?

                      Following after activating client-ip:

                      00010 0 0 fwd ::1 tcp from 10.3.128.10 443 to any in recv cxl0.79
                      00011 108 20412 fwd ::1 tcp from 10.3.128.11 443 to any in recv cxl0.79
                      65535 48732381 4651172490 allow ip from any to any

                      P B 2 Replies Last reply Reply Quote 0
                      • P
                        PiBa @brlamnr
                        last edited by PiBa

                        @brlamnr
                        hmm it looks like that has IPv6 and IPv4 mixed together..

                        Mine currently look like:

                        00012           0              0 fwd 127.0.0.1 tcp from 192.168.8.15 444 to any in recv em1

                        Maybe there lies the root cause..
                        Can you try and add a rule manually?:

                        ipfw add 50 fwd 127.0.0.1 tcp from 10.3.128.11 443 to any in recv cxl0.79

                        even though the rule is counting traffic.. it 'seems' to work..

                        1 Reply Last reply Reply Quote 0
                        • B
                          brlamnr @brlamnr
                          last edited by

                          @brlamnr said in HAProxy TCP/use client ip and carp cluster problem:

                          @piba said in HAProxy TCP/use client ip and carp cluster problem:

                          @brlamnr
                          Can you check result of command 'ipfw show' ?

                          Following after activating client-ip:

                          00010 0 0 fwd ::1 tcp from 10.3.128.10 443 to any in recv cxl0.79
                          00011 108 20412 fwd ::1 tcp from 10.3.128.11 443 to any in recv cxl0.79
                          65535 48732381 4651172490 allow ip from any to any

                          It didn't work. Same behavior. Thanks.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.