Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CARP VIP not reachable

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    26 Posts 2 Posters 12.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      surfdude
      last edited by

      Ok, the vlans have now the same order on both boxes (see above). But no effect.

      tcpdump -i vlan0 -n vrrp

      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on vlan0, link-type EN10MB (Ethernet), capture size 96 bytes
      16:18:24.486895 IP 192.168.50.1 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36
      16:18:25.487915 IP 192.168.50.1 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36
      16:18:26.488924 IP 192.168.50.1 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36
      16:18:27.489942 IP 192.168.50.1 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36

      tcpdump -i vlan1 -n vrrp

      [nothing]

      1 Reply Last reply Reply Quote 0
      • E
        Eugene
        last edited by

        tcpdump -i vlan1 -n vrrp on both boxes give nothing?
        what about
        tcpdump -i vlan2 -n vrrp

        http://ru.doc.pfsense.org

        1 Reply Last reply Reply Quote 0
        • S
          surfdude
          last edited by

          Yes, nothing on both. vlan2 is also nothing, but thats the SYNC interface

          1 Reply Last reply Reply Quote 0
          • E
            Eugene
            last edited by

            it's very very wird. Can you post

            
	 <interfaces>... and ...
	 <virtualip></virtualip></interfaces> 
```sections from your configs?

            http://ru.doc.pfsense.org

            1 Reply Last reply Reply Quote 0
            • S
              surfdude
              last edited by

              <interfaces><lan><if>vlan0</if>
              <ipaddr>192.168.50.1</ipaddr>
              <subnet>24</subnet>
              <media><mediaopt><bandwidth>100</bandwidth>
              <bandwidthtype>Mb</bandwidthtype></mediaopt></media></lan>
              <wan><if>fxp0</if>
              <mtu><media><mediaopt><bandwidth>100</bandwidth>
              <bandwidthtype>Mb</bandwidthtype>
              <spoofmac><disableftpproxy><ipaddr>192.168.1.200</ipaddr>
              <subnet>1</subnet>
              <gateway>192.168.1.1</gateway></disableftpproxy></spoofmac></mediaopt></media></mtu></wan>
              <opt1><descr>DMZ</descr>
              <if>vlan1</if>
              <bridge><ipaddr>192.168.51.1</ipaddr>
              <subnet>24</subnet>
              <gateway><spoofmac><mtu><enable></enable></mtu></spoofmac></gateway></bridge></opt1>
              <opt2><descr>SYNC</descr>
              <if>vlan2</if>
              <bridge><enable><ipaddr>192.168.49.1</ipaddr>
              <subnet>24</subnet>
              <gateway><spoofmac></spoofmac></gateway></enable></bridge></opt2>
              <opt3><descr>DMZExt</descr>
              <if>vlan3</if>
              <bridge><ipaddr>192.168.52.1</ipaddr>
              <subnet>24</subnet>
              <gateway><spoofmac></spoofmac></gateway></bridge></opt3></interfaces>

              <virtualip><vip><mode>carp</mode>
              <interface>lan</interface>
              <vhid>2</vhid>
              <advskew>0</advskew>
              <password>test</password>
              <descr><type>single</type>
              <subnet_bits>24</subnet_bits>
              <subnet>192.168.50.3</subnet></descr></vip>
              <vip><mode>carp</mode>
              <interface>opt1</interface>
              <vhid>8</vhid>
              <advskew>0</advskew>
              <password>test</password>
              <descr><type>single</type>
              <subnet_bits>24</subnet_bits>
              <subnet>192.168.51.3</subnet></descr></vip></virtualip>

              1 Reply Last reply Reply Quote 0
              • S
                surfdude
                last edited by

                pfsense 2

                <interfaces><lan><if>vlan0</if>
                <ipaddr>192.168.50.2</ipaddr>
                <subnet>24</subnet>
                <media><mediaopt><bandwidth>100</bandwidth>
                <bandwidthtype>Mb</bandwidthtype></mediaopt></media></lan>
                <wan><if>fxp0</if>
                <mtu><ipaddr>192.168.1.201</ipaddr>
                <subnet>1</subnet>
                <gateway>192.168.1.1</gateway>
                <disableftpproxy><dhcphostname><media><mediaopt><bandwidth>100</bandwidth>
                <bandwidthtype>Mb</bandwidthtype>
                <spoofmac></spoofmac></mediaopt></media></dhcphostname></disableftpproxy></mtu></wan>
                <opt1><descr>DMZ</descr>
                <if>vlan1</if>
                <bridge><enable><ipaddr>192.168.51.2</ipaddr>
                <subnet>24</subnet>
                <gateway><spoofmac></spoofmac></gateway></enable></bridge></opt1>
                <opt2><descr>SYNC</descr>
                <if>vlan2</if>
                <bridge><enable><ipaddr>192.168.49.2</ipaddr>
                <subnet>24</subnet>
                <gateway><spoofmac></spoofmac></gateway></enable></bridge></opt2>
                <opt3><descr>DMZExt</descr>
                <if>vlan3</if>
                <bridge><ipaddr>192.168.52.2</ipaddr>
                <subnet>24</subnet>
                <gateway><spoofmac></spoofmac></gateway></bridge></opt3></interfaces>

                <virtualip><vip><mode>carp</mode>
                <interface>lan</interface>
                <vhid>2</vhid>
                <advskew>100</advskew>
                <password>test</password>
                <descr><type>single</type>
                <subnet_bits>24</subnet_bits>
                <subnet>192.168.50.3</subnet></descr></vip>
                <vip><mode>carp</mode>
                <interface>opt1</interface>
                <vhid>8</vhid>
                <advskew>100</advskew>
                <password>test</password>
                <descr><type>single</type>
                <subnet_bits>24</subnet_bits>
                <subnet>192.168.51.3</subnet></descr></vip></virtualip>

                1 Reply Last reply Reply Quote 0
                • E
                  Eugene
                  last edited by

                  everything seems to be ok. If we forget about vip for the moment… can you ping real IP?

                  http://ru.doc.pfsense.org

                  1 Reply Last reply Reply Quote 0
                  • S
                    surfdude
                    last edited by

                    Now i'm getting crazy. I ran a ping for a few minutes and did add a new VIP for testing. While pressing the pfsense "Apply Changes" button, this happens:

                    ping 192.168.51.3

                    PING 192.168.51.3 (192.168.51.3): 56 data bytes
                    64 bytes from 192.168.51.3: icmp_seq=108 ttl=64 time=0.191 ms
                    64 bytes from 192.168.51.3: icmp_seq=109 ttl=64 time=0.101 ms
                    –- 192.168.51.3 ping statistics ---
                    115 packets transmitted, 2 packets received, 98.3% packet loss
                    round-trip min/avg/max/stddev = 0.101/0.146/0.191/0.045 ms

                    after this two packets, now its silence again. I can ping the real ip.

                    Thank you for hour help!

                    1 Reply Last reply Reply Quote 0
                    • E
                      Eugene
                      last edited by

                      may be stupid question but you have rule allowing icmp to this ip, right?
                      pfctl -sr | grep vlan1

                      http://ru.doc.pfsense.org

                      1 Reply Last reply Reply Quote 0
                      • S
                        surfdude
                        last edited by

                        I hope so!

                        pfctl -sr | grep vlan1

                        block drop in on ! vlan1 inet from 192.168.51.0/24 to any
                        block drop in on vlan1 inet6 from fe80::250:8bff:feec:84d0 to any
                        pass out quick on vlan1 all flags S/SA keep state label "let out anything from firewall host itself"
                        pass out quick on vlan1 proto icmp all keep state (tcp.closed 5) label "let out anything from firewall host itself"
                        pass out quick on vlan1 all flags S/SA keep state (tcp.closed 5) label "let out anything from firewall host itself"
                        pass in quick on vlan1 inet proto tcp from 192.168.51.0/24 to any port = domain flags S/SA keep state label "USER_RULE: DNS Requests"
                        pass in quick on vlan1 inet proto udp from 192.168.51.0/24 to any port = domain keep state label "USER_RULE: DNS Requests"
                        pass in quick on vlan1 inet proto tcp from 192.168.51.0/24 to any port = http flags S/SA keep state label "USER_RULE"
                        pass in quick on vlan1 inet from 192.168.51.0/24 to 192.168.51.0/24 flags S/SA keep state label "USER_RULE"
                        pass in quick on vlan1 all flags S/SA keep state label "USER_RULE"
                        pass in quick on vlan1 inet proto tcp from 192.168.51.0/24 to any port = https flags S/SA keep state label "USER_RULE"
                        pass in quick on vlan1 inet proto tcp from any to 127.0.0.1 port = 8022 flags S/SA keep state label "FTP PROXY: Allow traffic to localhost"
                        pass in quick on vlan1 inet proto tcp from any to 127.0.0.1 port = ftp flags S/SA keep state label "FTP PROXY: Allow traffic to localhost"

                        1 Reply Last reply Reply Quote 0
                        • E
                          Eugene
                          last edited by

                          Ok, sounds interesting. Can you please run tcpdump -i vlan1 -n vrrp on pfsense1 and click edit any vip and then save apply on pfsense1?

                          http://ru.doc.pfsense.org

                          1 Reply Last reply Reply Quote 0
                          • S
                            surfdude
                            last edited by

                            Still nothing.

                            But, if I execute this command on pfsense2, I get sometimes a packet:

                            /sbin/ifconfig carp1 192.168.51.3/24 vhid 8 advskew 100 pass test

                            1 Reply Last reply Reply Quote 0
                            • E
                              Eugene
                              last edited by

                              i think I run out of ideas… Your carp does not work.

                              1. pfsctl -sr | grep carp
                              2. Try swap nics fxp0 and fxp1.
                                Sorry

                              http://ru.doc.pfsense.org

                              1 Reply Last reply Reply Quote 0
                              • S
                                surfdude
                                last edited by

                                pfctl -sr | grep carp

                                anchor "carp" all
                                pass in quick on carp0 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server on LAN"
                                pass in quick on carp0 inet proto udp from any port = bootpc to 192.168.50.2 port = bootps keep state label "allow access to DHCP server on LAN"
                                pass out quick on carp0 inet proto udp from 192.168.50.2 port = bootps to any port = bootpc keep state label "allow access to DHCP server on LAN"
                                pass out quick on carp0 proto icmp all keep state label "let out anything from firewall host itself"
                                pass out quick on carp1 all flags S/SA keep state (tcp.closed 5) label "let out anything from firewall host itself"
                                pass out quick on carp2 all flags S/SA keep state (tcp.closed 5) label "let out anything from firewall host itself"
                                pass quick proto carp all keep state
                                pass in quick on carp2 all flags S/SA keep state label "USER_RULE: Trust the Firewall CARP net"
                                pass in quick on carp0 inet from 192.168.50.0/24 to any flags S/SA keep state label "USER_RULE: Default LAN - any"
                                pass in quick on carp1 all flags S/SA keep state label "USER_RULE"

                                Isn't here something wrong?

                                1 Reply Last reply Reply Quote 0
                                • E
                                  Eugene
                                  last edited by

                                  you have this: pass quick proto carp all keep state
                                  It means at least carp packets (vrrp) shouldn't have any restrictions.
                                  you should have the same (almost) dump on vlan1 as you have on WAN for vrrp.

                                  http://ru.doc.pfsense.org

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    surfdude
                                    last edited by

                                    <wan><if>fxp0</if>
                                            <mtu><ipaddr>192.168.1.201</ipaddr>
                                            <subnet>1</subnet>
                                      [..]

                                    I finally found the problem. Simple little configuration mistake, see above. I apologize for the trouble caused.</mtu></wan>

                                    1 Reply Last reply Reply Quote 0
                                    • NobleKangarooN NobleKangaroo referenced this topic on
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.