Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    BUG? CARP group failover of all devices not working

    2.1 Snapshot Feedback and Problems - RETIRED
    2
    3
    3.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      Reiner030
      last edited by

      Hi,

      it's hard to find post and pages with this topic…

      1. the BSD doc: http://www.freebsd.org/doc/en/books/handbook/carp.html
          ( http://books.google.de/books?id=43jomY21hIEC&pg=PA122&lpg=PA122&dq=net.inet.carp.preempt+net.inet.carp.suppress_preempt&source=bl&ots=GRkrKdSKsv&sig=FYjEC5hBlTW2evcYoakRBJRRhgU&hl=de&sa=X&ei=IzYdUeX6HMextAaR2ICIDA&ved=0CG4Q6AEwBg#v=onepage&q=net.inet.carp.preempt%20net.inet.carp.suppress_preempt&f=false )

      2. the BSD FAQ: http://www.openbsd.org/faq/pf/carp.html

      3. common CARP problem FAQ without this topic: http://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting

      4. a bug "misconfigured": http://redmine.pfsense.org/issues/1248
          but here is shown how sysctl values should be when a failover is done

      5. a bug "get help at forum/RTFM": http://redmine.pfsense.org/issues/1732

      6. a workaround which is not so nice to use: http://verb.bz/2008/08/31/forcing-carp-failover-using-pfsense/

      so ... after searching a long time no helpful things can be found ^^

      Let's see what we can do with the docu which was mostly helpful:
      http://www.openbsd.org/faq/pf/carp.html#carpex

      opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
              inet 192.168.6.251 netmask 0xffffff00
              carp: MASTER vhid 215 advbase 1 advskew 0

      opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
              inet 192.168.6.251 netmask 0xffffff00
              carp: BACKUP vhid 215 advbase 1 advskew 100

      => in compare of docu/FAQ the interface is not named carpX and there is no line with group info.
      But groups is defined (default: carp):

      [2.1-BETA1][root@gw1.zws8.local]/root(2): ifconfig -g carp
      pfsync0
      wan_vip211
      wan_vip212
      lan_vip213
      lan_vip214
      opt2_vip215

      And I had set under System => Advanced => System Tunables "net.inet.carp.preempt=1" on both machines:

      [2.1-BETA1][root@gw1.zws8.local]/root(3): sysctl net.inet.carp
      net.inet.carp.allow: 1
      net.inet.carp.preempt: 1
      net.inet.carp.log: 1
      net.inet.carp.arpbalance: 0
      net.inet.carp.suppress_preempt: 0

      [2.1-BETA1][root@gw2.zws8.local]/root(2): sysctl net.inet.carp
      net.inet.carp.allow: 1
      net.inet.carp.preempt: 1
      net.inet.carp.log: 1
      net.inet.carp.arpbalance: 0
      net.inet.carp.suppress_preempt: 0

      But when I drop VLAN on switch to simulate a network or when I do an interface shutdown like in docu described to test behavior, nothing changes in behavior:

      [2.1-BETA1][root@gw1.zws8.local]/root(4): ifconfig opt2_vip215 down
      [2.1-BETA1][root@gw1.zws8.local]/root(5): ifconfig opt2_vip215
      opt2_vip215: flags=8 <loopback>metric 0 mtu 1500
              inet 192.168.6.251 netmask 0xffffff00
              carp: INIT vhid 215 advbase 1 advskew 0
      [2.1-BETA1][root@gw1.zws8.local]/root(6): sysctl net.inet.carp
      net.inet.carp.allow: 1
      net.inet.carp.preempt: 1
      net.inet.carp.log: 1
      net.inet.carp.arpbalance: 0
      net.inet.carp.suppress_preempt: 0

      But only one device is switching:
      [2.1-BETA1][root@gw2.zws8.local]/root(4): ifconfig opt2_vip215
      opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
              inet 192.168.6.251 netmask 0xffffff00
              carp: MASTER vhid 215 advbase 1 advskew 100

      others stays backup like this:
      [2.1-BETA1][root@gw2.zws8.local]/root(5): ifconfig lan_vip214
      lan_vip214: flags=49 <up,loopback,running>metric 0 mtu 1500
              inet6 fe80::200:5eff:fe00:1d6%lan_vip214 prefixlen 64 scopeid 0xd
              inet6 xxx:xxx::fe prefixlen 48
              nd6 options=3 <performnud,accept_rtadv>carp: BACKUP vhid 214 advbase 1 advskew 100

      As in documentation / bug report 4) written the net.inet.carp.suppress_preempt counter should be incremented … but isn't :(

      So what is perhaps wrong configured or is it a real bug?

      Bests

      Reiner</performnud,accept_rtadv></up,loopback,running></up,loopback,running></loopback></up,loopback,running></up,loopback,running>

      1 Reply Last reply Reply Quote 0
      • E
        eri--
        last edited by

        You are marking the interface down which would not count in suppress_preempt.
        There are still some rough edges in general with carp but if you do trigger a linkdown through cable removal or somesuch it will switch.

        More improvements are expected in 2.1 for this since a lot of time has been spent to make carp stable enough to work the general situation that it work 98% of the time from our statistics.

        1 Reply Last reply Reply Quote 0
        • R
          Reiner030
          last edited by

          Hi,

          @ermal:

          You are marking the interface down which would not count in suppress_preempt.
          There are still some rough edges in general with carp but if you do trigger a linkdown through cable removal or somesuch it will switch.

          More improvements are expected in 2.1 for this since a lot of time has been spent to make carp stable enough to work the general situation that it work 98% of the time from our statistics.

          ah ok thx… Then I'm only wondering how the writer of these posts have done it with marking interface down :D
          But it works as expected when I'm disabling port on switch  - below documentation for other people.

          One question left:
          What happens if the gateway goes down (it's behind a switch, so "virtual" disconnection like the "interface down")?
          Is it possible to use some mechanism to combine Gateway failover and CARP failover as group between master-slave pfSense boxes if the slave has still connection to the gateway?

          Init state with Master/Slave:
          [2.1-BETA1][root@gw1.zws8.local]/root(1): ifconfig | grep -e mtu -e carp
          em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
          em3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          enc0: flags=0<> metric 0 mtu 1536
          pfsync0: flags=41 <up,running>metric 0 mtu 1460
          lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
          pflog0: flags=100 <promisc>metric 0 mtu 33664
          wan_vip211: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 211 advbase 1 advskew 0
          wan_vip212: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 212 advbase 1 advskew 0
          lan_vip213: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 213 advbase 1 advskew 0
          lan_vip214: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 214 advbase 1 advskew 0
          opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 215 advbase 1 advskew 0

          [2.1-BETA1][root@gw1.zws8.local]/root(2): sysctl net.inet.carp
          net.inet.carp.allow: 1
          net.inet.carp.preempt: 1
          net.inet.carp.log: 1
          net.inet.carp.arpbalance: 0
          net.inet.carp.suppress_preempt: 0

          Failover state (I've IPv4 & IPv6 CARP on same interface):
          [2.1-BETA1][root@gw1.zws8.local]/root(3): sysctl net.inet.carp
          net.inet.carp.allow: 1
          net.inet.carp.preempt: 1
          net.inet.carp.log: 1
          net.inet.carp.arpbalance: 0
          net.inet.carp.suppress_preempt: 2

          [2.1-BETA1][root@gw1.zws8.local]/root(4): ifconfig | grep -e mtu -e carp
          em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
          em3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          enc0: flags=0<> metric 0 mtu 1536
          pfsync0: flags=41 <up,running>metric 0 mtu 1460
          lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
          pflog0: flags=100 <promisc>metric 0 mtu 33664
          wan_vip211: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: BACKUP vhid 211 advbase 1 advskew 0
          wan_vip212: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: BACKUP vhid 212 advbase 1 advskew 0
          lan_vip213: flags=8 <loopback>metric 0 mtu 1500
          carp: INIT vhid 213 advbase 1 advskew 0
          lan_vip214: flags=8 <loopback>metric 0 mtu 1500
          carp: INIT vhid 214 advbase 1 advskew 0
          opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: BACKUP vhid 215 advbase 1 advskew 0

          And after falling back all normal again:
          [2.1-BETA1][root@gw1.zws8.local]/root(6): ifconfig | grep -e mtu -e carp
          em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
          em3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
          enc0: flags=0<> metric 0 mtu 1536
          pfsync0: flags=41 <up,running>metric 0 mtu 1460
          lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
          pflog0: flags=100 <promisc>metric 0 mtu 33664
          wan_vip211: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 211 advbase 1 advskew 0
          wan_vip212: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 212 advbase 1 advskew 0
          lan_vip213: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 213 advbase 1 advskew 0
          lan_vip214: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 214 advbase 1 advskew 0
          opt2_vip215: flags=49 <up,loopback,running>metric 0 mtu 1500
          carp: MASTER vhid 215 advbase 1 advskew 0

          [2.1-BETA1][root@gw1.zws8.local]/root(7): sysctl net.inet.carp
          net.inet.carp.allow: 1
          net.inet.carp.preempt: 1
          net.inet.carp.log: 1
          net.inet.carp.arpbalance: 0
          net.inet.carp.suppress_preempt: 0

          Bests

          Reiner</up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></promisc></up,loopback,running,multicast></up,running></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></loopback></loopback></up,loopback,running></up,loopback,running></promisc></up,loopback,running,multicast></up,running></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></promisc></up,loopback,running,multicast></up,running></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,promisc,simplex,multicast>

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.