Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Failover: WAN Master / LAN Backup on Secondary

    HA/CARP/VIPs
    2
    8
    2.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kerplunk
      last edited by

      I've configured CARP using the pfSense Book instructions as a guide. If I pull the WAN cable from the Primary unit to test failover (as suggested in the pfSense book as a test), the Secondary unit WAN VIP will become Master, but the Secondary LAN VIP will stay Backup. My expectation was that the Secondary LAN VIP would also become Master so that traffic would flow through the Secondary unit. I can get the Secondary LAN VIP to become Master if I pull the LAN cable as well from the Primary unit or cause a full shutdown or disable of CARP on Primary unit.

      Is the behavior that I am observing expected? I'd ideally like to see a single port failure on the Primary cause the Secondary to gain Master state on both the WAN and LAN VIPs so that traffic flow will not be interrupted.

      The basic configuration on an SG-2440 running 2.3.2-RELEASE-p1 is as follows:

      Public IP/28 (CARP VIP, VHID 73)
                                                                      |
                                                              WAN switch
                  |–-------------------------------------|---------------------------------------|
      192.168.40.1/28                                                                              192.168.40.2/28
              WAN                                                                                                WAN

      Primary    OPT1(192.168.50.1/24) --------- (192.168.50.2/24)OPT1    Secondary

      LAN                                                                                                  LAN
      172.27.40.2/24                                                                                  172.27.40.3/24
                  |---------------------------------------|---------------------------------------|
                                                              LAN switch           
                                                                      |
                                            172.27.40.1/24 (CARP VIP, VHID 1)

      Any guidance on this is appreciated.

      1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate
        last edited by

        What is in the system log when you do this test?

        All CARP VIPs should get their advskew raised by 240 by default for every interface with a CARP VIP that goes into a down state. That should be enough to make all VIPs on the secondary become master in a default 1/0 1/100 advbase/advskew configuration.

        Nov 16 04:21:20 kernel carp: 237@xn2: MASTER -> INIT (hardware interface down)
        Nov 16 04:21:20 kernel carp: demoted by 240 to 240 (interface down)
        Nov 16 04:21:20 kernel xn2: link state changed to DOWN
        Nov 16 04:21:20 kernel carp: 228@xn1: MASTER -> BACKUP (more frequent advertisement received)
        Nov 16 04:21:20 kernel carp: 236@xn0: MASTER -> BACKUP (more frequent advertisement received)

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        1 Reply Last reply Reply Quote 0
        • K
          kerplunk
          last edited by

          Here's what I see in the logs. In this case, I pulled the LAN cable out of the Primary. On the WAN VIP, I can see that at least for a moment, the Secondary did become Master, but then the Primary promptly became the Master.

          Primary Log:

          Nov 16 22:53:53 check_reload_status Carp backup event
          Nov 16 22:53:53 kernel carp: demoted by 240 to 240 (interface down)
          Nov 16 22:53:53 kernel igb1: link state changed to DOWN
          Nov 16 22:53:53 kernel carp: VHID 73@igb0: MASTER -> BACKUP (more frequent advertisement received)
          Nov 16 22:53:53 check_reload_status Linkup starting igb1
          Nov 16 22:53:53 check_reload_status Carp backup event
          Nov 16 22:53:54 php-fpm 97843 /rc.carpbackup: HA cluster member "(172.27.40.1@igb1): (LAN)" has resumed CARP state "BACKUP" for vhid 1
          Nov 16 22:53:54 php-fpm 97843 /rc.linkup: DEVD Ethernet detached event for lan
          Nov 16 22:53:54 check_reload_status Reloading filter
          Nov 16 22:53:54 kernel carp: demoted by -240 to 0 (vhid removed)
          Nov 16 22:53:54 kernel igb1: promiscuous mode disabled
          Nov 16 22:53:54 php-fpm 97843 /rc.carpbackup: HA cluster member "(x.x.x.x@igb0): (WAN)" has resumed CARP state "BACKUP" for vhid 73
          Nov 16 22:53:55 check_reload_status Carp master event
          Nov 16 22:53:55 kernel carp: VHID 73@igb0: BACKUP -> MASTER (preempting a slower master)
          Nov 16 22:53:55 xinetd 17605 Starting reconfiguration
          Nov 16 22:53:55 xinetd 17605 Swapping defaults
          Nov 16 22:53:55 xinetd 17605 readjusting service 6969-udp
          Nov 16 22:53:55 xinetd 17605 Reconfigured: new=0 old=1 dropped=0 (services)
          Nov 16 22:53:56 php-fpm 97843 /rc.carpmaster: HA cluster member "(x.x.x.x@igb0): (WAN)" has resumed CARP state "MASTER" for vhid 73

          Secondary Log:

          Nov 16 22:53:52 check_reload_status Carp master event
          Nov 16 22:53:52 kernel carp: VHID 73@igb0: BACKUP -> MASTER (preempting a slower master)
          Nov 16 22:53:53 php-fpm 33089 /rc.carpmaster: HA cluster member "(x.x.x.x@igb0): (WAN)" has resumed CARP state "MASTER" for vhid 73
          Nov 16 22:53:54 check_reload_status Carp backup event
          Nov 16 22:53:54 kernel carp: VHID 73@igb0: MASTER -> BACKUP (more frequent advertisement received)
          Nov 16 22:53:55 check_reload_status Carp master event
          Nov 16 22:53:55 kernel carp: VHID 1@igb1: BACKUP -> MASTER (master down)
          Nov 16 22:53:55 php-fpm 33089 /rc.carpbackup: HA cluster member "(x.x.x.x@igb0): (WAN)" has resumed CARP state "BACKUP" for vhid 73
          Nov 16 22:53:56 php-fpm 33089 /rc.carpmaster: HA cluster member "(172.27.40.1@igb1): (LAN)" has resumed CARP state "MASTER" for vhid 1

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            Nov 16 22:53:53    kernel      carp: demoted by 240 to 240 (interface down)
            Nov 16 22:53:54    kernel      carp: demoted by -240 to 0 (vhid removed)

            Hmm. demotes then apparently the vhid is removed from the interface so no more interface with a CARP VIP down so no more demotion. I think I can dig up a couple 2440s. Have to test it physical.

            If you feel like it take a backup config.xml from the primary (without rrd) and paste it into a PM so I can test your config more accurately.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate
              last edited by

              Are you trying to cheat with private addresses on the interfaces and one public CARP VIP?

              That is not how it is done in the book, and is not a supported configuration, even though some appear to have gotten it to basically work.

              Please do it right with public IP addresses on the interfaces out of the same WAN /28 and see how it goes.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • K
                kerplunk
                last edited by

                Yes, I am using private address space for the WAN interfaces with a public VIP. I'm not certain whether I can obtain additional public IPs, but will see what I can do. That's too bad that this isn't a supported scenario as I imagine obtaining multiple public IPs may present additional complications for some situations.

                1 Reply Last reply Reply Quote 0
                • DerelictD
                  Derelict LAYER 8 Netgate
                  last edited by

                  Found it.

                  Set your LAN IPv6 to None. It's set to Track Interface.

                  Seems like the logic there could be better but CARP/HA is not compatible with dynamic addressing on any CARP interface so setting to none there fixes what you're seeing.

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  1 Reply Last reply Reply Quote 0
                  • K
                    kerplunk
                    last edited by

                    I believe this may have been the solution. I seem to be able to pull the WAN or LAN cables from Primary now and get a full failover.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.