Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    high availability w/ redundant layer 2 switches causing loop on my test network

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    19 Posts 4 Posters 1.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • se_marcS
      se_marc
      last edited by

      hey everyone, I'm attempting to test High availability with two layer 2 redundant switches and I'm causing a loop somehow. My lab network is set up a bit different than the official netgate diagram (see second diagrams).

      • Is Unmanaged Switch A from the second diagram causing the loop or might it be the unifi 8 port PoE managed switch? My gut is telling me its the managed switch.

      • The managed switch has RSTP enabled.

      • When I plug the cable between Unmanaged Switch B and Unmanaged Switch C, the loop occurs and my 192.168.1.0/24 network loses local connectivity as well as internet.

      • When I unplug the cable, 192.168.1.0/24 regains connectivity to both local and internet.

      • The reason I am using the managed switch is so VLAN 100 can act as an ISP. I am using a VLAN due to lack of ports on my EdgeRouter 4 for a second physical network as a simulated ISP

      Is there a way I can prevent the loop from happening with my current setup, maybe creating a rule that segregates each LAN from each other?

      Official Netgate Diagram:

      alt text

      My full network (minus connected clients):

      alt text

      johnpozJ 1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator @se_marc
        last edited by johnpoz

        @se_marc said in high availability w/ redundant layer 2 switches causing loop on my test network:

        When I plug the cable between Unmanaged Switch B and Unmanaged Switch C, the loop occurs

        Well Duh! ;)

        I do not know of any unmanaged switch that supports STP.. So yeah you have a loop.. How would some upstream or downstream managed switch fix that?

        This will not work with unmanaged switches - you will have a loop

        borked.png

        If you want to do that - get managed/smart switches that support spanning tree so the switches can shut down the redundant path(s) when not needed. Ie no loop.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.7.2, 24.11

        se_marcS 1 Reply Last reply Reply Quote 1
        • se_marcS
          se_marc @johnpoz
          last edited by

          @johnpoz much appreciated.

          so it sounds like i'll need to free up a port on my EdgeRouter4 move my VLAN to that port? This will allow my to remove the managed switch from my diagram completely.

          With that being said, I still see the same loop. if STP is non-existent and the only network at play is 10.10.1.0/24, will the setup below work?

          alt text

          johnpozJ 1 Reply Last reply Reply Quote 0
          • johnpozJ
            johnpoz LAYER 8 Global Moderator @se_marc
            last edited by johnpoz

            No you can not connected dumb switches like that.. Just pick up some smart 5 ports.. You can find them for like $25-40 for sure..

            Or juts replace all 3 of those with 1 smart switch ;)

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 24.11 | Lab VMs 2.7.2, 24.11

            se_marcS 2 Replies Last reply Reply Quote 0
            • se_marcS
              se_marc @johnpoz
              last edited by se_marc

              @johnpoz okay so, i probably should just ask what i actually need to know without the whole lab network. Can I set up HA with 2 ISPs and redundant switches? This is how I imagine it would need to be setup. Do you see any issues with this? Since I will be using the switches directly connected to the ISP modem, I feel like I can use dumb switches.

              According to this documentation: "When using multiple switches, the switches should be interconnected. As long as there is a single connection between the two switches, and no bridge on either of the firewalls, this is safe with any type of switch."

              alt text

              1 Reply Last reply Reply Quote 0
              • se_marcS
                se_marc @johnpoz
                last edited by

                @johnpoz im trying to combine THIS and THIS

                johnpozJ 1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator @se_marc
                  last edited by

                  that is not what you have.. You have dumb switch A..

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                  se_marcS 1 Reply Last reply Reply Quote 0
                  • se_marcS
                    se_marc @johnpoz
                    last edited by

                    @johnpoz i don't have an actual second ISP to test with so I was using LANs as my two "ISPs". I will get some managed switches to test on my lab internally but would the most recent diagram I posted work if i actually have 2 ISPs and keep the dumb switches?

                    johnpozJ 1 Reply Last reply Reply Quote 0
                    • johnpozJ
                      johnpoz LAYER 8 Global Moderator @se_marc
                      last edited by

                      You can keep the dumb switches, but you have dumb switch A connecting them, and then you connect them together again with a cable = LOOP!

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                      se_marcS 1 Reply Last reply Reply Quote 0
                      • se_marcS
                        se_marc @johnpoz
                        last edited by

                        @johnpoz sorry, i'm a bit confused. i didn't include dumb switch A in the most recent diagram.

                        DerelictD 1 Reply Last reply Reply Quote 0
                        • DerelictD
                          Derelict LAYER 8 Netgate @se_marc
                          last edited by

                          @se_marc This is how you would do it.

                          This cannot be done with "cheap" gear.

                          Some gear that can do this that used to be expensive is now not very expensive on eBay, etc.

                          HA+LACP.png

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          1 Reply Last reply Reply Quote 0
                          • johnpozJ
                            johnpoz LAYER 8 Global Moderator
                            last edited by johnpoz

                            @se_marc said in high availability w/ redundant layer 2 switches causing loop on my test network:

                            i didn't include dumb switch A in the most recent diagram.

                            My point was what you posted as how to set it up, what not what you had before.

                            A dumb switch is not a ISP device that would not bridge the connections together..

                            Your drawing showing 2 isp, you can not replace that isp device with a dumb switch - which becomes really a wire.. And now you have 2 wires connecting 2 dumb switches - big fat loop!

                            To derelicts point - what your trying to do with a HA setup is not meant to be done with cheap dumb home gear. If you want to play with a lab to do this sort of stuff, you need equipment that can do a min amount of enterprise level sort of stuff. Which stp is common on even the cheapest of smart switches.

                            Or yeah get on ebay and get some enterprise gear for cheap..

                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                            If you get confused: Listen to the Music Play
                            Please don't Chat/PM me for help, unless mod related
                            SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                            se_marcS 1 Reply Last reply Reply Quote 0
                            • se_marcS
                              se_marc @johnpoz
                              last edited by se_marc

                              @johnpoz @Derelict I haven't given up just yet lol. I took some time to think things over and I feel like I'm getting closer. The managed switches will be ubiquiti so RSTP will be enabled. To my understanding that means ports 3 & 4 on Managed Switches B and D will be blocked unless something happens the the either ports 5 & 6 on Managed Switch A or one of the WAN ports on pfSense Main. same would go for the second ISP. Would i need to connect Managed Switch A to Managed Switch B, and then C to D?

                              HA Fault Tolerance LAGG A (1).png

                              johnpozJ 1 Reply Last reply Reply Quote 0
                              • johnpozJ
                                johnpoz LAYER 8 Global Moderator @se_marc
                                last edited by johnpoz

                                Trying to lagg into 2 disconnected switches not something that is normally an option. When you lagg into multiple switches like you have it draw. The switches are part of stack.

                                Are your A and B, and then C,D stacked?

                                Notice the drawing Derelict posted.

                                An intelligent man is sometimes forced to be drunk to spend time with his fools
                                If you get confused: Listen to the Music Play
                                Please don't Chat/PM me for help, unless mod related
                                SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                se_marcS 1 Reply Last reply Reply Quote 0
                                • se_marcS
                                  se_marc @johnpoz
                                  last edited by

                                  @johnpoz crud, to my knowledge unifi switches do not support official stacking but a common trend is to connect them together via SFP ports and then relying on RSTP. This causes half speed of the LAGG router ports.

                                  johnpozJ 1 Reply Last reply Reply Quote 0
                                  • johnpozJ
                                    johnpoz LAYER 8 Global Moderator @se_marc
                                    last edited by

                                    Then its not really a lagg now is it ;) hehehe

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                    DerelictD 1 Reply Last reply Reply Quote 0
                                    • DerelictD
                                      Derelict LAYER 8 Netgate @johnpoz
                                      last edited by

                                      People call all sorts of things a "lagg."

                                      I consider everything except proper LACP which generally requires either stacking or multi-chassis trunking, etc.

                                      Chattanooga, Tennessee, USA
                                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                      johnpozJ 1 Reply Last reply Reply Quote 0
                                      • AndyRHA
                                        AndyRH
                                        last edited by AndyRH

                                        I wanted to play a network guy on TV, so I bought one of these on eBay. It will do everything in your drawing in one switch. Great for play time.

                                        Aruba S2500-48P-US <- has 10Gb SFP+ ports, POE, VLANs, semi-loud fans... All the enterprise features for under $200.

                                        o||||o
                                        7100-1u

                                        1 Reply Last reply Reply Quote 0
                                        • johnpozJ
                                          johnpoz LAYER 8 Global Moderator @Derelict
                                          last edited by

                                          @derelict said in high availability w/ redundant layer 2 switches causing loop on my test network:

                                          People call all sorts of things a "lagg."

                                          Very true - its a kind of a catch all.. I was thinking lacp, which yeah you need a stack..

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.