VLANs having same mac address causing flapping error on cisco switch



  • Host 0008.a2**.**b2 in vlan 1 is flapping between port Po2 and port Po1
    Host 0008.a2**.**b2 in vlan 1 is flapping between port Po1 and port Po2

    I would assume this is because each one of my vlans on pfsense reports as having the same mac, thus causing the cisco switch to think that the host is flapping between channels, when in reality all the interfaces have the same mac?



  • @erasedhammer

    Well yeah, all VLANs from the same interface will have the same MAC. What is it you're trying to do?


  • LAYER 8 Netgate

    The MAC address table (CAM table) is segregated between VLANs (broadcast domains)

    It is absolutely not a problem to have the same MAC address from a device on different VLANs on the same properly-configured switch port.

    In fact, that's pretty much how the world works.



  • @Derelict said in VLANs having same mac address causing flapping error on cisco switch:

    It is absolutely not a problem to have the same MAC address from a device on different VLANs on the same properly-configured switch port.

    The OP's post implies the switch thinks the MAC is on 2 different ports and that shouldn't happen, unless they're doing something strange.



  • What im trying to do is have each vlan on the fw have its own group of physical ports for lagg.
    I have the xg-7100 so it has that marvell integrated switch.
    If I run an ifconfig on the fw it shows all the vlans as having the same MAC (and every port on the marvell switch as having the same mac)
    Now if I have each vlan with its own channel group on the switch, and the switch sees the same MAC on every lagg group, it sends the flapping error message.

    I am not 100% whats happening in detail, but I do know when I went in and changed the mac addr for each vlan using ifconfig lladdr, the flapping errors stopped.



  • @erasedhammer

    Perhaps you could provide a sketch, as I'm having a WTF? moment.

    How many ports are you using on pfSense for the VLANs? If only 1, then every VLAN that uses that interface will have same MAC. There's no way around that, as it's determined by the hardware. What comes after psSense?



  • You generally see this type of issue in a HA setup, i've seen it many times with an all Cisco setup and HSRP.

    Are you running a HA pair, if you are there probably is a connectivity issue between the pair.



  • I am using 7 ports on the marvell switch on the xg7100 connected to my cisco 3560.
    eth2-3 are vlan 5
    eth4-6 are vlan 10
    eth7-8 are vlan 30

    I wanted to setup a lagg for each vlan, but since the xg7100 marvell switch does not support lacp, I had to throw everything to basic load balancing. I have the cisco ports set up within channel groups, and the pfsense ports I had to manually add them to their own lagg groups through ssh using etherswitchcfg.

    As far as I can tell, it works properly, on the switch it shows the proper ports per vlan have bundled.

    The 3560 will have the flapping error if I leave each vlan mac addr the same (as shown in ifconfig when run through ssh on the xg7100).

    I can draw something up later today to better illustrate.


  • LAYER 8 Netgate

    You should be setting up the load balance laggs using the web gui and changing the lagg numbers by clicking on the lagg number in Interfaces > Switches, Ports. Just use a unique number for each group.



  • I should be, but when I go to add a LAGG in the gui, the ports on the marvell switch do not show up. So I cannot make a LAGG with the ports I want to use.


  • LAYER 8 Netgate

    Screenshot please. That should not be the case. Unless maybe something you did under the hood is preventing that from working any longer somehow.



  • laggs.PNG

    ix1 is only the 10gig sfp+.
    Eth1-8 are not shown.


  • LAYER 8 Netgate

    No.

    Interfaces > Switches, Ports



  • If you are referring to inputting the lagg id manually there... that instantly destroys the web gui and requires the firewall to restart.


  • LAYER 8 Netgate

    Don't do it on a port you are trying to manage the firewall from. Same way you would not make a change to a switch port you are managing the switch from.



  • Sure, I understand that.

    Referencing back to the original question, the xg7100 has the same MAC addr assigned to eth1-8 on the marvell switch, right?

    So wouldnt creating separate LAGGs across that marvell switch cause a downstream switch to see the same mac for multiple channel groups, then cause the flapping error?


  • LAYER 8 Netgate

    No. MAC Address tables should be separated in the switch by VLAN/broadcast domain.

    The MAC address being seen by the switch is actually the MAC address of the lagg0 created by ix2 and ix3 on the pfSense software on the uplink.

    Layer 2 switch ports do not have individual MAC addresses. The have MAC address tables and forward traffic between different MAC addresses in the same broadcast domain.



  • Okay, but the downstream switch has its own mac address table and when the vlans all show up as the same mac, I'm pretty sure that causes the flapping error.

    I'm not 100% sure what the flapping error exactly is, but I don't really see what else could cause the error since it stops when I change the macs for the vlans.


  • LAYER 8 Netgate

    Perhaps the load balance lagg/port channel in that switch is incompatible with the load-balance lagg in the Marvell chipset. Really hard to tell from here.

    The port channel should allow input from any MAC on any of the ports at any time.

    Are Po1 and Po2 in your OP members of the same port channel? Is the port channel properly-configured?



  • @erasedhammer

    Your original post showed the MAC as switching between interfaces. That wouldn't normally happen with VLANs.



  • That's what Im trying to explain, that the switch thinks its switching because the vlans all show up as the same mac.

    My original post showed the switch saying it was switching, but if all the interfaces have the same mac, how would it know it was switching?


  • LAYER 8 Netgate

    Because the same MAC is showing up on the same VLAN on two different ports (or port channels in this case.) It is not just the same MAC address, it is the same MAC address on the same VLAN. You should get the same log moving any host from one port to another, though it might be suppressed because of the link down clearing the CAM table when you move it.

    Both of those in your OP are VLAN 1, not two different VLANs.



  • Im not sure the vlan1 makes a difference. Vlan1 is not in use on my switch, it is simply the default vlan and since the trunk ports on the switch cant have a vlan assigned directly, they only can have allowed vlans assigned, it shows up in the logging as vlan1.



  • @erasedhammer said in VLANs having same mac address causing flapping error on cisco switch:

    My original post showed the switch saying it was switching, but if all the interfaces have the same mac, how would it know it was switching?

    I think you may be getting things confused. The MAC address refers to the original interface. I assume that's the pfSense box. Switches do not change the MAC address of frames passing through them. So, no matter where that frame goes, the MAC address will remain the same. Switches then learn which interface is closest to that interface, though spanning tree can change that, when the configuration changes.



  • Okay, so then my question is why doesn't the xg7100 have the ability to assign macs to the switch ports on the marvell?
    Is that a software or hardware limitation?


  • LAYER 8 Netgate

    Because Layer 2 switch ports don't have MAC addresses.

    Please post your current etherswitchcfg output.

    And the port channel configurations on the switch and what ports are connected to what.

    Thanks.



  • Ah! I see what youre saying now.

    So what solutions are there to ensure separate mac addresses for the separate vlan ? I am referring to potentially assigning the vlan on pfsense with a mac addr, although I know this is not possible from what I've read.
    The reason I ask is because ifconfig lladdr does not survive a reboot.

    pfsense etherswitchcfg:

    etherswitch0: VLAN mode: DOT1Q
    port1:
    pvid: 5
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port2:
    pvid: 5
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port3:
    pvid: 5
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port4:
    pvid: 10
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port5:
    pvid: 10
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port6:
    pvid: 10
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port7:
    pvid: 30
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    port8:
    pvid: 30
    state=8<FORWARDING>
    flags=0<>
    media: Ethernet autoselect (none)
    status: no carrier
    port9:
    pvid: 1
    state=8<FORWARDING>
    flags=1<CPUPORT>
    media: Ethernet 2500Base-KX <full-duplex>
    status: active
    port10:
    pvid: 1
    state=8<FORWARDING>
    flags=1<CPUPORT>
    media: Ethernet 2500Base-KX <full-duplex>
    status: active
    laggroup0:
    members 9,10
    laggroup1:
    members 2,3
    laggroup2:
    members 4,5,6
    laggroup3:
    members 7,8
    vlangroup0:
    vlan: 1
    members none
    vlangroup1:
    vlan: 5
    members 1,2,3,9t,10t
    vlangroup3:
    vlan: 30
    members 7,8,9t,10t
    vlangroup4:
    vlan: 10
    members 4,5,6,9t,10t

    cisco:
    !
    interface Port-channel1
    switchport trunk encapsulation dot1q
    !
    interface Port-channel2
    switchport trunk encapsulation dot1q
    !
    interface Port-channel3
    switchport trunk encapsulation dot1q
    !
    !
    interface GigabitEthernet0/2
    description RTR-UPLINK-MGNT
    switchport trunk encapsulation dot1q
    channel-group 1 mode on
    !
    interface GigabitEthernet0/3
    description RTR-UPLINK-MGNT
    switchport trunk encapsulation dot1q
    channel-group 1 mode on
    !
    interface GigabitEthernet0/4
    description RTR-UPLINK-USERS
    switchport trunk encapsulation dot1q
    channel-group 2 mode on
    !
    interface GigabitEthernet0/5
    description RTR-UPLINK-USERS
    switchport trunk encapsulation dot1q
    channel-group 2 mode on
    !
    interface GigabitEthernet0/6
    description RTR-UPLINK-USERS
    switchport trunk encapsulation dot1q
    channel-group 2 mode on
    !
    interface GigabitEthernet0/7
    description RTR-UPLINK-LAB
    switchport trunk encapsulation dot1q
    channel-group 3 mode on
    !
    interface GigabitEthernet0/8
    description RTR-UPLINK-LAB
    switchport trunk encapsulation dot1q
    channel-group 3 mode on


  • LAYER 8 Netgate

    You should probably set the PVID of channel-group 1 to 5, the PVID on channel-group 2 to 10 and the PVID of channel-group 3 to 30.

    At least I think that's what I see there.

    You are sending untagged traffic for three different VLANs into three different switch ports (port-channels, actually) you need to separate those VLANs on the cisco switch side too. The switch has no way to tell the traffic for one VLAN from another because it has no VLAN tags to work with.



  • @erasedhammer said in VLANs having same mac address causing flapping error on cisco switch:

    So what solutions are there to ensure separate mac addresses for the separate vlan ? I am referring to potentially assigning the vlan on pfsense with a mac addr,

    Once again, not possible. You're asking software to change hardware. While you can change the MAC, it will change for every frame transmitted by that NIC. The difference between different VLANs is the content of the VLAN tag. That's it. So, when a frame goes out on a VLAN, the contents of the tag are set for that VLAN.

    One thing to bear in mind is that all communication on the LAN is via MAC address. If you were to somehow change it when transmitting a frame, you might have the wrong MAC for a frame sent to another VLAN and that frame will be lost.



  • @Derelict
    I would assume setting native vlan (5, 10, 30) on those would be sufficient? Little divergent in the topic, but your help is much appreciated.



  • @erasedhammer

    Native means no VLAN tag. So, you could configure a managed switch to bring out VLAN 5, for example, to a port that carries only VLAN 5 traffic. That is called an access port and will pass the frames without VLAN tag. A trunk port passes multiple VLANs and the frames have VLAN tags.



  • @JKnott
    So it should be best to set all ports within the same vlan to switch access vlan X, including the ports to lead to other switches?



  • @erasedhammer

    There are 2 ways to configure a port. First, to pass only a single VLAN. It will then strip the tag from packets leaving through that port or add it to one entering it. The other way is to configure a trunk port to pass multiple VLANs. Depending on the switch, it may be possible to pass only specific VLANs and not others.

    So, to answer your question, you have to determine what you want each port to do and configure accordingly. So, if you have some IoT devices, you'd configure an access port for that VLAN. If you have a phone and computer sharing a port, then you'd have to configure to pass the tagged VLAN frames, as well an untagged frame. Same with an access point with multiple SSIDs. As always, determine your requirements and go from there.


  • LAYER 8 Netgate

    I would assume setting native vlan (5, 10, 30) on those would be sufficient? Little divergent in the topic, but your help is much appreciated.

    @erasedhammer Whatever your switch calls it, yes.


Log in to reply