VLANs having same mac address causing flapping error on cisco switch

Derelict · Mar 27, 2020, 7:52 PM

Don't do it on a port you are trying to manage the firewall from. Same way you would not make a change to a switch port you are managing the switch from.

erasedhammer · Mar 27, 2020, 7:59 PM

Sure, I understand that.

Referencing back to the original question, the xg7100 has the same MAC addr assigned to eth1-8 on the marvell switch, right?

So wouldnt creating separate LAGGs across that marvell switch cause a downstream switch to see the same mac for multiple channel groups, then cause the flapping error?

Derelict · Mar 27, 2020, 8:03 PM

No. MAC Address tables should be separated in the switch by VLAN/broadcast domain.

The MAC address being seen by the switch is actually the MAC address of the lagg0 created by ix2 and ix3 on the pfSense software on the uplink.

Layer 2 switch ports do not have individual MAC addresses. The have MAC address tables and forward traffic between different MAC addresses in the same broadcast domain.

erasedhammer · Mar 27, 2020, 8:10 PM

Okay, but the downstream switch has its own mac address table and when the vlans all show up as the same mac, I'm pretty sure that causes the flapping error.

I'm not 100% sure what the flapping error exactly is, but I don't really see what else could cause the error since it stops when I change the macs for the vlans.

Derelict · Mar 27, 2020, 8:24 PM

Perhaps the load balance lagg/port channel in that switch is incompatible with the load-balance lagg in the Marvell chipset. Really hard to tell from here.

The port channel should allow input from any MAC on any of the ports at any time.

Are Po1 and Po2 in your OP members of the same port channel? Is the port channel properly-configured?

JKnott · Mar 27, 2020, 8:24 PM

@erasedhammer

Your original post showed the MAC as switching between interfaces. That wouldn't normally happen with VLANs.

erasedhammer · Mar 27, 2020, 8:32 PM

That's what Im trying to explain, that the switch thinks its switching because the vlans all show up as the same mac.

My original post showed the switch saying it was switching, but if all the interfaces have the same mac, how would it know it was switching?

Derelict · Mar 27, 2020, 8:39 PM

Because the same MAC is showing up on the same VLAN on two different ports (or port channels in this case.) It is not just the same MAC address, it is the same MAC address on the same VLAN. You should get the same log moving any host from one port to another, though it might be suppressed because of the link down clearing the CAM table when you move it.

Both of those in your OP are VLAN 1, not two different VLANs.

erasedhammer · Mar 27, 2020, 9:26 PM

Im not sure the vlan1 makes a difference. Vlan1 is not in use on my switch, it is simply the default vlan and since the trunk ports on the switch cant have a vlan assigned directly, they only can have allowed vlans assigned, it shows up in the logging as vlan1.

JKnott · Mar 27, 2020, 9:29 PM

@erasedhammer said in VLANs having same mac address causing flapping error on cisco switch:

My original post showed the switch saying it was switching, but if all the interfaces have the same mac, how would it know it was switching?

I think you may be getting things confused. The MAC address refers to the original interface. I assume that's the pfSense box. Switches do not change the MAC address of frames passing through them. So, no matter where that frame goes, the MAC address will remain the same. Switches then learn which interface is closest to that interface, though spanning tree can change that, when the configuration changes.

erasedhammer · Mar 27, 2020, 11:00 PM

Okay, so then my question is why doesn't the xg7100 have the ability to assign macs to the switch ports on the marvell?
Is that a software or hardware limitation?

Derelict · Mar 28, 2020, 12:51 AM

Because Layer 2 switch ports don't have MAC addresses.

Please post your current etherswitchcfg output.

And the port channel configurations on the switch and what ports are connected to what.

Thanks.

erasedhammer · Mar 28, 2020, 1:15 AM

Ah! I see what youre saying now.

So what solutions are there to ensure separate mac addresses for the separate vlan ? I am referring to potentially assigning the vlan on pfsense with a mac addr, although I know this is not possible from what I've read.
The reason I ask is because ifconfig lladdr does not survive a reboot.

pfsense etherswitchcfg:

etherswitch0: VLAN mode: DOT1Q
port1:
pvid: 5
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port2:
pvid: 5
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port3:
pvid: 5
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port4:
pvid: 10
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port5:
pvid: 10
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port6:
pvid: 10
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port7:
pvid: 30
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
port8:
pvid: 30
state=8<FORWARDING>
flags=0<>
media: Ethernet autoselect (none)
status: no carrier
port9:
pvid: 1
state=8<FORWARDING>
flags=1<CPUPORT>
media: Ethernet 2500Base-KX <full-duplex>
status: active
port10:
pvid: 1
state=8<FORWARDING>
flags=1<CPUPORT>
media: Ethernet 2500Base-KX <full-duplex>
status: active
laggroup0:
members 9,10
laggroup1:
members 2,3
laggroup2:
members 4,5,6
laggroup3:
members 7,8
vlangroup0:
vlan: 1
members none
vlangroup1:
vlan: 5
members 1,2,3,9t,10t
vlangroup3:
vlan: 30
members 7,8,9t,10t
vlangroup4:
vlan: 10
members 4,5,6,9t,10t

cisco:
!
interface Port-channel1
switchport trunk encapsulation dot1q
!
interface Port-channel2
switchport trunk encapsulation dot1q
!
interface Port-channel3
switchport trunk encapsulation dot1q
!
!
interface GigabitEthernet0/2
description RTR-UPLINK-MGNT
switchport trunk encapsulation dot1q
channel-group 1 mode on
!
interface GigabitEthernet0/3
description RTR-UPLINK-MGNT
switchport trunk encapsulation dot1q
channel-group 1 mode on
!
interface GigabitEthernet0/4
description RTR-UPLINK-USERS
switchport trunk encapsulation dot1q
channel-group 2 mode on
!
interface GigabitEthernet0/5
description RTR-UPLINK-USERS
switchport trunk encapsulation dot1q
channel-group 2 mode on
!
interface GigabitEthernet0/6
description RTR-UPLINK-USERS
switchport trunk encapsulation dot1q
channel-group 2 mode on
!
interface GigabitEthernet0/7
description RTR-UPLINK-LAB
switchport trunk encapsulation dot1q
channel-group 3 mode on
!
interface GigabitEthernet0/8
description RTR-UPLINK-LAB
switchport trunk encapsulation dot1q
channel-group 3 mode on

Derelict · Mar 28, 2020, 1:32 AM

You should probably set the PVID of channel-group 1 to 5, the PVID on channel-group 2 to 10 and the PVID of channel-group 3 to 30.

At least I think that's what I see there.

You are sending untagged traffic for three different VLANs into three different switch ports (port-channels, actually) you need to separate those VLANs on the cisco switch side too. The switch has no way to tell the traffic for one VLAN from another because it has no VLAN tags to work with.

JKnott · Mar 28, 2020, 1:36 AM

@erasedhammer said in VLANs having same mac address causing flapping error on cisco switch:

So what solutions are there to ensure separate mac addresses for the separate vlan ? I am referring to potentially assigning the vlan on pfsense with a mac addr,

Once again, not possible. You're asking software to change hardware. While you can change the MAC, it will change for every frame transmitted by that NIC. The difference between different VLANs is the content of the VLAN tag. That's it. So, when a frame goes out on a VLAN, the contents of the tag are set for that VLAN.

One thing to bear in mind is that all communication on the LAN is via MAC address. If you were to somehow change it when transmitting a frame, you might have the wrong MAC for a frame sent to another VLAN and that frame will be lost.

erasedhammer · Mar 28, 2020, 2:22 AM

@Derelict
I would assume setting native vlan (5, 10, 30) on those would be sufficient? Little divergent in the topic, but your help is much appreciated.

JKnott · Mar 28, 2020, 3:22 AM

@erasedhammer

Native means no VLAN tag. So, you could configure a managed switch to bring out VLAN 5, for example, to a port that carries only VLAN 5 traffic. That is called an access port and will pass the frames without VLAN tag. A trunk port passes multiple VLANs and the frames have VLAN tags.

erasedhammer · Mar 28, 2020, 1:33 PM

@JKnott
So it should be best to set all ports within the same vlan to switch access vlan X, including the ports to lead to other switches?

JKnott · Mar 28, 2020, 1:40 PM

@erasedhammer

There are 2 ways to configure a port. First, to pass only a single VLAN. It will then strip the tag from packets leaving through that port or add it to one entering it. The other way is to configure a trunk port to pass multiple VLANs. Depending on the switch, it may be possible to pass only specific VLANs and not others.

So, to answer your question, you have to determine what you want each port to do and configure accordingly. So, if you have some IoT devices, you'd configure an access port for that VLAN. If you have a phone and computer sharing a port, then you'd have to configure to pass the tagged VLAN frames, as well an untagged frame. Same with an access point with multiple SSIDs. As always, determine your requirements and go from there.

Derelict · Mar 28, 2020, 3:32 PM

I would assume setting native vlan (5, 10, 30) on those would be sufficient? Little divergent in the topic, but your help is much appreciated.

@erasedhammer Whatever your switch calls it, yes.