Bridge + trunking in/out + mac learning problem

formsim

After countless hours trying to confiure the bridging side of pfsense to work with trunking on both Wan/Opt (Spanning tree on bridge and both switches can be a issue when troubleshooting ;) ) i am now successfully passing multible vlans accross from one trunk to another with filtering bridge engaged and working. Now the problem i am having is the mac address's from vlan A are crossing over into Vlan b's address table. The result is the following error on the switch facing my wan port.

2006 Apr 04 21:00:38 %MLS-4-RESUMESC:Resume MLS after detecting too many moves
2006 Apr 04 21:00:38 %MLS-4-MOVEOVERFLOW:Too many moves, stop MLS for 5 sec(40000000)

This normaly occurs when i either create a loop on the network or bridge Vlan A directly with Vlan B (causing mac address's to be read on both vlans).

I have looked at both vlans to verify there is no traffic going accrosss Vlan A that should be going accross Vlan B and all appears fine (Minus the mac entries).

The above error has not hindered the functionality of the firewall and is currently working the way it was planned but before more vlans can be added i need make sure the machines on Vlan A will not be read on every vlan i create causing some serious log problems and un needed cpu load. Here is my layout

Example output of switch A's "Sh cam dynamic port" command, notice 1 mac being on 2 vlans

VLAN Dest MAC/Route Des [CoS] Destination Ports or VCs / [Protocol Type]
–-- ------------------ ----- -------------------------------------------
A 00-05-5d-33-8e-71 PfSense
B 00-02-b3-3d-7d-71 PfSense
A 00-30-48-53-db-cb PfSense
B 00-30-48-53-db-cb PfSense
A 00-50-da-ba-83-80 PfSense
B 00-50-da-ba-83-80 PfSense
A 00-30-48-55-f2-94 PfSense
B 00-30-48-55-f2-94 PfSense

I am using Beta 2 and i installed the arpwatch package but the service does not start and the arp option for same physical interface also does not resolve this issue.

Any help would be greatly apreciated. i assume if it is something misconfiugred in either the switch or firewall the traffic would either not be passing or would be creating a loop of some type.

formsim

Any suggestions, this is killing me….

Is it possible this is a ethernet bridge problem and not a firewall problem? A problem with the briding module itself?

I'm googlin like crazy but not finding much about briding vlan to vlans on the net.

formsim

Fyi,

i've gone ahead and added 10 more vlans of which i made 20 more optional interfaces and bridged outside vlan with inside.
Traffic for all vlans are crossing the bridge without a problem and i performed a packet dump on one of the vlans to see if i'm getting any packets from other vlans jumping across.

I am not seeing anything unusual. Back to the switch facing the firewall my mac count doubled for every vlan added. Now with 40 machines across 10 vlans i'm getting 403 mac address's in the mac table of the switch and more constant errors like stated before.

Is this possibly a if_bri problem or pfsesnse. I'm not even sure where to start investigating next.

http://lists.freebsd.org/pipermail/freebsd-bugs/2005-September/014766.html

shows a bug for freebsd that is almost identical to the problem i'm having but this was dated back in september and shows fixed as of freebsd 6's release.

sullrich

Thanks, I have brought this to the attention of the FreeBSD bridge maintainer. We'll see what he says.

formsim

Thankx, i'm prety sure its not a pfsense problem but don't know enough about the packaging pfsense uses to start posting bugs on bridge boards.

this is a show mac table entry from SWITCH b

Edge3.2924#sh mac-address-table address 0030.4856.8b94
Non-static Address Table:
Destination Address Address Type VLAN Destination Port
–----------------- ------------ ---- --------------------
0030.4856.8b94 Dynamic 10 FastEthernet0/4
0030.4856.8b94 Dynamic 11 FastEthernet0/4
0030.4856.8b94 Dynamic 102 FastEthernet0/4
0030.4856.8b94 Dynamic 108 FastEthernet0/4
0030.4856.8b94 Dynamic 110 FastEthernet0/4
0030.4856.8b94 Dynamic 111 FastEthernet0/4
0030.4856.8b94 Dynamic 225 FastEthernet0/3 <--- originating port / vlan

and the the output of my cam table on SWITCH A of the same mac

Core1.5505> (enable) sh ca 00-30-48-56-8b-94

VLAN Dest MAC/Route Des [CoS] Destination Ports or VCs / [Protocol Type]
–-- ------------------ ----- -------------------------------------------
111 00-30-48-56-8b-94 4/16 [ALL]
11 00-30-48-56-8b-94 4/16 [ALL]
110 00-30-48-56-8b-94 4/16 [ALL]
10 00-30-48-56-8b-94 4/16 [ALL]
225 00-30-48-56-8b-94 4/16 [ALL] <–--- originating vlan
108 00-30-48-56-8b-94 4/16 [ALL]
102 00-30-48-56-8b-94 4/16 [ALL]
Total Matching CAM Entries Displayed =7

i'm going to see if i can some packet dumps and look at it a tad closer.

p.s. Great firewall , if i can get this problem resolved it will be placed inline with my core network and setup for all my clients asap.

sullrich

Its a known limitation, unfortunately.

http://www.freebsd.org/cgi/query-pr.cgi?pr=86361

formsim

Update for those that might be trying to impliment the same thing i am doing.

Essentialy i am trying to put 2 pfsesne boxes between my core routers and my edge switches with backup links going around the firewal for redundancy.
I've upgraded my config from beta 2 ta 3.

Thank god ftp is fixed that was a huge issue thankx guys.

I've upgraded my in and out nics from 100mb to fiber 1gig cards, and changed the config accordingly.

After a few hours of troubleshooting loop issues and up down interfaces i finnaly was able to get 4 vlans trunked together across my fiber link passing 4-5 megs with about 15-20 physical machines.

Of course like sullrich said, the mac entries are still being broadcasted across all vlans with no fix in sight (might actually look at freebsd code to see if i can make a quick fix) but now i also have the spanning tree packets beign blocked as well.

If i shut filtered bridge off all 4 vlans go into stp disabled status for recieving ther other 4 vlan's stp packets.if i enable the filtered bridge all works but not stp packets pass at all.Apparently the bridge does not know after recieving these packets where to send them on the other side.

This wouldn't be that much of a problem being the bridge on the firewall itself has stp enabled, but being i want to place this inline with my core and edge switches i can not run a secondary (non firewalled) connection to my switches without creating 2 links on the same vlan in stp forward status (IE loop). Its either one link or the other.

So it boils down to this.

You can have 2 pfsense boxes on one switch, you can even have them on multible switches with the redundancy u want. But if you run a trunk of more than 2 vlans (1 natvie and 1 non native) across the firewall stp and mac entries will be screwed to high heaven causing extensive cpu load from learning mac entries over and over again(depending on how many machines u have, i got a few hundred) and possible stp loops even when enabled.

From what i am seeing on google and freebsd boards, this is a limitation of the kernel and not pfsense so no complaints here, but thought i'd give you guys a heads up.

Any suggestions would be apreciated. Anyone curious and want more details about my setup feel free to ask.