Bridge + trunking in/out + mac learning problem

  • After countless hours trying to confiure the bridging side of pfsense to work with trunking on both Wan/Opt (Spanning tree on bridge and both switches can be a issue when troubleshooting ;) ) i am now successfully passing multible vlans accross  from one trunk to another with filtering bridge engaged and working. Now the problem i am having is the mac address's from vlan A are crossing over into Vlan b's address table. The result is the following error on the switch facing my wan port.

    2006 Apr 04 21:00:38 %MLS-4-RESUMESC:Resume MLS after detecting too many moves
    2006 Apr 04 21:00:38 %MLS-4-MOVEOVERFLOW:Too many moves, stop MLS for 5 sec(40000000)

    This normaly occurs when i either create a loop on the network or bridge Vlan A directly with Vlan B (causing mac address's to be read on both vlans).

    I have looked at both vlans to verify there is no traffic going accrosss Vlan A that should be going accross Vlan B and all appears fine (Minus the mac entries).

    The above error has not hindered the functionality of the firewall and is currently working the way it was planned but before more vlans can be added i need make sure the machines on Vlan A will not be read on every vlan i create causing some serious  log problems and un needed cpu load. Here is my layout

    Vlan A      Vlan B      Van Lan
      |              |                |
      |              |                |
      |_________|              |
                      |              |
                      |              |
                Switch A          |
                      |              |
                      |              |
                PfSense          |
                      |              |
                      |              |
                SwitchB            |
                      |              |
      __________|              |
      |                |              |
      |                |              |
    Vlan A      Vlan B      Van Lan

    Example output of switch A's "Sh cam dynamic port" command, notice 1 mac being on 2 vlans

    VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type]
    –--  ------------------    -----  -------------------------------------------
    A 00-05-5d-33-8e-71            PfSense
    B 00-02-b3-3d-7d-71            PfSense
    A 00-30-48-53-db-cb            PfSense
    B 00-30-48-53-db-cb            PfSense
    A 00-50-da-ba-83-80            PfSense
    B 00-50-da-ba-83-80            PfSense
    A 00-30-48-55-f2-94            PfSense
    B 00-30-48-55-f2-94            PfSense

    I am using Beta 2 and i installed the arpwatch package but the service does not start and the arp option for same physical interface also does not resolve this issue.

    Any help would be greatly apreciated. i assume if it is something misconfiugred in either the switch or firewall the traffic would either not be passing or would be creating a loop of some type.

  • Any suggestions, this is killing me….

    Is it possible this is a ethernet bridge problem and not a firewall problem? A problem with the briding module itself?

    I'm googlin like crazy but not finding much about briding vlan to vlans on the net.

  • Fyi,

    i've gone ahead and added 10 more vlans of which i made 20 more optional interfaces and bridged outside vlan with inside.
    Traffic for all vlans are crossing the bridge without a problem and i performed a packet dump on one of the vlans to see if i'm getting any packets from other vlans jumping across.

    I am not seeing anything unusual. Back to the switch facing the firewall my mac count doubled for every vlan added. Now with 40 machines across 10 vlans i'm getting 403 mac address's in the mac table of the switch and more constant errors like stated before.

    Is this possibly a if_bri problem or pfsesnse. I'm not even sure where to start investigating next.

    shows a bug for freebsd that is almost identical to the problem i'm having but this was dated back in september and shows fixed as of freebsd 6's release.

  • Thanks, I have brought this to the attention of the FreeBSD bridge maintainer.  We'll see what he says.

  • Thankx, i'm prety sure its not a pfsense problem but don't know enough about the packaging pfsense uses to start posting bugs on bridge boards.

    this is a show mac table entry from SWITCH b

    Edge3.2924#sh mac-address-table address 0030.4856.8b94
    Non-static Address Table:
    Destination Address  Address Type  VLAN  Destination Port
    –-----------------  ------------  ----  --------------------
    0030.4856.8b94      Dynamic        10  FastEthernet0/4
    0030.4856.8b94      Dynamic        11  FastEthernet0/4
    0030.4856.8b94      Dynamic        102  FastEthernet0/4
    0030.4856.8b94      Dynamic        108  FastEthernet0/4
    0030.4856.8b94      Dynamic        110  FastEthernet0/4
    0030.4856.8b94      Dynamic        111  FastEthernet0/4
    0030.4856.8b94      Dynamic        225  FastEthernet0/3 <--- originating port / vlan

    and the the output of my cam table on SWITCH A of the same mac

    Core1.5505> (enable) sh ca 00-30-48-56-8b-94

    VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type]
    –--  ------------------    -----  -------------------------------------------
    111  00-30-48-56-8b-94            4/16 [ALL]
    11    00-30-48-56-8b-94            4/16 [ALL]
    110  00-30-48-56-8b-94            4/16 [ALL]
    10    00-30-48-56-8b-94            4/16 [ALL]
    225  00-30-48-56-8b-94            4/16 [ALL] <–--- originating vlan
    108  00-30-48-56-8b-94            4/16 [ALL]
    102  00-30-48-56-8b-94            4/16 [ALL]
    Total Matching CAM Entries Displayed  =7

    i'm going to see if i can some packet dumps and look at it a tad closer.

    p.s. Great firewall , if i can get this problem resolved it will be placed inline with my core network and setup for all my clients asap.

  • Its a known limitation, unfortunately.

  • Update for those that might be trying to impliment the same thing i am doing.

    Essentialy i am trying to put 2 pfsesne boxes between my core routers and my edge switches with backup links going around the firewal for redundancy.
    I've upgraded my config from beta 2 ta 3.

    Thank god ftp is fixed that was a huge issue thankx guys.

    I've upgraded my in and out nics from 100mb to fiber 1gig cards, and changed the config accordingly.

    After a few hours of troubleshooting loop issues and up down interfaces i finnaly was able to get 4 vlans trunked together across my fiber link passing 4-5 megs with about 15-20 physical machines.

    Of course like sullrich said, the mac entries are still being broadcasted across all vlans with no fix in sight (might actually look at freebsd code to see if i can make a quick fix) but now i also have the spanning tree packets beign blocked as well.

    If i shut filtered bridge off all 4 vlans go into stp disabled status for recieving ther other 4 vlan's stp packets.if i enable the filtered bridge all works but not stp packets pass at all.Apparently the bridge does not know after recieving these packets where to send them on the other side.

    This wouldn't be that much of a problem being the bridge on the firewall itself has stp enabled, but being i want to place this inline with my core and edge switches i can not run a secondary (non firewalled) connection to my switches without creating 2 links on the same vlan in stp forward status (IE loop). Its either one link or the other.

    So it boils down to this.

    You can have 2 pfsense boxes on one switch, you can even have them on multible switches with the redundancy u want. But if you run a trunk of more than 2 vlans (1 natvie and 1 non native) across the firewall stp and mac entries will be screwed to high heaven causing extensive cpu load from learning mac entries over and over again(depending on how many machines u have, i got a few hundred) and possible stp loops even when enabled.

    From what i am seeing on google and freebsd boards, this is a limitation of the kernel and not pfsense so no complaints here, but thought i'd give you guys a heads up.

    Any suggestions would be apreciated. Anyone curious and want more details about my setup feel free to ask.

Log in to reply