Connecting VLANs via Catalyst 3560


  • Hello!

    I have pfSense 1.2.3 with internal network built with VLANs:
    vlan0 (VLAN ID 1)
    vlan1 (VLAN ID 13)
    vlan2 (VLAN ID 16)

    The internal NIC (em0 based) is connected to a port on 3com Superstack 3300, this port is added to VLANs 1,13 and 16 with 802.1Q tagging. Everything is working beautifully.

    Now I need to move the internal connection to Catalyst 3560G. I have the port configured like this:

    interface GigabitEthernet0/22
     switchport trunk encapsulation dot1q
     switchport trunk allowed vlan 1,13,16
     switchport mode trunk
     switchport nonegotiate
    
    

    However, when connecting pfSense's internal NIC to this port, I can't ping anything on the internal network from the pfSense, nor can I ping the pfSense box from any of the internal networks.
    There are other ports on the Catalyst belonging to VLANs 1,13 and 16, and they have no problem passing traffic. Maybe someone who has experience with pfSense+Catalyst can see if anything is wrong with the above port setup.


  • You may need to set the "native vlan" statement on this port:

    • config t
    • int gig 0/22
    • switchport native-vlan 13

    BTW - Cisco does not have VLAN0, the VLANs start at 1.  I would renumber your VLANs such as vlan1, vlan2, vlan3 just to keep things consistent.

    Also, what does "sh int gige0/22" give you?  What about "show int trunk"?


  • @vatson:

    
     switchport nonegotiate
    
    

    Curious, why do you have nonegotiate set on the link?

  • Rebel Alliance Developer Netgate

    @rkelleyrtp:

    BTW - Cisco does not have VLAN0, the VLANs start at 1.  I would renumber your VLANs such as vlan1, vlan2, vlan3 just to keep things consistent.

    I'd take that a step further and not use VLAN1 either. Start numbering at 10 or 100 if it's easier, but avoid VLAN1. Too much stuff defaults to going there, and it's typically things that shouldn't be mixed into a network exposed to end users.

    The OP may also want to try without the "allowed" line and see if it works.


  • Agree about vlan1.  Normally, network guys start at >1 because vlan1 is considered the Admin vlan.

    If it were me, I would start at 100 and work my way up from there.  In fact, I have this identical config on a pair of 3750G switches using this interface config:

    
    interface GigabitEthernet1/0/38
     description CAB-B pfSense1 INSIDE TRUNK
     switchport access vlan 300
     switchport trunk encapsulation dot1q
     switchport trunk allowed vlan 300,2000-2999
     switchport mode trunk
    
    

    Here, I am trunking VLANs 300 and 2000-2999.  The "switchport access vlan 300" statement seems to be necessary on pfSense because that is the primary (native) vlan on that interface.

    Hope this helps…


  • Thanks a lot for all the comments, and sorry for not replying sooner. I've been overwhelmed with other, non-pfSense-related issues. I really appreciate all your advice, but unfortunately I can't try it out until maybe Sunday. I'll certainly let you know how it goes.

    The most significant thing that I seem to be missing is the native-VLAN setting.

    I have nonegotiate set on all the trunk ports. My thinking with this was that I really don't want the switch to "negotiate" and start carryng some VLANs across the trunks that I didn't intend. Maybe I'm missing the point here.

    BTW, I only have this one Catalyst in my network, all the other equipment connecting to it is non-Cisco.

    I totally agree with you regarding not using VLAN 1, and if I'd start creating my network today I would avoid it. Unfortunately I wasn't aware of this practice many years ago when I first implemented VLANs, and now migrating away from VLAN 1 would mean reconfiguring dozens of switches…


  • No problem.  Let us know how it goes.

    BTW - I would make sure the VLAN IDs match between your pfSense box and your Cisco switch.  In your original email, you mentioned:

    pfSense      CISCO
    –-------    --------
    vlan0    --> VLAN1
    vlan1    --> VLAN13
    vlan2    --> VLAN16

    I would make sure the VLAN IDs match (like this:)

    pfSense        CISCO
    –-------    --------
    vlan1      --> VLAN1
    vlan13    --> VLAN13
    vlan16    --> VLAN16

    This way, you probably won't run into the native vlan problem…

  • Rebel Alliance Developer Netgate

    @rkelleyrtp:

    No problem.  Let us know how it goes.

    BTW - I would make sure the VLAN IDs match between your pfSense box and your Cisco switch.  In your original email, you mentioned:

    pfSense      CISCO
    –-------    --------
    vlan0    --> VLAN1
    vlan1    --> VLAN13
    vlan2    --> VLAN16

    That is just how pfSense numbers them in 1.2.x. The first one is 0, next is 1, etc, etc. In pfSense 2 they are tagged with the vlan id, but not in 1.2.x. That's fine, as long as their VLAN IDs are set right.


  • Yes, in my understanding also the numbers behind interface name are not the same thing as VLAN tags. VLAN tags appear separately in ifconfig, as seen below:

    vlan0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:0a:e4:01:38:f0
            inet6 fe80::20a:e4ff:fe01:38f0%vlan0 prefixlen 64 scopeid 0x7
            inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
            media: Ethernet autoselect (100baseTX <half-duplex>)
            status: active
            vlan: 1 parent interface: em0
    vlan1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:0a:e4:01:38:f0
            inet6 fe80::20a:e4ff:fe01:38f0%vlan1 prefixlen 64 scopeid 0x8
            inet 192.168.14.1 netmask 0xffffff00 broadcast 192.168.14.255
            media: Ethernet autoselect (100baseTX <half-duplex>)
            status: active
            vlan: 13 parent interface: em0
    vlan2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 00:0a:e4:01:38:f0
            inet6 fe80::20a:e4ff:fe01:38f0%vlan2 prefixlen 64 scopeid 0x9
            inet 192.168.6.1 netmask 0xffffff00 broadcast 192.168.6.255
            media: Ethernet autoselect (100baseTX <half-duplex>)
            status: active
            vlan: 16 parent interface: em0</half-duplex></rxcsum,txcsum></up,broadcast,running,simplex,multicast></half-duplex></rxcsum,txcsum></up,broadcast,running,simplex,multicast></half-duplex></rxcsum,txcsum></up,broadcast,running,simplex,multicast>

  • Rebel Alliance Developer Netgate

    Yeah those look fine.


  • I did some tests, but without success…

    First I tried this, on top of my existing setup:

    config t
    int gi0/22
    switchport access vlan 1
    
    

    This seemed to have no noticeable effect on the config. Also, when connecting pfSense to the port, things still didn't work

    'switchport native-vlan' doesn't seem to be an accepted command.

    Then I reduced the port config to just:

    switchport mode trunk
    switchport trunk encapsulation dot1q
    

    Still no joy…

    Then I tried this

    switchport mode access
    switchport access vlan 1
    switchport mode trunk
    swithcport trunk encapsulation dot1q
    switchport trunk allowed vlan 1,3,16
    
    

    No change…

    I also tried fixing the speed to 100FD, which also didn't help, so I changed back to auto.

    This is the output of "sh int gi0/22" when pfSense is connected:

    GigabitEthernet0/22 is up, line protocol is up (connected)
      Hardware is Gigabit Ethernet, address is 0025.8306.4096 (bia 0025.8306.4096)
      MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
         reliability 255/255, txload 1/255, rxload 1/255
      Encapsulation ARPA, loopback not set
      Keepalive set (10 sec)
      Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
      input flow-control is off, output flow-control is unsupported
      ARP type: ARPA, ARP Timeout 04:00:00
      Last input never, output 00:00:00, output hang never
      Last clearing of "show interface" counters never
      Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
      Queueing strategy: fifo
      Output queue: 0/40 (size/max)
      5 minute input rate 326000 bits/sec, 44 packets/sec
      5 minute output rate 37000 bits/sec, 47 packets/sec
         41587 packets input, 46170281 bytes, 0 no buffer
         Received 24 broadcasts (0 multicasts)
         37 runts, 0 giants, 0 throttles
         98 input errors, 61 CRC, 0 frame, 0 overrun, 0 ignored
         0 watchdog, 8 multicast, 0 pause input
         0 input packets with dribble condition detected
         62462 packets output, 8175017 bytes, 0 underruns
         0 output errors, 0 collisions, 1 interface resets
         0 babbles, 0 late collision, 0 deferred
         0 lost carrier, 0 no carrier, 0 PAUSE output
         0 output buffer failures, 0 output buffers swapped out
    

    And the output of "sh int trunk" (I removed other ports for brevity):

    Port        Mode         Encapsulation  Status        Native vlan
    Gi0/22      on           802.1q         trunking      1
    
    Port        Vlans allowed on trunk
    Gi0/22      1,13,16
    
    Port        Vlans allowed and active in management domain
    Gi0/22      1,13,16
    
    Port        Vlans in spanning tree forwarding state and not pruned
    Gi0/22      1,13,16
    

    I'm starting to suspect the issue may not be in VLAN tagging but something else entirely… But what might it be?


  • I was mistaken before about the native vlan command.  Use this:  "switchport trunk native vlan <xxx>".  But, since you are trunking VLAN 1, that should be the native vlan.

    Can you give me the output of "sh run | int gig 0/22" ?</xxx>


  • Current configuration : 137 bytes
    !
    interface GigabitEthernet0/22
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 1,13,16
    switchport mode trunk
    end


  • What kind of NIC are you using in your pfSense box?  I have seen other with NICs that have hardware VLAN support causing similar issues.  Disabling the hardware VLAN support fixed the problem.

    Aside from that, I would do the following:

    • Disable VLANs all together (conf t ; no switchport trunk ; no switchport trunk allowed vlan 1,13,16; no switchport trunk encapsulation dot1q ; no switchport trunk)
    • Enable the port for static VLAN of 13 (conf t ; switchport host ; switchport access vlan 13)
    • Ping something in the 13 subnet (assuming you gave pfSense "em0" and ip address in vlan 13)

    If this works, the pfSense box has good connectivity to your switch and network.  Next:
    –--------------------------------------------------------------------------------------------------------------

    • Enable VLANs 13 and 16 (conf t ; no switchport host ; no switchport access vlan 13 ; switchport trunk encapsulation dot1q ; switchport trunk; switchport trunk allowed vlan 13,16)
    • Set pfSense with the corresponding VLAN  13 and 16 IP addresses
    • Ping something in both VLANs

    Let me know what you find.

    BTW - I specifically left out VLAN1 in the second test case.


  • Well, I finally got it working.

    What I needed to do was set the native VLAN on the trunk port to something other than one of the VLANs which I actually want to connect to pfSense. So I set

    conf t
    int gi0/22
    switchport trunk native vlan 2
    

    and everything is working now. VLAN 2 is not actually used anymore on our network, so I'll probably keep it as a "ghost native vlan" for the pfSense uplink port.

    For future reference, this is my final working port config, for linking VLANs 1, 13 and 16 to pfSense:

    interface GigabitEthernet0/22
     switchport trunk encapsulation dot1q
     switchport trunk native vlan 2
     switchport trunk allowed vlan 1,13,16
     switchport mode trunk
    end
    

  • @vatson:

    VLAN 2 is not actually used anymore on our network, so I'll probably keep it as a "ghost native vlan" for the pfSense uplink port.

    Interesting.  I do that for all my Cisco switches as a matter of best practice, but I've never seen it required.