VLAN fail on a SG-4860, what am I missing?



  • I've been fighting with pfSense on my SG-4860-1U for several days now trying & failing to get VLANs working. No matter what I do I'm not able to get my pfSense box to even reply to any pings.

    I have several Cisco & TPlink switches that are working properly over VLAN 10 (I assume other VLANs are working, but as my pfSense box will be the router I haven't been able to validate). I can ping everything that has an IP on VLAN 10 as expected, except the pfSense box. The TPlink switch & Cisco switches are communicating fine & I've tried the pfSense box connected to appropriately VLAN tagged ports on both with the same results.

    I've followed the guides at http://mcisageek.net/?p=125 & https://www.netgate.com/docs/pfsense/book/vlan/pfsense-vlan-configuration.html with no luck. I've reset the config several times & tried setting up the LAN port (igb0)& OPT1 (igb2) as the one plugged into the trunked port to no avail with several different tagging & cable configurations.

    igb0    00:08:a2:0b:05:ec   (up) Intel(R) PRO/1000 Network Connection, Version
    igb1    00:08:a2:0b:05:ed   (up) Intel(R) PRO/1000 Network Connection, Version
    igb2    00:08:a2:0b:05:e8   (up) Intel(R) PRO/1000 Network Connection, Version
    igb3    00:08:a2:0b:05:e9 (down) Intel(R) PRO/1000 Network Connection, Version
    igb4    00:08:a2:0b:05:ea (down) Intel(R) PRO/1000 Network Connection, Version
    igb5    00:08:a2:0b:05:eb (down) Intel(R) PRO/1000 Network Connection, Version
    
    WAN (wan)       -> igb1       -> v4/DHCP4: 192.168.0.34/24
    LAN (lan)       -> igb0       -> v4: 192.168.172.1/24
    OPT1 (opt1)     -> igb2       ->
    OPT2 (opt2)     -> igb3       ->
    OPT3 (opt3)     -> igb4       ->
    OPT4 (opt4)     -> igb5       ->
    HOME (opt5)     -> igb2.10    -> v4: 10.10.10.12/24
    GUEST (opt6)    -> igb2.11    -> v4: 10.10.11.1/24
    OTHER (opt7)    -> igb2.12    -> v4: 10.10.12.1/24
    OTHER2 (opt8)   -> igb2.13    -> v4: 10.10.13.1/24
    LONDONWAN (opt9) -> igb2.254   -> v4: 10.254.0.6/24
    

    I have CDP & LLDP enabled on the pfSense box which is currently plugged into 2 different ports on the switch, 13 which isn't VLAN tagged & port 23 with is is tagged for all my VLANS. The access port 13 works fine & I can get to the pfSense box fine from a couple switches away. But I have yet to get a ping from whatever IP I have configured when connected to port 23 with all the VLANs tagged.

    rack-switch#sh cdp neighbors
    Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                      S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
                      D - Remote, C - CVTA, M - Two-port Mac Relay
    
    Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
    gibraltar          Gig 1/0/23        96               R H   FreeBSD   igb2
    gibraltar          Gig 1/0/13        96               R H   FreeBSD   igb0
    2960-s            Gig 1/0/24        133              S I   WS-C2960S Gig 1/0/46
    
    Total cdp entries displayed : 3
    
    interface GigabitEthernet1/0/13
     switchport access vlan 10
     switchport mode access
    
    interface GigabitEthernet1/0/23
     description Gibraltar
     switchport access vlan 10
     switchport trunk native vlan 10
     switchport trunk allowed vlan 3,4,10-13,254
     switchport mode trunk
    

    Can anybody tell me what I'm missing? I should easily be able to deal with the rest of the VLANS & routing stuff later on once I get once interface on the pfSense box working.


  • Global Moderator

    Can you attach screenshots for VLAN10 interface:

    • Status/Interfaces
    • Firewall/Rules

    Did you try to run " no switchport access vlan 10" on your Cisco switch?
    I think it has to be like this:
    interface GigabitEthernet1/0/23
    description Gibraltar
    switchport trunk native vlan 10
    switchport trunk allowed vlan 3,4,10-13,254
    switchport mode trunk



  • Somewhat sure that "switchport access vlan 10" doesn't do anything unless I set "switchport mode access", like on port 13. But tried it anyway as obviously I'm missing something. Still nothing even after a shut & no shut.

    interface GigabitEthernet1/0/23
     description Gibraltar
     switchport trunk native vlan 10
     switchport trunk allowed vlan 3,4,10-13,254
     switchport mode trunk
    

    I see inbound packets on the root interface, but none on the VLAN interface.

    OPT1 Interface (opt1, igb2)
    Status up
    MAC Address 00:08:a2:0b:05:e8
    IPv6 Link Local fe80::208:a2ff:fe0b:5e8%igb2
    MTU 1500
    Media 1000baseT <full-duplex>
    In/out packets 61730/5 (16.64 MiB/416 B)
    In/out packets (pass) 61730/5 (16.64 MiB/416 B)
    In/out packets (block) 9252/0 (2.23 MiB/0 B)
    In/out errors 0/0
    Collisions 0
    
    HOME Interface (opt5, igb2.10)
    Status up
    MAC Address 00:08:a2:0b:05:e8
    IPv4 Address 10.10.10.12
    Subnet mask IPv4 255.255.255.0
    IPv6 Link Local fe80::208:a2ff:fe0b:5e8%igb2.10
    MTU 1500
    Media 1000baseT <full-duplex>
    In/out packets 0/24377896 (0 B/4.66 GiB)
    In/out packets (pass) 0/24377896 (0 B/4.66 GiB)
    In/out packets (block) 0/0 (0 B/0 B)
    In/out errors 0/0
    Collisions 0
    
    GigabitEthernet1/0/23 is up, line protocol is up (connected)
      Hardware is Gigabit Ethernet, address is 843d.c6de.9397 (bia 843d.c6de.9397)
      Description: Gibraltar
      MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
         reliability 255/255, txload 1/255, rxload 1/255
      Encapsulation ARPA, loopback not set
      Keepalive set (10 sec)
      Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
      input flow-control is off, output flow-control is unsupported
      ARP type: ARPA, ARP Timeout 04:00:00
      Last input never, output 00:00:00, output hang never
      Last clearing of "show interface" counters never
      Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
      Queueing strategy: fifo
      Output queue: 0/40 (size/max)
      5 minute input rate 0 bits/sec, 0 packets/sec
      5 minute output rate 9000 bits/sec, 8 packets/sec
         29836704 packets input, 5911943386 bytes, 0 no buffer
         Received 29835116 broadcasts (20101073 multicasts)
         0 runts, 0 giants, 0 throttles
         2 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
         0 watchdog, 20101073 multicast, 0 pause input
         0 input packets with dribble condition detected
         2256173 packets output, 273479815 bytes, 0 underruns
         0 output errors, 0 collisions, 4 interface resets
         0 unknown protocol drops
         0 babbles, 0 late collision, 0 deferred
         0 lost carrier, 0 no carrier, 0 pause output
         0 output buffer failures, 0 output buffers swapped out
    
    GigabitEthernet1/0/13 is up, line protocol is up (connected)
      Hardware is Gigabit Ethernet, address is 843d.c6de.938d (bia 843d.c6de.938d)
      MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
         reliability 255/255, txload 1/255, rxload 1/255
      Encapsulation ARPA, loopback not set
      Keepalive set (10 sec)
      Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
      input flow-control is off, output flow-control is unsupported
      ARP type: ARPA, ARP Timeout 04:00:00
      Last input never, output 00:00:00, output hang never
      Last clearing of "show interface" counters never
      Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
      Queueing strategy: fifo
      Output queue: 0/40 (size/max)
      5 minute input rate 5000 bits/sec, 4 packets/sec
      5 minute output rate 15000 bits/sec, 13 packets/sec
         384894 packets input, 168947177 bytes, 0 no buffer
         Received 5182 broadcasts (5135 multicasts)
         0 runts, 0 giants, 0 throttles
         0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
         0 watchdog, 5135 multicast, 0 pause input
         0 input packets with dribble condition detected
         31154750 packets output, 5933320875 bytes, 0 underruns
         0 output errors, 0 collisions, 2 interface resets
         0 unknown protocol drops
         0 babbles, 0 late collision, 0 deferred
         0 lost carrier, 0 no carrier, 0 pause output
         0 output buffer failures, 0 output buffers swapped out
    

    I see inbound & outbound traffic on the working non-VLAN tagged port 13, but no inbound traffic from pfSense on the tagged port 23

    alt text

    Pretty sure firewall rules on the root port shouldn't affect the VLAN, but added in a permit any as a futile act of desperation with no luck.

    alt text


  • Netgate

    You will never see traffic on igb2.10 because you are telling your switch to send that traffic untagged by setting 10 as the native vlan. igb2.10 is expecting traffic tagged with vlan 10.

    In your configuration, VLAN 10 on pfSense will be igb2, VLAN 3 will be igb2.3, VLAN 4 will be igb2.4, VLAN 11 will be igb2.11, etc.



  • @derelict said in VLAN fail on a SG-4860, what am I missing?:

    You will never see traffic on igb2.10 because you are telling your switch to send that traffic untagged by setting 10 as the native vlan. igb2.10 is expecting traffic tagged with vlan 10.

    In your configuration, VLAN 10 on pfSense will be igb2, VLAN 3 will be igb2.3, VLAN 4 will be igb2.4, VLAN 11 will be igb2.11, etc.

    Sweet, if I set the IP on OPT1 (igb2) instead of Home (igb2.10) i get pings.

    If I remove the default VLAN (no switchport trunk native vlan 10) on that switchport then properly reassign the IP back to igb2.10 in pfSense I get pings as well.

    From a security point of view I'm going to assume removing the default VLAN & only permitting tagged traffic on that trunk port is going to be more secure, correct? It would also make it more explicitly visible in the pfSense configuration that it's VLAN 10 rather than just an interface. God only knows how much I need all the help I can get keeping things straightened out at times. Any downsides to this?


  • Netgate

    You can do either. If you don't want untagged traffic on that link, don't. It you don't care, do.

    You have moved the untagged traffic off of VLAN 1 (the actual default VLAN, not to be confused with the native VLAN on a port) and changed the native traffic to VLAN 10. VLAN 1 has historically been the avenue where people can make mistakes with mixing tagged and untagged traffic.



  • lol, so many VLAN issues & misconfigurations in my lab (home network) now that I finally have a router online. Thanks for getting me pointed in the right direction on that roadblock that was killing me for days.