VLAN interface on WAN interface not tagging frames



  • Synopsis:

    Can't access management VLAN 10 that's configured on Microtik from pfSense (ARP entry incomplete on pfSense, suspect VLAN tagging issue).

    Can access management VLAN 10 that's configured on Microtik via StarTech ASIX USB NIC with VLAN 10 configured on it from PC via same cable (unplug pfSense > WAN > ethernet cable, plug into USB NIC on PC).

    Hardware:

    Microtik wAP LTE Kit (Latest RouterOS; single NIC with built-in LTE modem) patched into WAN (igb0) of
    ALIX 2d4 running pfSense 2.4.4-RELEASE-p1

    Microtik is configured as a LTE bridge to ether1; config here if you know/care about RouterOS:

    aug/07/2019 17:32:03 by RouterOS 6.45.2
    software id = E3Q7-3NTE
    
    model = RouterBOARD wAP R-2nD
    serial number = xxxxxxx46
    /interface lte
    set [ find ] mac-address=xx:xx:xx:xx:xx:31 mtu=1480 name=lte1
    /interface wireless
    set [ find default-name=wlan1 ] ssid=MikroTik
    /interface vlan
    add interface=ether1 name=mgmt vlan-id=10
    /interface lte apn
    set [ find default=yes ] apn=inet.someisp.com passthrough-interface=ether1 \
        passthrough-mac=xx:xx:xx:xx:E7:9A
    /interface wireless security-profiles
    set [ find default=yes ] supplicant-identity=MikroTik
    /ip address
    add address=192.168.88.1/24 interface=mgmt network=192.168.88.0
    /system clock
    set time-zone-name=America/New_York
    

    pfSense:

    I have configured VLAN 10 on the WAN interface (igb0) by following these steps:

    Interfaces > Assignments then VLANs tab.

    Add VLAN:
    Interface: igb0 (wan)
    VLAN tag: 10
    Priority: (blank)
    Description: Microtik mgmt

    back to Interfaces > Assignments
    Add available network ports: "VLAN 10 on igb0 - wan (Microtik mgmt)"

    Edit Interfaces/LTEMGMT (igb0.10)
    Enable Interface
    IPv4 configuration type: Static IPv4
    Static IPv4 configuration:
    192.168.88.254/24

    Under Reserved Networks, block private/bogon unchecked.

    Cannot ping from pfSense > Diagnostics and Diagnostics > ARP Table shows incomplete entry for 192.168.88.1, which means it's still waiting for an ARP reply.

    Packet capture on LTEMGMT interface (host 192.168.88.1, full, promiscuous mode) with another tab open and running a Diagnostics > Ping (10 packets, source interface LTEMGMT, destination host 192.168.88.1) just shows the ARP Request who-has responses). Ethernet frames do not show VLAN in the header information in WireShark.

    Thanks in advance.


  • LAYER 8 Netgate

    Whatever you are connecting there needs to expect that traffic with a VLAN tag of 10.

    If that was the case it would be working. Cannot speak to whether or not that MikroTik configuration is correct.



  • @Derelict said in VLAN interface on WAN interface not tagging frames:

    Whatever you are connecting there needs to expect that traffic with a VLAN tag of 10.

    While I don't know about his situation, business connections over fibre will often use VLANs even 2 levels of it (Q-in-Q).



  • @Derelict as stated above, if I simply take the cable from the pfSense WAN port and plug it into a USB 3.0 NIC on my PC that has VLAN 10 tagged on it, it works fine.


  • Galactic Empire

    @gravyface said in VLAN interface on WAN interface not tagging frames:

    Can't access management VLAN 10 that's configured on Microtik from pfSense (ARP entry incomplete on pfSense, suspect VLAN tagging issue).
    Can access management VLAN 10 that's configured on Microtik via StarTech ASIX USB NIC with VLAN 10 configured on it from PC via same cable (unplug pfSense > WAN > ethernet cable, plug into USB NIC on PC).

    Do a packetcapture on the WAN interface, open it up in Wireshark and create a column for vlan and use the following, that will tell you if pfSense is tagging or not.

    Screenshot 2019-08-12 at 13.33.47.png



  • @NogBadTheBad I did; it is not. But nice tip re: adding a column for VLAN ID.

    I210-AT Intel NICs on that ALIX do support VLANs too.


  • Galactic Empire

    @gravyface said in VLAN interface on WAN interface not tagging frames:

    I210-AT

    No worries, it was worth a try ☺


  • LAYER 8 Netgate

    I have certainly never heard of an igb interface not supporting VLAN tags.

    Post Interfaces > Assignments and, for good measure, the output of ifconfig -vma.



  • @Derelict

    [2.4.4-RELEASE][root@pfSense.localdomain]/root: ifconfig -vma
    igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=6500bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                                                                                                                                                             UM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
            capabilities=753fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN                                                                                                                                                             _HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NET                                                                                                                                                             MAP,RXCSUM_IPV6,TXCSUM_IPV6>
            ether 00:1f:29:bc:e7:9a
            hwaddr 00:0d:b9:52:3b:e8
            inet6 fe80::20d:b9ff:fe52:3be8%igb0 prefixlen 64 scopeid 0x1
            inet xx.xx.xx.220 netmask 0xfffffff8 broadcast xx.xx.xx.223
            nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
            supported media:
                    media autoselect
                    media 1000baseT
                    media 1000baseT mediaopt full-duplex
                    media 100baseTX mediaopt full-duplex
                    media 100baseTX
                    media 10baseT/UTP mediaopt full-duplex
                    media 10baseT/UTP
    igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=6400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                                                                                                                                                             UM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
            capabilities=753fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN                                                                                                                                                             _HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NET                                                                                                                                                             MAP,RXCSUM_IPV6,TXCSUM_IPV6>
            ether 00:0d:b9:52:3b:e9
            hwaddr 00:0d:b9:52:3b:e9
            inet 10.171.1.1 netmask 0xffffff00 broadcast 10.171.1.255
            inet6 fe80::1:1%igb1 prefixlen 64 scopeid 0x2
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            supported media:
                    media autoselect
                    media 1000baseT
                    media 1000baseT mediaopt full-duplex
                    media 100baseTX mediaopt full-duplex
                    media 100baseTX
                    media 10baseT/UTP mediaopt full-duplex
                    media 10baseT/UTP
    igb2: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                                                                                                                                                             UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
            capabilities=753fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN                                                                                                                                                             _HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NET                                                                                                                                                             MAP,RXCSUM_IPV6,TXCSUM_IPV6>
            ether 00:0d:b9:52:3b:ea
            hwaddr 00:0d:b9:52:3b:ea
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            media: Ethernet autoselect
            status: no carrier
            supported media:
                    media autoselect
                    media 1000baseT
                    media 1000baseT mediaopt full-duplex
                    media 100baseTX mediaopt full-duplex
                    media 100baseTX
                    media 10baseT/UTP mediaopt full-duplex
                    media 10baseT/UTP
    enc0: flags=41<UP,RUNNING> metric 0 mtu 1536
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            groups: enc
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
            options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
            capabilities=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
            inet 127.0.0.1 netmask 0xff000000
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            groups: lo
    pflog0: flags=100<PROMISC> metric 0 mtu 33160
            groups: pflog
    pfsync0: flags=0<> metric 0 mtu 1500
            groups: pfsync
            syncpeer: 224.0.0.240 maxupd: 128 defer: on
            syncok: 1
    igb0.10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
            capabilities=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
            ether 00:1f:29:bc:e7:9a
            inet6 fe80::20d:b9ff:fe52:3be8%igb0.10 prefixlen 64 scopeid 0x8
            inet 192.168.88.254 netmask 0xffffff00 broadcast 192.168.88.255
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
            supported media:
                    media autoselect
            vlan: 10 vlanpcp: 0 parent interface: igb0
            groups: vlan
    
    

    interface.assignments.PNG



  • Something is definitely unstable with this Microtik: I've attempted to remove the MAC address restriction from the passthrough options and it's now unresponsive.


  • LAYER 8 Netgate

    I have never, ever, seen an igb port (or any port) not tag in that case. I would look elsewhere for the problem.

    You will not see VLAN tags capturing on LTEMGMT there. You will have to capture on WAN.

    If you don't want to trust pfSense's tcpdump/packet capture, capture on a mirror port on a switch between igb0 and the mikrotik/wan.



  • I think the problem may be due to the fact that parent/child interfaces share the same MAC address. I have passthrough enabled on the lte1/ether1 interfaces, which is locked to the MAC address of pfSense's WAN interface, but on the same physical interface, igb0.10 shares the same MAC. Might be throwing off the Mikrotik.


  • LAYER 8 Netgate

    That is 100% expected for VLANs.



  • @Derelict I changed it via ifconfig and it didn't make a difference anyways.



  • Wondering if I'd have better luck getting the Sierra Wireless MC7700 running on the ALIX and ditch the Microtik (which honestly feels kind of Fisher Price to me).



  • @Derelict Ok, found a Microtik post on the parameters around the passthrough and it will reject traffic from a device with the same MAC as the passthrough device. As a workaround, you can create another VLAN interface on Microtik (I created VLAN 11) and did likewise on the pfSense.


Log in to reply