VLAN -> VLAN Routing Misbehaving



  • First off, the following has been done/verified:

    • End Device Firewalls do NOT block/drop ICMP, they respond to pings.
    • Firewall rules have been set up between then, these rules log
    • I have packet captured both interfaces
    • I have packet captured the target device (192.168.10.2) and saw no ICMP items

    The Problem

    I want to route between VLAN2 and VLAN10, I have routing rules setup to permit traffic between them, these rules are set to log (for debugging). On VLAN2 I try and access a service hosted on VLAN10 (or ping), and it just times out. When I packet capture VLAN2 (the interface, from pfsense) I see the requests as expected, when I packet capture VLAN10 none of those packets show up. However, the firewall rules have logged that they have passed the traffic for VLAN2 -> VLAN10.

    Same results in the opposite direction.

    I can ping devices and access services from within the same VLAN as expected.

    Network:

    *Simplified to remove other VLANS that exhibit the same behavior

    • VLAN2 - 192.168.10.0/24 - Clients
    • VLAN10 - 192.168.10.0/24 - Management
    • Downstream Switch: UniFi Switch PRO 24
      • Uplink is tagged for all VLANS
      • Target Device (VLAN10) port is tagged for VLAN10
      • Device sending pings (VLAN2) port is tagged for VLAN2
    • Note: LAGG is 2 ports bonded since I'm using router-on-a-stick for now, that works as expected.

    Screenshots & Info

    VLAN2 Packet Capture: (VLAN10 Packet capture is empty for ICMP)

    20:44:03.949586 IP 192.168.2.10 > 192.168.10.2: ICMP echo request, id 1, seq 84, length 40
    20:44:08.940697 IP 192.168.2.10 > 192.168.10.2: ICMP echo request, id 1, seq 85, length 40
    20:44:13.958786 IP 192.168.2.10 > 192.168.10.2: ICMP echo request, id 1, seq 86, length 40
    

    VLAN2 Rules:
    7769454f-e4e4-4ce2-8a66-27a5aaf44c0c-image.png

    VLAN10 Rules: b2298c8d-5820-4a47-88b0-f014f92728f6-image.png

    Relevant Interfaces:
    f9a958f1-3d74-400e-ab1a-c4a799a31047-image.png

    Firewall Pass Log Entries: (The one mgmt -> client was a ping the other direction I tried, same results.)
    d1e993af-2d7b-4f33-bcef-165257ff9ffa-image.png

    Oddities

    These may help diagnose, they may not help, but they are weird.

    • I can ping 192.168.10.1 (PFSense) from 192.168.10.2. However, I CANNOT ping 192.168.10.2 from 192.168.10.1.

    Packet capture when attempting to ping device from PFSense note the 192.168.2.3 bit? That's a freenas server, not sure why it's cropping up there?

    00:10:11.913248 IP 192.168.10.1 > 192.168.10.2: ICMP 192.168.2.3 udp port 111 unreachable, length 36
    00:10:22.260001 IP 192.168.10.1 > 192.168.10.2: ICMP 192.168.2.3 udp port 111 unreachable, length 36
    
    • I have the unifi controller on the same device with pfsense. However, I only allow access to the controller on the VLAN10 network, I can access it just fine from VLAN2 (Same routing rules log pass entries)


  • Side note, this was difficult to get posted, I kept getting:
    14c599ce-ea3c-4dc3-8b6a-aa45f8994017-image.png

    This doesn't help things either...:
    d7426e17-38ad-4bbd-9f1b-d89f80fbe295-image.png

    Last thing I promise: The preview doesn't match posts, the preview adds a newline before images, when posted that's gone, leaving text on the same line as it was typed.

    Don't let this distract form the OP though!! Just some feedback.



  • More info:

    If I try and ping via shell (root) to any VLAN other than VLAN2 I get ping: sendto: Permission denied. I can ping anything on VLAN2 and on the LAN interface.

    Is there some hidden routing issue?



  • @douglasg14b

    VLAN2 - 192.168.10.0/24 - Clients

    VLAN10 - 192.168.10.0/24 - Management

    Is this a typo? Cause if you're routing between these VLANS, the subnets have to be unique.



  • This is a question for the mods... Can you actually have a VLAN tag, on pfsense, as "02" or should it simply be "2"?

    Jeff



  • @marvosa Yeah, it's a typo 192.168.x.0, the "x" is the VLAN #. For some reason I can't reliably edit my post, nor can I post comments it keeps telling me "Post content was flagged as spam by Akismet.com"... This forum really needs to address that...

    I have solved this problem. I had a floating rule that blocked private ranges, which of course was blocking subnet -> subnet routing.


Log in to reply