Very Confused, A couple questions from new VLAN setup relating to MDNS & Firewall Rules

[striker-pl]

Hello,

I am trying to add some more control and security to my network. I'm still learning, so I've been reading forum posts and articles, but i'm really struggling with a few things. Hoping someone can help me understand what i'm missing!

I'll post a diagram below that shows my relevant network setup. I can post more info from my pfsense config if it's helpful.

My questions are:

1. I have an IOT device in VLAN40. It connects to its server and streams content without issue, however when I try to run a packet capture on its IP, I can't find its traffic. I'd like to setup some restrictive firewall rules to help lock down my IOT network, but this device is outsmarting me.

2. I would like to setup mDNS using Avahi service so my smart phone can cast to my smart TV without having to link via the YouTube TV code. I've installed Avahi and configured to run on LAN and IOTVLAN. I've created Any to Any firewall rules (on both IOTVLAN and regular LAN) and enabled "Allow IP Options" on the rules as well just to see if I can get things flowing before restricting traffic, but I can't get my phone to connect to my TV. Service shows that Avahi is running, so I'm not sure what next troubleshooting steps would be.

Happy to post other info if it's helpful, just trying to keep this initial post short and to the point.

Thanks!

[striker-pl]

perhaps a better way of phrasing one of my questions, is:

If I use the Packet Capture feature in pfsense, I target 10.0.140.103 (the media player on IOTVLAN) and I have the interface set to IOTVLAN, I would think that I could capture all traffic going from the media player back to the SageTV docker running on my Unraid server. However when I do this, I get no traffic captured.

Is anyone able to shed any insight on why this would be? As far as I understand it, all traffic needs to talk to pfSense to ensure it passes firewall rules, so I should have an opportunity to capture/inspect the traffic, but the behavior i'm seeing suggests that the traffic is not going to pfSense and is instead going directly between the docker and the player.

[jagradang]

@striker-pl I spent weeks trying to get carting working over vlans... Short answer Its a nightmare and doesn't work! I gave up and out everything on the same vlan and controlled iot devices with firewall rules instead.

[hieroglyph]

@striker-pl
TLDR:
The pfsense device does not see this LAN traffic between the SageTV Docker and Media Player because the switch between them sends the packets directly to each device. The switch has no need to "ask" the pfsense firewall rules if inter-LAN communication is allowed. It just assumes it is.

Long Answer:
From what I see the SageTV Docker (10.0.140.57) and the Media Player (10.0.140.103) are on the same VLAN network. I also see they have to go thru the HP-1810-24G switch (10.0.100.10) in order to talk to each other. When devices are on the same LAN-network if there is a switch that "connects" them the switch does not bother sending those packets to the pfsense device because it already knows where the two devices are. It is much more efficient for the switch to send the packets from the Media Player directly to the port the SageTV Docker is reachable on.

This is also why you cannot see the traffic between these two devices during a packet capture using the pfsense box.

Example1: If for instance the SageTV Docker and Media Player were directly connected to the pfsense box, you would see them communicating with each other.

Example2: If the SageTV Docker and Media Player were on different VLANs and the switches between them only had basic VLANning enabled (i.e. no routing capability enable on the switches) the switches would have to send the packets between SageTV and Media Player to the pfsense device so pfsense can use the firewall rules to decide if these two devices are allowed to talk to each other across different LANs.

Example3: Last example; if the SageTV Docker was plugged into switchA and switchA was plugged into port1 of the pfsense device. Along with the Media Player being plugged into switchB and switchB was plugged into port2 of the pfsense device. And switchA and switchB have no direct connections between them. Then the traffic from both devices would have to go thru the pfsense box to reach each other.

[striker-pl]

@hieroglyph thanks for the detailed explanation, that really helps!

Did you have any insights to offer towards my second question about repeating mDNS / UDP broadcast packets between VLAN's for device casting?

[hieroglyph]

@striker-pl
Here are some screenshots from my setup.

Avahi:

• Interface action is set to "Allow Intefaces" and the networks I want to allow mDNS/Avahi cross traffic I have selected.
• Idk if it is needed, but in Avahi under Advanced Settings > Override Domain I copied the text from System > General Setup > Domain into the text box

(Sounds like you did this, but going to spell it out for anyone else who may read this) Even though the Avahi/mDNS service has been enabled, there still needs to be firewall rules allowing traffic to traverse the different VLANs.

Firewall Rule:

Aliases:

• Multicast_Allowed_Devices = Devices/Networks allowed to initiate mDNS packets (i.e. 192.168.1.11, 192.168.2.0/24, etc...)
• 5_Port_mDNS = 5353
• 1_Device_Multicast_DNS_Address = 224.0.0.251 This is the mDNS/Avahi destination address

Reference:
The TCP/UDP Ports wiki page is were I go when I need to understand how a certain protocol (i.e. mDNS/Avahi) works. Search for "mDNS" or "5353" on this wiki page and select the Multicast DNS link it finds. About a third of the way down the Multicast DNS page it shows mDNS uses UDP, port 5353, and address 224.0.0.251.

Outbound NAT
I am no mDNS expert. But I think after the mDNS handshake is completed the two devices will need to send some data between them. When the phone (10.0.100.144) sends packets to any IP address not in the 10.0.100.0/24 network, the default NAT rules assume this traffic needs to exit on the WAN interface; 99% of the time this is true when you are checking the Insta Feed, shopping on Amazon, searching Google, etc... But in this case because the packets need to exit the VLAN40 interface and not the WAN interface an Outbound NAT rule needs to be created so pfsense knows to send traffic destined for 10.0.140.57 or the 10.0.140.0/24 network from the source 10.0.100.144 or the 10.0.100.0/24 network to pfsense's VLAN40 interface.

Aliases:

• All_Admin_VLAN_VPN_Networks = All of my networks (i.e. 192.168.1.0/24, 192.168.2.0/24, 192.168.3.0/24, etc...)
• 0_Network_VLAN10 = My critical/private VLAN network (i.e. 192.168.1.0/24)

This rule says; Allow traffic from ALL VLANS to be NAT'd to my critical/private VLAN network. I use these generic "Allow All NAT'd traffic" rules because it says on the Ordering Of NAT And Firewall Processing Page, the firewall rules are evaluated before NAT rules, so even though this could allow guest VLAN devices to access my critical/private network; my firewall rules do not allow that to happen. For example, I have firewall rules setup to allow guest traffic to stream from port:80 of my media server on my critical/private network, but drop all other guest traffic destined to any other ports on the media server and to drop all other guest traffic destined for critical/private IP addresses. I also do it this way so I do not have to make several NAT rules for each device; or remember to update another alias.

[johnpoz]

@hieroglyph said in Very Confused, A couple questions from new VLAN setup relating to MDNS & Firewall Rules:

the firewall rules are evaluated before NAT rules

No says the opposite of that..
https://docs.netgate.com/pfsense/en/latest/nat/process-order.html

In terms of how the ruleset is processed, the order is:

Outbound NAT rules
Inbound NAT rules such as Port Forwards (including rdr pass and UPnP)
Rules dynamically received from RADIUS for IPsec and OpenVPN clients
Internal automatic rules (pass and block for various items like lockout, snort, DHCP, etc.)
User-defined rules:
    Rules defined on the floating tab
    Rules defined on interface group tabs (Including IPsec and OpenVPN)
    Rules defined on interface tabs (WAN, LAN, OPTx, etc)
Automatic VPN rules

But I think after the mDNS handshake

Why would you think there is some sort of handshake? Its UDP..

[hieroglyph]

@johnpoz
Ok, I must not understand what I was reading here about LAN initiated traffic:

I see below it, it also says WAN initiated traffic has the order reversed. But it still shows them in the same order. So yeah, I am definitely not understanding something here.

And you are correct. Now you point out it is UDP it does not make sense for there to be a handshake.

[johnpoz]

No offense but I really didn't get what you were going on about. The blatant wrong stuff jumped out at me like firewall rules before nat, and handshakes for mdns??

I had to point out the wrong stuff for the next guy for sure..

I take it what you were trying to say is clients on the same network don't talk to the router to talk to other devices on the same network - be it L2 or L3.. Which is trying to help - so thank for contributing..

But maybe you should re-evaluated how you have you rules setup if you think you should nat between your own network, or that it comes into play when routing between your own local networks.. You for sure shouldn't be setting up nat betwen rfc1918 and rfc1918..

[striker-pl]

@johnpoz @hieroglyph I appreciate the responses!

if implementing NAT rules isn't the proper way to go, are there any other suggestions as to why my setup isn't working? It seems like the Avahi tool should be fairly plug'n'play but it doesn't seem to be that way for me....

[johnpoz]

Avahi is not setting up 2 networks. That is how you break L2 boundaries for your multicast traffic.

Which maybe what your trying to do with that doesn't work with Avahi??

What exactly is not working.. What I would do is get your different vlans setup, no natting between them. Make sure devices can talk to each other. Then worry about whatever your trying to do with multicast across vlans.

[striker-pl]

@johnpoz I think I do have the VLANs setup and traffic working between them. I can ping back and forth etc. Workstations can access the internet. Workstations can print to the printer (although they can't auto discover it). Things are failing when I want to cast from a phone to a smart tv. Right now we have to use the "enter a code" method to connect to the TV instead of being able to discover it and connect to it more easily.

I'm specifically looking for the YouTube app to work for device discovery when casting.

I'm using a Pixel 5 and my wife is using an iPhone XR. The TV in question is a Sony Bravia running Android OS.

From the videos and reading i've found, Avahi is supposed to help enable this (unless i'm misunderstanding something).