mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.

johnpoz

So your blocking outbound traffic?

Lets see the details of this rule please
1000001570

You can view your antispoof rules like this

[2.4.5-RELEASE][admin@sg4860.local.lan]/: cat /tmp/rules.debug | grep spoof
antispoof  for $WAN tracker 1000001570
antispoof  for $LAN tracker 1000002620
antispoof  for $WLAN tracker 1000003670
antispoof  for $TEST tracker 1000004720
antispoof  for $NS1VPN tracker 1000005770
antispoof  for $W_PSK tracker 1000006820
antispoof  for $W_GUEST tracker 1000007870
antispoof  for $W_ROKU tracker 1000008920
antispoof  for $DMZ tracker 1000009970
[2.4.5-RELEASE][admin@sg4860.local.lan]/:

From there you have the ID of the rules... Then you can look at a specific rule by ID via

[2.4.5-RELEASE][admin@sg4860.local.lan]/: pfctl -vvsr | grep 1000002620
@65(1000002620) block drop in on ! igb0 inet6 from fd80:80c6:5beb:6b7c::/64 to any
@66(1000002620) block drop in on igb0 inet6 from fe80::208:a2ff:fe0c:e624 to any
@67(1000002620) block drop in inet6 from fd80:80c6:5beb:6b7c::1 to any
@68(1000002620) block drop in on ! igb0 inet from 192.168.9.0/24 to any
@69(1000002620) block drop in inet from 192.168.9.253 to any
[2.4.5-RELEASE][admin@sg4860.local.lan]/:

that 1570 rule should be your wan antispoof.. But it is an inbound rule..

Again how do you have this physically wired??? Your doing vlans? What interfaces are connected where, and what vlans are on what.. And where do those wires go? I can an image a issue with this if your doing vlans and your isolation is not correct for your L2 networks.. Where pfsense would be able to see that as inbound traffic to its own interface after it put it on the wire from a vlan.. You stated you have avahi setup..

Keep in mind that that traffic from a vlan would have the same mac address as the physical interface the vlan sits on..

A drawing showing physical and logical setup would help us track down your problem.. How many physical interfaces do your pfsense box have? Where the vlans are? What switch are you using? It supports vlans? And is configured correctly?

Lets see output of ifconfig on pfsense.. example

Here is physical interface

igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6500bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:08:a2:0c:e6:20
        hwaddr 00:08:a2:0c:e6:20
        inet6 fe80::208:a2ff:fe0c:e620%igb2 prefixlen 64 scopeid 0x3
        inet 192.168.2.253 netmask 0xffffff00 broadcast 192.168.2.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

Here is a vlan that sits on that physical.. Notice the ether address is the same.

igb2.6: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:08:a2:0c:e6:20
        inet6 fe80::208:a2ff:fe0c:e620%igb2.6 prefixlen 64 scopeid 0xc
        inet 192.168.6.253 netmask 0xffffff00 broadcast 192.168.6.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        vlan: 6 vlanpcp: 0 parent interface: igb2
        groups: vlan

@johnpoz said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

cat /tmp/rules.debug | grep spoof

Thanks Johnpoz! I must admit this is fairly new still to me, as I am learning my way within pfSense, so bear with me :)
Here's the output of 'cat /tmp/rules.debug | grep spoof'.:

antispoof  for $WAN tracker 1000001570
antispoof  for $MGMT tracker 1000002620
antispoof  for $TRUSTED tracker 1000003670
antispoof  for $IOT tracker 1000004720

The specific rule ID is as follows (what I would expect, dropping traffic IN, but allowing OUT):

@27(1000001570) block drop in on ! igb0 inet from WANIP/23 to any
@28(1000001570) block drop in inet from WANIP to any
@29(1000001570) block drop in on igb0 inet6 from WANIP to any

I indeed have VLANs configured within pfSense, and configured on the UniFi switch. This works well with other devices that I have attached (e.g. a printer, or wireless devices like Chromecasts or e-readers).

My pfSense box (APU2D4) has 3 interfaces, of which the 3rd one is not configured yet. The switch supports VLANs and is a UniFi 8 port 150W PoE switch.
igb0 is my WAN interface which goes to the cable modem.
igb1 is my LAN interface, that has 2 VLANs configured: igb1.10 (TRUSTED VLAN) and igb1.20 (IOT VLAN). This goes into the UniFi switch with the port profile all (which resembles a trunk port).
The TV is connected to a port on the UniFi switch that has the IOT port profile configured. It gets the correct IP address assigned that I have set as DHCP reservation in pfSense. All of that works just good.

Here's the output of the interfaces. First igb0 (LAN):

igb1: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6500bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:0d:b9:52:d9:59
	hwaddr 00:0d:b9:52:d9:59
	inet6 fe80::20d:b9ff:fe52:d959%igb1 prefixlen 64 scopeid 0x2
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
	inet 10.10.10.1 netmask 0xffffffff broadcast 10.10.10.1
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

And the VLANs:

igb1.10: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:0d:b9:52:d9:59
	inet6 fe80::20d:b9ff:fe52:d959%igb1.10 prefixlen 64 scopeid 0x8
	inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	vlan: 10 vlanpcp: 0 parent interface: igb1
	groups: vlan
igb1.20: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:0d:b9:52:d9:59
	inet6 fe80::20d:b9ff:fe52:d959%igb1.20 prefixlen 64 scopeid 0x9
	inet 192.168.20.1 netmask 0xffffff00 broadcast 192.168.20.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	vlan: 20 vlanpcp: 0 parent interface: igb1
	groups: vlan

johnpoz

So igb0 is from pfsense to the cable modem. just a wire between these 2 devices.. There are no other wires on your "modem" that connect into your switching environment.. This cable modem is not providing wireless is it?

If your wan and lan side L2 networks were isolated as they should be - it would be impossible for wan side interface to see any traffic from the lan side be it broadcast or multicast or even unicast.

@johnpoz said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

So igb0 is from pfsense to the cable modem. just a wire between these 2 devices.. There are no other wires on your "modem" that connect into your switching environment.. This cable modem is not providing wireless is it?

If your wan and lan side L2 networks were isolated as they should be - it would be impossible for wan side interface to see any traffic from the lan side be it broadcast or multicast or even unicast.

Yes, that is correct. My modem just has the pfSense firewall attached to it (other ports are disabled as the modem is in bridge mode). Routing and wireless capabilities of the modem are disabled.

Only one flotating WAN rule is configured, leaving the defaults intact (default block ingress), which is an outbound WAN rule to mitigate bufferbloat as outlined here https://forum.netgate.com/topic/112527/playing-with-fq_codel-in-2-4/815.
Once I disable the rule, the mDNS reject and log rule I temporarily put in place still logs, even when I cleared all states.

On the IOT and TRUSTED VLAN I have just allowed mDNS traffic to 224.0.0.251, UDP on port 5353. IP Options are ticked in the rule. It surprises me that it seems to take a route towards the WAN address.

Do you have any other ideas on what I could check please? Thanks for your patience by the way and your willingness to help out, much appreciated!

claudio69

Hi, I just had a problem similar to yours on the WAN connected to the bridge modem that uses the PPPOE protocol.
In my case it was due to the USB port on the TP-Link modem which was not completely disabled.
In your case it seems that something that does this problem may be enabled on the modem.
It's just my idea and it could be another problem, in this case I apologize for the intervention

johnpoz

@claudio69 said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

In my case it was due to the USB port on the TP-Link modem which was not completely disabled.

Huh? How would USB port cause this.. That makes no sense either, just like this issue.

I don't see how this could be happening if his wan and lan are actually isolated from each other how stated.. Lets say for whatever reason pfsense was putting this traffic out his wan port.. I don't see how the firewall rule would be trigger on that as ingress traffic..

Only way that could happen is if something on the wan network was repeating the traffic back to the wire.. Like his cable modem, or something upstream?

He hid that last part of the mac, but said it was his wan mac.. Could very well have been his lan mac.. they are only going to normally be off by like 1.. And then if he has wan and lan connected - then this would make sense how he is seeing the traffic.

This post is deleted!

Do you have any advice on what else I could check? If not, thanks a lot and I will remain to have that reject rule in place which gets rid of it.

johnpoz

I can try and call in @Derelict, he might have some ideas..

Reject rule isn't getting rid of anything.. Its just not logging it.. And with a reject rule your just putting more noise onto the wire.. If you just don't want to see it, just drop and not log it..

There is clearly a piece of the puzzle missing... But what your saying isn't really possible.. Pfsense would have zero reason to put such traffic on the wire, and if it did put that on the wire it wouldn't log it as blocked since the rules are ingress not egress.

Turn off your avahi setup - does it go away? But I don't see how that could be it even - since even if you were sending it out via that.. Again the rules shouldn't log it.

Can you sniff that traffic again, and download the capture and send it to me... I can give you my email address, or you might be able to attach it via a PM.. Or you could really just post it here, it doesn't have your public IP in it, and it shouldn't have any sort of actual info in it you should be worried about.

Also like to see sniffs from your lan interfaces as well.. If pfsense was sending it on from your lan, then the traffic would be there as well..

Derelict

My initial thought would be you installed avahi and told it to use WAN.

johnpoz

Yeah I thought that as well - but why would that trigger what clearly is an ingress only rule - is it not?

Those antispoof rules are clearly listed as drop "in" Could the modem be reflecting it back?

@Derelict said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

My initial thought would be you installed avahi and told it to use WAN.

Hi @Derelict, I can tell you that is not the case. I have selected 2 LAN interfaces. I also believe you can only make it listen to WAN by manually editing code, which of course I would not consider, given the security risk you introduce.

@johnpoz I will have to test with Avahi off indeed to see if the traffic is still seen in the logs.
I am also doing some new pcaps that I can provide.

Meanwhile, I found another topic on the Netgate forum, exactly the same, with no response: https://forum.netgate.com/topic/142877/avahi-listening-on-wan-interface, as well as on Reddit: https://www.reddit.com/r/PFSENSE/comments/7t78bq/avahi_plugin_generating_firewall_block_log/. So I am not the only one :-). Responses in those topics did not yield any solution however, so I have to keep on troubleshooting.

johnpoz

So I downloaded avahi, and will duplicate sending mdns from 1 vlan to another vlan..

Please post up your settings..

johnpoz

Ok I installed avahi.. I then had it setup doing my lan and wlan

I then started a mdns-scan on linux box on the wlan network..

It floods the network with mdns

root@NewUC:/home/user# mdns-scan
+ NAS._http._tcp.local   
+ NAS._sftp._tcp.local
+ NAS._ftp._tcp.local
+ NAS._afpovertcp._tcp.local
+ NAS._smb._tcp.local
+ NAS._device-info._tcp.local
+ NAS._adisk._tcp.local
+ Brother HL-3170CDW series._pdl-datastream._tcp.local
+ Brother HL-3170CDW series._printer._tcp.local
+ Brother HL-3170CDW series._ipp._tcp.local
+ Brother HL-3170CDW series._ipps._tcp.local
+ Brother HL-3170CDW series._ipp-tls._tcp.local
+ Brother HL-3170CDW series._http._tcp.local
+ John's Air2._companion-link._tcp.local

As you can see it finds my nas that is on my lan network, and printer that is on my wlan network... And I can see the packets being forwarded via avahi.. And it also finds my ipad it seems on the wlan network.

Which I have setup like this

with only lan and wlan selected..

I see no blocks about antispoof.. If I sniff on my wan I see zero mdns traffic...

I then turn off avahi and now mdns-scan does not find my nas.. It only finds stuff on that same network printer and my ipad, which is the 192.168.2/24, and my lan above is the 192.168.9/24

So I can not duplicate what your seeing.. So your going to have to provide more info.. Lets see if I select my wan, in avahi if I see blocks... I don't think I would because its not ingress it would be egress..

BRB..

Well that is impossible - it doesn't list my wan to pick... So again I can not duplicate your problem.. Using avahi to allow mdns across 2 networks.. Works just fine... No traffic is sent out wan interface, via sniff.. No logs are being seen about antispoof... mdns traffic is being forwarded via avahi..

So there is either something else going on with your install.. Or you have a loop in your network?? Or what you think is your wan is not really your wan? Do you have PPPoe? Do you have multiple wans? etc etc.. Are you using other packages as well for multicast, igmp proxy, pimd? Do you not have a gateway on your wan interface, and manually set a gateway? Are you not doing nat on your wan... What is different with your setup other than default out of the box turn it on pfsense..

I can not duplicate your issue, and from my testing avahi does exactly what its suppose to do.

JeGr

Just as a thought, @MG85 tcpdump showed

19:09:14.933859 00:0d:b9:52:xx:xx > 01:00:5e:00:xx:xx, ethertype IPv4 (0x0800), length 88: (tos 0x0, ttl 1, id 64688, offset 0, flags [none], proto UDP (17), length 74)
    WANIP.20743 > 224.0.0.251.5353: [bad udp cksum 0x996d -> 0x4a51!] 2400 PTR (QU)? _services._dns-sd._udp.local. (46)
19:09:14.934402 00:0d:b9:52:xx:xx > 01:00:5e:00:xx:xx, ethertype IPv4 (0x0800), length 119: (tos 0x0, ttl 1, id 20152, offset 0, flags [none], proto UDP (17), length 105)
    WANIP.20743 > 224.0.0.251.5353: [bad udp cksum 0x998c -> 0x75b7!] 2401 [2q] PTR (QU)? _sonos._tcp.local. PTR (QU)? Sonos-949F3E8F1EBA._sonos._tcp.local. (77)
19:09:14.934685 00:0d:b9:52:xx:xx > 01:00:5e:00:xx:xx, ethertype IPv4 (0x0800), length 88: (tos 0x0, ttl 1, id 46782, offset 0, flags [none], proto UDP (17), length 74)
    WANIP.20743 > 224.0.0.251.5353: [bad udp cksum 0x996d -> 0x4a4f!] 2402 PTR (QU)? _services._dns-sd._udp.local. (46)
19:09:14.935472 00:0d:b9:52:xx:xx > 01:00:5e:00:xx:xx, ethertype IPv4 (0x0800), length 203: (tos 0x0, ttl 1, id 53220, offset 0, flags [none], proto UDP (17), length 189)
    WANIP.20743 > 224.0.0.251.5353: [bad udp cksum 0x99e0 -> 0x5fba!] 2403 [4q] PTR (QU)? _sonos._tcp.local. PTR (QU)? _spotify-connect._tcp.local. PTR (QU)? sonos7828CA338235._spotify-connect._tcp.local. PTR (QU)? Sonos-7828CA338235._sonos._tcp.local. (161)
19:09:14.936255 00:0d:b9:52:xx:xx > 01:00:5e:00:xx:xx, ethertype IPv4 (0x0800), length 119: (tos 0x0, ttl 1, id 3105, offset 0, flags [none], proto UDP (17), length 105)

Would this be what you are looking for? '00:0d:b9:52:xx:xx' is the MAC address of the WAN interface of pfSense.

And in the PCAPs the PTR that is called is _sonos._tcp.local and/or/for spotify-connect etc. etc.

So that doesn't strike me as something coming from his inside from an LG TV?

@MG85 are you running Sonos devices?

If not I'd guess perhaps someone has a faulty setup connected to some Router/Modem that is on the same upstream switch from the ISP and is spweing that out to all devices on the same uplink card/box from the ISP?

I'm running german cable internet on my box and boy do I see weird shit in that segment from some other modems/routers connectedd to the same distribution box as my line...

Derelict

But the source MAC is claimed to be the pfSense WAN MAC.

(Please don't redact things like private addresses and MAC addresses. It only makes it harder to help you.)

JeGr

Ah because of the redaction I didn't see that... damn ;)

Nevertheless Sonos-949F3E8F1EBA seems like a "normal" Sonos device for me (including its MAC in the name, like they commonly do). So I'd check things on my IOT net if there are Sonos speakers in action or drop&block that on WAN if I don't have Sonos running :)

@johnpoz @Derelict @JeGr after the great tips you guys have been sharing, I have managed to find the ultimate root cause.

I filtered again the states of the WAN interface, filtering on port 5353. You can see that there is traffic initiated.
I redacted the WAN IP address, but left the port visible. This is of course not what I expected, as mDNS normally only establishes traffic over UDP/5353. In this case it's 43075.

Secondly, I went into the states again, filtering on the IOT interface this time. This is the result:

Here you can see that the TV (192.168.20.2) initiated traffic to 224.0.0.251 with the local port 43075. Same port that I saw on WAN.

The firewall log shows the same traffic being dropped:

Right.. I am getting somewhere :-).
I went ahead and edited the mDNS allow rule to only allow mDNS traffic with the source port being 5353.

I never set the source port before, as I implemented things according to some nice tutorials on Youtube.
After the filters were reloaded, I deleted the single connection I mentioned earlier that was still open on the WAN interface.
I also noticed that the reject rule I put in place was no longer logging traffic. Great!

Summarized, it was indeed the TV that was using a random high port to establish mDNS traffic, and even though the destination was set to :5353, it seems to have taken a different route, towards WAN. Something I still can't explain why.
I have now corrected it, and it works like a charm.
@johnpoz @Derelict there is one thing however I would like to have your advice on, which is the fact that Avahi seems to be having 2 sockets open, one on :5353 and one on a random high port.
Do you see that as well in your setup? (Diagnostics > Sockets):

.

Again, MANY thanks for your help!

johnpoz

@MG85 said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

it seems to have taken a different route, towards WAN

This is the whole point!!! Doesn't matter what source port it was using... There should be no way it can get to wan.. You have a cross over in your L2 networks!!

You have Sonos - that create a bridge.. There is your problem with your L2 networks being connected!!!

But you said your wan is a wire to your modem, and your modem is not running wireless.. Clearly that is NOT the case and you have sonos connecting these two L2 networks together..

@johnpoz said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

@MG85 said in mDNS traffic from WAN to 224.0.0.251:5353, but why? Please help.:

it seems to have taken a different route, towards WAN

This is the whole point!!! Doesn't matter what source port it was using... There should be no way it can get to wan.. You have a cross over in your L2 networks!!

You have Sonos - that create a bridge.. There is your problem with your L2 networks being connected!!!

But you said your wan is a wire to your modem, and your modem is not running wireless.. Clearly that is NOT the case and you have sonos connecting these two L2 networks together..

Lol. What does SONOS have to do with it, while I have found out the TV to be the culprit? If I tell you the modem is in bridge mode with all routing and wireless features off, it is like that (it is already like that since we moved in here). I manually moved the speakers over to the IOT VLAN, so I know what I did.

For my SONOS speakers I have implemented PIMD which is running just fine for over a year (it even is released as an official package now ), and all of this with strict firewall rules. No strange stuff in the logs, no network loops whatsoever, clearly ruled out.
Yes, Sonos speakers do use multicast (but just for IGMP packets from the speakers and controllers to destination 224.0.0.22). Besides, SSDP is leveraged for discovery (239.255.255.250.1900 UDP) through the app, and broadcast to find other players within the network (which of course is non-routable anyway).

This stuff started to happen just after I connected the smart TV. And that was again confirmed as it flooded the logs.
Also, you did not answer my question about the sockets. I would be happy to see if you have just one socket open for Avahi on port 5353, or also have multiple, just like my last screenshot.

Thanks.