multicast inconsistant

maximushugus

Hello,
I have a weird problem with my pfSense since I recently updated to 2.7.2 : it looks as if my pfSense router doesn't receive some multicast packets.

Here is my configuration : my pfSense is connected via 2 links to a switch, with a lagg. I have VLANs on this LAGG.
I have IGMP snooping enabled on my switch.
I have IGMP proxy enabled on my pfSense so I can access my ISP IPTV.
This configuration was working perfecly but it looks as if it broke after updating to 2.7.2 (from 2.7.0).
I have "allowopts" enabled on a rule on my LAN firewall on pfSense

Here is what I found :

If I do a packet capture on the LAN side of my pfSense, and filter packets to see IGMP packets, pfSense only receive MDNS multicast packets, or IGMP membership report from local adresses (of pfSense itself) or IGMP Leave group, but it doesn't receive IGMP membership report group from other IP on my LAN.
Even if I didn't change my switch configuration, I wanted to see if those missing IGMP packets were hitting my pfSense, so on the switch I did a port miroring of my LAGG (on witch is connect pfSense) to another port, and I did a packet capture with Wireshark on a PC connected to this mirror port : on this capture I can see those missing IGMP report group from other IP on my LAN. So those packets seems to be going to my pfSense router.
I also tried to disable IGMP snooping on my switch but it didn't change anything

Does someone knows why this is happening. Maybe there is a firewall block but I'm not able to see it.

I can show my packet captures if needed.

Thanks a lot for your help

johnpoz

@maximushugus said in multicast inconsistant:

Maybe there is a firewall block but I'm not able to see it.

While pfsense would or should see multicast from any client on a network its attached too.. Pfsense has zero to do with devices on the same network talking multicast to each other..

JonathanLee

There is a multicast package you could install to allow multicast traffic to be implemented correctly.

maximushugus

My pfSense box should see a PC on my LAN sending a membership report to a mutlicast address (for example 239.255.255.250)

JonathanLee

@maximushugus I would try out that multicast package that should fix your issues.

maximushugus

@JonathanLee thanks for your answer. What package are you talking about ?
Before updating I didn't have any package installed for multicast (execpt IGMPproxy)

maximushugus

For example you can compare the capture of IGMP packets for my pfSense and from the port mirroring of my switch :
pfsense capture.pcap miroring.pcapng

PS : on the mirroring capture you can see all VLANs (so you see 192.168.3.0 and 192.168.99.0 subnets)

JonathanLee

@maximushugus I have to look last time I was researching this it was related to gaming systems. I have to look it up again after I get back.

JonathanLee

Avahi Package!!

johnpoz

@JonathanLee avahi is for mdns only.

@maximushugus mentions IGMP.. And not working with update - this would indicate that he is running into the IP options that is new..

JonathanLee

@johnpoz it’s multicast dns right and others. I was reading about this a while ago.

johnpoz

@JonathanLee avahi doesn't do anything other than mdns, which is multicast sure, on a specific address and port 5353, it not going to do anything with some IPTV connection over multicast.

Pfsense has recently enabled IGMP filter for ip options. Even a rule would of allowed the traffic, the rule also has to allow for IP options or the traffic with IP options set will be blocked..

There has been multiple multiple threads about it.

https://forum.netgate.com/post/1167069

dennypage

@johnpoz said in multicast inconsistant:

Pfsense has recently enabled IGMP filter for ip options. Even a rule would of allowed the traffic, the rule also has to allow for IP options or the traffic with IP options set will be blocked..

I don't believe that this represents a change in packet behavior. I believe that previously IGMP packets were still blocked unless the IP option box was checked, even though there was no logging of it.

johnpoz

@dennypage That is quite possible.. I don't do anything with igmp, and I even block quite a bit of multicast at the switch level..

So I haven't looked into the exact details.. But I thought I saw a thread around here where it was stated it was allowed before - which was kind a security issue, and the new blocking without allowing ip options was a security enhancement.. I will see if I can dig up what I at least thought I read.

Maybe I misinterpreted this statement in the post I linked too?

"actually the correct behaviour but was broken in previous versions."

The broken in previous versions could be interpreted a few different ways, like before they were allowed or just which rule was listed as blocking them..

The take away I would say is, if you want to pass igmp, you should allow for IP options ;)

dennypage

@johnpoz I think this is probably the most clear statement I saw was from the redmine:

"Specifically, the code to check for options is this: https://cgit.freebsd.org/src/tree/sys/netpfil/pf/pf.c#n8312
It forces logging to be enabled if it drops a packet due to IP options. This used to be overwritten again so no logging happened, but that's been fixed in https://cgit.freebsd.org/src/commit/sys/netpfil/pf/pf.c?id=5f840a1758b4bbb4892118f43f40c6487c17aeba".

Of course, it's possible I misinterpreted the statement. I didn't dig through the kernel code.

maximushugus

Thanks for your answers.
But I still have this problem. As you can see, It looks as if pfSense is not seeing some multicast packets (see the difference between the packet capture from the switch and from pfSense).
Does someone know what is the problem and how to fix it ?

johnpoz

@maximushugus if your not seeing it via a sniff on pfsense, that has nothing to do with firewall rules on pfsense or blocking or not blocking igmp with ip options set.. Because the sniff would happen before any firewall rules were used.. Now up the stack could be blocked from "seeing" that traffic - but sniff at the wire would show it.

If pfsense is not seeing the traffic via sniff points to switch not sending it out the port pfsense is connected too.. Or your sniffing on a vlan and the traffic is not tagged, etc.

maximushugus

@johnpoz I agree with you but what is strange is that my problem happened after updating pfSense, and at this moment I didn't touch my switch, that's why I was searching for a pfSense problem

johnpoz

@maximushugus I hear ya - but I am not aware of anything in pfsense that would prevent a "sniff" ie packet capture from seeing the traffic coming into the interface.. If that traffic gets passed up the stack to be further processed either by something running on pfsense or routed, etc. sure... But at the sniff, if not seeing it on the sniff - that tells me its not there.

The only thing would be if your sniffing for traffic tagged for vlan X, but the traffic itself is not tagged.. You wouldn't see it then in your capture.

You know who might be good resource - @bmeeks he handles all the IPS stuff, fairly sure he would know if something could prevent being seen at the sniff level in pfsense.

I can't recall ever running into a scenario where not being seen by a packet capture, but traffic actually there - unless you not sniffing for the actual traffic, be it wrong vlan related or wrong interface, wrong protocol or wrong ip or port, etc... And I have been doing this a really long time ;)

stephenw10

Driver issue can hide traffic from pcaps. The ix driver had a bug that was filtering vlan0 for example. pcap showed no traffic.