multicast inconsistant

maximushugus

For example you can compare the capture of IGMP packets for my pfSense and from the port mirroring of my switch :
pfsense capture.pcap miroring.pcapng

PS : on the mirroring capture you can see all VLANs (so you see 192.168.3.0 and 192.168.99.0 subnets)

JonathanLee

@maximushugus I have to look last time I was researching this it was related to gaming systems. I have to look it up again after I get back.

JonathanLee

Avahi Package!!

johnpoz

@JonathanLee avahi is for mdns only.

@maximushugus mentions IGMP.. And not working with update - this would indicate that he is running into the IP options that is new..

JonathanLee

@johnpoz it’s multicast dns right and others. I was reading about this a while ago.

johnpoz

@JonathanLee avahi doesn't do anything other than mdns, which is multicast sure, on a specific address and port 5353, it not going to do anything with some IPTV connection over multicast.

Pfsense has recently enabled IGMP filter for ip options. Even a rule would of allowed the traffic, the rule also has to allow for IP options or the traffic with IP options set will be blocked..

There has been multiple multiple threads about it.

https://forum.netgate.com/post/1167069

dennypage

@johnpoz said in multicast inconsistant:

Pfsense has recently enabled IGMP filter for ip options. Even a rule would of allowed the traffic, the rule also has to allow for IP options or the traffic with IP options set will be blocked..

I don't believe that this represents a change in packet behavior. I believe that previously IGMP packets were still blocked unless the IP option box was checked, even though there was no logging of it.

johnpoz

@dennypage That is quite possible.. I don't do anything with igmp, and I even block quite a bit of multicast at the switch level..

So I haven't looked into the exact details.. But I thought I saw a thread around here where it was stated it was allowed before - which was kind a security issue, and the new blocking without allowing ip options was a security enhancement.. I will see if I can dig up what I at least thought I read.

Maybe I misinterpreted this statement in the post I linked too?

"actually the correct behaviour but was broken in previous versions."

The broken in previous versions could be interpreted a few different ways, like before they were allowed or just which rule was listed as blocking them..

The take away I would say is, if you want to pass igmp, you should allow for IP options ;)

dennypage

@johnpoz I think this is probably the most clear statement I saw was from the redmine:

"Specifically, the code to check for options is this: https://cgit.freebsd.org/src/tree/sys/netpfil/pf/pf.c#n8312
It forces logging to be enabled if it drops a packet due to IP options. This used to be overwritten again so no logging happened, but that's been fixed in https://cgit.freebsd.org/src/commit/sys/netpfil/pf/pf.c?id=5f840a1758b4bbb4892118f43f40c6487c17aeba".

Of course, it's possible I misinterpreted the statement. I didn't dig through the kernel code.

maximushugus

Thanks for your answers.
But I still have this problem. As you can see, It looks as if pfSense is not seeing some multicast packets (see the difference between the packet capture from the switch and from pfSense).
Does someone know what is the problem and how to fix it ?

johnpoz

@maximushugus if your not seeing it via a sniff on pfsense, that has nothing to do with firewall rules on pfsense or blocking or not blocking igmp with ip options set.. Because the sniff would happen before any firewall rules were used.. Now up the stack could be blocked from "seeing" that traffic - but sniff at the wire would show it.

If pfsense is not seeing the traffic via sniff points to switch not sending it out the port pfsense is connected too.. Or your sniffing on a vlan and the traffic is not tagged, etc.

maximushugus

@johnpoz I agree with you but what is strange is that my problem happened after updating pfSense, and at this moment I didn't touch my switch, that's why I was searching for a pfSense problem

johnpoz

@maximushugus I hear ya - but I am not aware of anything in pfsense that would prevent a "sniff" ie packet capture from seeing the traffic coming into the interface.. If that traffic gets passed up the stack to be further processed either by something running on pfsense or routed, etc. sure... But at the sniff, if not seeing it on the sniff - that tells me its not there.

The only thing would be if your sniffing for traffic tagged for vlan X, but the traffic itself is not tagged.. You wouldn't see it then in your capture.

You know who might be good resource - @bmeeks he handles all the IPS stuff, fairly sure he would know if something could prevent being seen at the sniff level in pfsense.

I can't recall ever running into a scenario where not being seen by a packet capture, but traffic actually there - unless you not sniffing for the actual traffic, be it wrong vlan related or wrong interface, wrong protocol or wrong ip or port, etc... And I have been doing this a really long time ;)

stephenw10

Driver issue can hide traffic from pcaps. The ix driver had a bug that was filtering vlan0 for example. pcap showed no traffic.

johnpoz

@stephenw10 ah - good info.. But that is a error at the driver level.. Not some software in pfsense.. But guess there could be a problem after he updated with the driver.. Good info!!

dennypage

Of note is that the whole thing is a LAGG, and often packet captures do not function as one would expect. Several times in the past I have ended up having to run individual captures on the interfaces involved in the LAGG to get a complete picture.

If it were me, the first diagnostic thing I would do is to remove the LAGG from the picture. YMMV.

johnpoz

@dennypage also great info - and I have seen that in the past.. When troubleshooting where trying to validate traffic gets to where it is suppose to go and there is a lag at the endpoint, we have always turned down all but one interface in the lag

maximushugus

I restored a know working configuration from last year to my switch : it didn't solve the problem.

But by doing a packet capture on pfSense this time, after restoring my switch config, I have a weird behavior :

First after restoring (and restarting) my switch config, everything was working, I asked a multicast IPTV stream from my ISP on my PC and I was able to receive the stream.
But as soon as I stopped the stream, it stopped, but I was not able anymore to ask this multicast stream.

Here is the capture : as you can see, after packet n°10 with my PC leaving the multicast group, pfSense only see multicast from its own IP sourcecapture.pcap

bmeeks

@johnpoz said in multicast inconsistant:

You know who might be good resource - @bmeeks he handles all the IPS stuff, fairly sure he would know if something could prevent being seen at the sniff level in pfsense.

The only possible way for IPS to interfere with a traffic sniff would be when using Inline IPS Mode with the netmap kernel device. That could theoretically intercept and drop a packet before PCAP on the interface could see it, but I actually doubt that both PCAP and netmap can coexist simultaneously on the same physical interface. I suspect one of them is sure to complain about the other during initialization.

But simply stopping the IDS/IPS service is sufficient to completely remove it from any possible interference in the sniff.

maximushugus

By reseting the port link on my switch for my pfSense, I can reproduce this behavior