how to stop logging blocked LAN IGMP?
-
@johnpoz said in how to stop logging blocked LAN IGMP?:
@dennypage said in how to stop logging blocked LAN IGMP?:
You will see it only when a host turns off a multicast subscription.
Why is his doing it every few seconds?
A good question. There are a couple of things that can be at play here.
The first is that IGMP requires all packets to be sent multiple times with a short interval in-between. This is how IGMP deals with lost packets. The number of times is the "robustness" value and defaults to 2, but implementations are allowed to choose any value of 2 or above.
The other is that some systems simply frequently join and leave groups on a frequent basis. Apple devices do this frequently on 224.0.0.251 (mDNS). I don't know if this is an attempt to force mDNS rediscovery (doubtful), service restarts, or some other thing I don't know of. Pretty inefficient in my book, but it only happens with mDNS --they don't do it on other addresses.
-
@Mission-Ghost said in how to stop logging blocked LAN IGMP?:
Curiously, the switch that the Roku box (and the router-on-a-stick) plug into reports no Multicast packets. The Roku box is on Port 4.
Is pfSense hallucinating?
FWIW, I would trust pfSense a lot more than I would trust multicast accounting of consumer grade switches.
-
Good morning!
I had a short service window this morning. The logs citing a nonexistent rule had continued overnight.
So I rebooted and the log entry attribution switched to my policy routing rule on the Ent vlan, which was not set to log. So I toggled the log checkbox on, saved, applied, and then repeated turning it off. No change…still log entries kept coming from it for igmp. Rebooted again. No change.
I then enabled my igmp rule at the top of the Ent vlan to pass igmp packets without logging and I got one pass log entry for that rule and the block log entries attributed to my policy routing rule resumed. So I copied the policy routing rules to a new one and deleted the one being attributed in the logs as blocking when not set to block nor log. The logs continued and were still being attributed to the now-deleted rule.
I rebooted again but the problem continued. So I then shut off udpbroadcastrelay but the logs continued. I rebooted again and the logs stopped. I then turned udpbroadcastrelay back on and the logs have remained stopped. I’m leaving my igmp pass rule in place and not changing its logging option (currently off) for now. I don’t know if the pass rule is now working or something else happened with the last reboot. Nothing obvious is apparent.
Note I have not made any changes to the roku box to which the mystery logs attribute the igmp packet. I have never seen a roku option related to igmp. Recall this all started when i was setting up the broadcast tv reciever/streamer (the HDHomeRun box) on that vlan.
In this case I don’t have D-Link access points. I have netgear. The roku and hdhomerun box though are ethernet, not wireless.
Clearly there is something screwy and buggy going on with rules, logging and igmp. I have no idea how to reproduce the problem and can’t easily experiment because it’s a production system. I’ll try the suggested packet capture later when I get a chance.
However, we should not overlook that the logs have been coming from nonexistent end-user rules or rules set to pass and not log, and rules set to capture these packets get substantially ignored.
-
@dennypage Thank you for the thoughtful response.
I ran (I have an 4200 so my interface was igc0):
tcpdump -i igc0 -w /tmp/igmp.pcap igmpfor about a full minute, and this is all I got:
I also ran it with igc0.50 for a couple of minutes to try to account for the VLAN and the results were:
Is this meaningful?
-
Can you open it in wireshark?
-
@Mission-Ghost It's a binary file. Can you post or PM it please?
-
@stephenw10 I'm getting Wireshark on my Linux box and will attempt to open the tcpdump file in due course. Thanks!
-
PMs with igmp.pcap sent with my thanks.
This afternoon I removed the two disabled IGMP pass rules, which I had set up to try to stop this logging, from pfSense and the mysterious logging still has not returned.
This seems to indicate the disabled (and even when enabled) pass rules were having no discernible effect on the logging of IGMP packets from a Roku box on the Ent network attributed to, at varying times, a non-existent or previously deleted rule, or, this morning after a reboot, the policy routing rule (!) of all things, which also was configured to not log.
This may be one of those things that is very difficult to catch and fix given it will probably happen on a production system that cannot be experimented with easily and given I don't know what triggered this to begin with. However, the three reboots seem to have cleared it for now, much like the OP discovered cleared his issue as well.
-
@Mission-Ghost I took a look at your pcap. Your Netgear (192.168.10.22) switch or WAP is operating as an IGMP querier, soliciting IGMP messages from hosts in the network. Nothing particularly unusual about that assuming you have the Netgear configured for IGMP snooping. I was curious to line this up with some of your error messages and went back to look at detail information in your prior logs.
Then, the sky fell. The Netgear has an address of 192.168.10.22. Looking at your prior posts, packets from address 192.168.10.22 are arriving on at least 4 different pfSense interfaces:
- 30_UTIL
- 40_CAM
- 50_ENT
- 60_GUEST_GM
Obviously, that should not happen, and is indicative of a significant issue. Either you have multicast packet flooding, or the VLANs are not configured/functioning as expected.
I recommend setting aside IGMP logging issues for a moment, and going back to validate the basic configuration of switch/WAP VLANs and pfSense interfaces. It's a good idea to start a new forum topic for that, with a link to this topic in your initial post.
-
Yikes, nice catch.
-
@dennypage Wow! Thank you for making the time and effort to examine the capture file. This is an interesting revelation.
I noticed the Netgear records in the capture file but did not recognize the meaning and importance of them. It does makes sense that the packets would arrive on all the interfaces: the Netgears are on trunk ports and host all six VLANs.
The Netgears are WAX610 WAPs. I'll go through them (there are four) and shut off or verify multicast functions are shut off.
Thanks again. Networking is so much more complex now than when I did it at work (Thick- and Thin-net!) 30 years ago. It's almost not the same thing at all anymore.
-
Yeah great catch @dennypage I knew something was off when was seeing that much multicast traffic from a roku - I see zero.. But thought maybe when he mentioned the hdhomerun device - was like maybe it does the multicast, I have played with them a tiny bit.. I had set one up for my son but he was on a flat network and it just worked.
Not really a fan of multicast - sure it has some great uses.. Hey if you want to send a video/audio stream to multiple devices at the same time - then yeah it rocks! But most of it is just noise, mdns, ssdp, llnmr - etc.. pure and utter crap noise that I want nowhere near my network. I have zero use for those protocols, and do everything I can to keep the noise at a min.. One thing I hate is opening up a pcap and seeing just noise, and more noise ;)
But great digging into details there - who was the guy that recently said we have to much free time <hahaha> clearly you have some - and fantastic that you help others and dig into a problem like that!! Impressive!
-
@Mission-Ghost said in how to stop logging blocked LAN IGMP?:
It does makes sense that the packets would arrive on all the interfaces: the Netgears are on trunk ports and host all six VLANs.
It does not make sense that packets arrive on multiple interfaces from a single IP address. The packets should come from a VLAN specific IP address in each VLAN.
-
@dennypage Would it make more sense given each of six SSIDs has an Enable/Disable Multicast/Broadcast setting and associated other settings on these APs, and each SSID goes through their own VLAN each of which has an interface on pfSense?
-
The "blocked" logging from 50.2 returned this morning without any pfSense changes:
Rule *3276 is my Entertainment VLAN pass rule for policy routing:
I have a IGMP pass rule at the top with Allow IP Options enabled and logging disabled:
-
@Mission-Ghost you may be encountering issues simply because the source IP address is not valid for the subnet defined on the interface. Fix the basic VLAN issues first.
-
@dennypage said in how to stop logging blocked LAN IGMP?:
@Mission-Ghost you may be encountering issues simply because the source IP address is not valid for the subnet defined on the interface. Fix the basic VLAN issues first.
Perhaps. Something I forgot to mention is that .10.1 is the pfSense router/firewall (which is also .50.1 on the .50 Entertainment VLAN), so while it showed up in the pcap it is the router, so while maybe it should have shown as .50.1, it's the same node as .10.1.
I spent time today shutting off all of the multicast functionality in the four access points. It turns out the HDHomeRun device needs multicast to function even on and within the same VLAN (.50 in this case) so I had to turn it all back on with the exception of IGMP Snooping, which was not required. Doing so enabled HDHomeRun to work properly within the .50 VLAN, which is sufficient. An IGMP pass rule is required in the .50 VLAN as well. I got occasional blocked log from non-existent or policy routing rules but they've stopped again for no apparent reason.
Thank you again for your help along with @johnpoz and @stephenw10. I appreciate the insights.
Setting all of this stuff above aside, what hasn't been acknowledged is that pfSense is, in this case, posting log messages, except when they're attributed to the policy routing rule, that can't be attributed to a rule the administrator can locate or do anything about and can't be stopped using documented procedures.
I think this is reasonable to call a bug and an investigation into why it could happen and making a fix would be worthwhile at improving the stability and trustworthiness of the product.
-
Spoke too soon. The mystery log messages have started up again while I was doing another pcap against igc0.50:
[.50.3 is the other Roku box (hard wired....50.2 is the WiFi Roku box)].
Rule 1761573276 is the policy routing rule for the .50 VLAN, which is set to not log and is below the IGMP pass rule. I modified the IGMP pass rule (with the Allow IP Options enabled) to match on the log source and destination addresses specifically, the log still comes out from the policy routing rule.
This just seems buggy. I'll have to reboot again when I get an maintenance hour and see if that stops it again.
-
@Mission-Ghost said in how to stop logging blocked LAN IGMP?:
Setting all of this stuff above aside, what hasn't been acknowledged is that pfSense is, in this case, posting log messages, except when they're attributed to the policy routing rule, that can't be attributed to a rule the administrator can locate or do anything about and can't be stopped using documented procedures.
I think this is reasonable to call a bug and an investigation into why it could happen and making a fix would be worthwhile at improving the stability and trustworthiness of the product.
It's not my call, but in my opinion it's a bit premature to waive the "there's a bug" flag. You have packets arriving on the wrong interface, with source addresses are outside the local address range of that subnet. By default, this alone will trigger logging.
Have a look at the checkbox options on the options Status / System Logs / Firewall (wrench icon). One of those options may affect what you are seeing. Also read here. @stephenw10 may have additional suggestions.
This really should be broken off into a new thread. Be sure to include information about your interface configurations and rules.
-
@dennypage Thanks again for the helpful discussion.
All of the check boxes for logging bogons etc. under Status/System Logs/Firewall > Manage Firewall Log are unchecked, and unchanged from prior to this issue (I have not set the system to do Firewall logging which is why this issue is noticeable. The firewall log should be empty on my system all the time.)
I re-read the section you pointed me to (I have read it before--but I did not have anything logging prior to this mystery so it was unneeded before). So to be scientific about it, I put a block-all-no-logging at the end of the .50 VLAN rule set, to no effect. First I put a block no log rule to 192.165.50.255. When that didn't work, I changed it to the one shown below to block all no-logging to the IGMP broadcast address. The mystery logs continue to be posted from the prior policy routing rule that is being attributed to by the log records.
So I moved this block-no-log rule above the policy routing rule, to no effect.
Still getting log records attributed to the policy routing rule.
To be thorough and consistent with the documentation example you referenced, I put a block-no-log rule on the Starlink WAN rule set to no effect (I set it to log, first, then to no log, neither one made a difference as inbound Starlink traffic has not been logging anyway):
Because overkill is underrated, I also then copied the rule to the TMHI backup-internet interface, to no effect.
Of course, the problem isn't incoming from the Internet, but I want to demonstrate that I have no firewall logging enabled anywhere I can think of.
This issue is frustrating because I've done nothing to deliberately start this logging and have apparently no direct control over stopping it.
A couple of days ago I rebooted three times and the log entries stopped on the third reboot, until they restarted today, both for no apparent reason.
(Aside, I disagree this should be a separate thread; it's consistent with the O.P.'s problem and unresolved.)
If this isn't a bug, it's not clear why it's a feature!
