23.01.b.20230106.0600 IGMP proxy stops TV stream
-
@stephenw10 Sorry for the noise in the thread, you were right about the checksum offloading.
I temporarily disabled IGMP snooping on the switch (to stop it filtering/handling the requests itself) and was able to observe the IGMP reports from the upstream side of IGMP proxy at another device arriving safely across the network with correct checksums. So checksums is not my issue so I never bothered to try turning off hardware checksuming.
However something really weird is going on - when the switch which is normally the IGMP querier (10.0.2.1) is the querier, it is somehow suppressing IGMP membership reports from the upstream interface of IGMP proxy on PFSense, (10.0.1.254) but when I make a different switch the querier (10.0.2.3 - the switch PFsense is connected to) suddenly IGMP proxy starts sending the upstream IGMP membership reports that it should be and it starts to work - at least partially, as I was able to find and play an SSDP advertised multicast stream between VLAN's.
A lot more debugging required on my part to figure out what's going on. One problem I have is a large mix of different switch models, with some of the older ones not necessarily playing nicely - I have a suspicion there is at least one other switch on the network which is not following the gentlemans rule of "don't be IGMP querier when a lower IP address device is already sending IGMP queries" as I have seen intermittent changes in the multicast router destination pointing towards these suspect downstream switches even though they are explicitly configured to never be an IGMP querier!
-
Unfortunately, I have the same problem here. Just updated from 22.05 to
23.01-RC (amd64)
built on Wed Feb 08 14:19:05 UTC 2023I can confirm that the IGMP proxy does not work and IPTV with this version unfortunately then also not.
Then the bug probably also applies to "CE 2.7.0", right?
-
-
I would expect it to be the same in 2.7 currently, yes.
-
Is threre any way the fix will make it into 23.01? That would be very appreciated.
https://redmine.pfsense.org/issues/13929
-
Not at this point, at least not in the release build. There is no fix yet as I understand it and the release is imminent. If it's something that can be patched at runtime it can be added to the system patches package later.
-
This is not good news...
Then I have to skip this update. :( -
-
I agree it's unfortunate but we couldn't hold the release for this. As soon as we pin down the issue we'll know more.
-
@stephenw10 is this issue affecting all IGMP proxy setups? Not just particullar driver or configuration. Combinations?
-
I haven't been able to replicate it here yet so I can't say for sure. However I would expect it to be something that affects all NICs/drivers. Other multicast traffic, like CARP, does not appear affected so it looks like something in igmpproxy or the libraries it uses.
Steve
-
I faced the same issue when upgraded to 23.01, Disable hardware checksum offload didnt help, bad checksum was in the packet capture
after reverting to BE 22.05 no bad checksum and ip tv works,
my interface is I226-V -
It would be nice if this issue was included in the release notes for 23.01. I read through all the documentation for 23.01 and then proceeded with the upgrade only to find that igmp-proxy did not work. Several hours of troubleshooting lead me to this forum post where it appears that the issue was known for some time before the release of 23.01. This is not the introduction to Netgate and pfSense that I wanted, having just migrated to Netgate products from Ubiquiti Unifi products specifically for the igmp-proxy functionality. Oh well, now I'll get an early introduction to the process for rolling back upgrades.
-
@jrueger said in 23.01.b.20230106.0600 IGMP proxy stops TV stream:
Oh well, now I'll get an early introduction to the process for rolling back upgrades.
Well that's not new to UniFi users right ;-) but yes I think the dev's underestimate the impact of this bug for home deployments. The list get longer and longer when users hit the update button.
At least it could be documented under know issues.
-
@thebear - no kidding with the Unifi gateways ... I finally abandoned Ubiquiti because of the opaqueness of their processes and capabilities. They did finally introduce igmp-proxy functionality in the latest release but with no documentation ... only a checkbox in the GUI and no explanation of just what exactly was enabled when checking the box. So far I am enjoying pfSense immensely so I can live with the igmp-proxy issue since I get to tinker more with things. My wife wasn't happy that her television show was interrupted!
-
Yes, I apologise, it should have been in the notes. The issue was undefined, and still is to some extent. Hopefully we can pin it down shortly.
Steve
-
@stephenw10
issue seems to be from upstream FreeBSD-14 , I was able reproduce checksum errors on FreeBSD-14.0-CURRENT-amd64-20230216-2894c8c96b9b-260969 while freeBSD 13.1-Release doesn't have the issue, pcap attached
13.pcap
14.pcap -
I think i found the issue updated bugtracker, I am attaching patch + binary , it is working for me now, if someone else can confirm that would be great
-
@mxt3rs said in 23.01.b.20230106.0600 IGMP proxy stops TV stream:
I think i found the issue updated bugtracker, I am attaching patch + binary , it is working for me now, if someone else can confirm that would be great
Sure happy to test, TV is currently in use so need to wait until tomorrow. File is loaded and ready for testing.
-
@mxt3rs very good work! IGMP if flowing and the TV stream also.
Every membership query is answered:
-
@thebear Thanks for the report! I've got the patch staged up and ready to bring into the tree. Until then, for anyone who wants a pkg that can be installed, I have bundled up the patched igmpproxy and attached it to the redmine issue:
https://redmine.pfsense.org/issues/13929#note-14
-
And then feeling brave and daring I went and managed to install the pkg on my 2100, not paying attention to the note that the pkg was for amd64 ... and now igmpproxy is borked and won't start (nice red x under Status - Services). Such is the life of a newbie to pfSense (if only I actually had some sense ).
-
You should be able to reinstall the default pkg using
pkg install
. You may need to force it.I'm surprised it allowed you to install that on aarch64 though.
-
I've managed to recover and am back to the original 23.01 version of igmp-proxy. Thanks for the help and now I'll be a good boy and wait until a version is available for aarch64.
-
@mxt3rs Installed 23.01 yesterday, had the same issue with igmpproxy. Patch works perfectly so far! Thx!
-
Upgraded on APU2. Fix works. Thank you
-
-
removed by author
-
removed by author
-
@thebear said in 23.01.b.20230106.0600 IGMP proxy stops TV stream:
It's available in the repo now:
Some packages are part of the base system and will not show up in Package Manager. If any such updates are listed below, run pkg upgrade from the shell to install them:
igmpproxy: 0.3_1 -> 0.3,1 [pfSense]
@stephenw10 Does the team released the fix to the repo's as shown above? Or is this the default pkg with the bug?
I run pkg upgrade and now the stream is dropping again. I'm new to FreeBSD.
tcpdump: listening on igc2, link-type EN10MB (Ethernet), capture size 262144 bytes 16:11:19.515840 IP (tos 0xc0, ttl 1, id 59170, offset 0, flags [none], proto IGMP (2), length 32, options (RA)) 172.16.27.1 > 224.0.0.1: igmp query v2 bad igmp cksum c16e! 16:11:21.703655 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 32, options (RA)) 172.16.27.10 > 224.0.252.129: igmp v2 report 224.0.252.129 16:11:27.763922 IP (tos 0xc0, ttl 1, id 17348, offset 0, flags [none], proto IGMP (2), length 32, options (RA)) 172.16.27.1 > 224.0.0.1: igmp query v2 bad igmp cksum c16e!
-
I don't see an updated pkg in the repos yet. 0.3,1 is the original pkg. I'll update here when it's available.
-
@stephenw10 tnx will edit my post about the upgrade cli, its not valid information then.
-
It will be.
You can see the build date in the pkg info:
[23.01-RELEASE][root@6100.stevew.lan]/root: pkg info igmpproxy igmpproxy-0.3,1 Name : igmpproxy Version : 0.3,1 Installed on : Sun Jan 8 16:49:53 2023 GMT Origin : net/igmpproxy Architecture : FreeBSD:14:amd64 Prefix : /usr/local Categories : net Licenses : GPLv2+ Maintainer : franco@opnsense.org WWW : https://github.com/pali/igmpproxy Comment : Multicast forwarding IGMP proxy Annotations : FreeBSD_version: 1400073 build_timestamp: 2023-01-07T20:09:23+0000 built_by : poudriere-git-3.3.99.20220831 port_checkout_unclean: no port_git_hash : c265aeb485a2 ports_top_checkout_unclean: yes ports_top_git_hash: c9915d224e16 repo_type : binary repository : pfSense Flat size : 45.6KiB Description : igmpproxy is a simple multicast routing daemon based on mrouted. It uses IGMP forwarding to dynamically route multicast traffic. Reqiures multicast forwarding enabled WWW: https://github.com/pali/igmpproxy
-
So in order to update this patch/fix for IGMP in 23.01 i should do the follow?
in Shell >
fetch https://redmine.pfsense.org/attachments/download/4714/igmpproxy-0.3_1.pkg
pkg add -f igmpproxy-0.3_1.pkgor is there another way to apply this fix?
-
Right now that's the best way.
-
@stephenw10 Will there also be a version for aarch64 in the repos? This would be for only the 2100 appliance (since I don't think 23.01 is available for the 1100). Maybe I should have splurged and purchased the 4100.
-
Yes, it should be available for all architectures. Including on the 1100.
-
Does that mean that it will be available as a System Patch in Patch Manager? Sorry for the dumb questions as I'm learning how to administer a pfsense box.
-
No it's not a runtime patch it requires a new binary so it will appear as an updatable pkg from the command line. That's not a usual way to upgrade/patch in pfSense but otherwise it would only be in the next release.
-
@stephenw10 said in 23.01.b.20230106.0600 IGMP proxy stops TV stream:
No it's not a runtime patch it requires a new binary so it will appear as an updatable pkg from the command line. That's not a usual way to upgrade/patch in pfSense but otherwise it would only be in the next release.
fetch https://redmine.pfsense.org/attachments/download/4714/igmpproxy-0.3_1.pkg pkg add -f igmpproxy-0.3_1.pkg
and after this a...
pkg install igmpproxy-0.3_1.pkg Y = for YES
And then all is done?
-
Yes, you only need to upgrade the pkg. Everything else will start using that.
-
@stephenw10 but this is still only for amd64?