IGMP proxy no longer works reliably after 2.7.1 update

mistergefahrensucher

thanks for testing.
At the moment i have another workaround (somthing from the past as IGMPProxy also not worked)
I configured a servicewathdog. Last 2 Days have seen 2 restarts in the morningtime.
This worked for me so far.
But indeed there must be somthing wrong the combination new Kernel and Proxy.
If you said 0.31 ist also not working. i noticed 0.31 is the former version working with Kernel 14 from PF+ 23.05.
So the Kernelnumber is matching. Look like someting is wrong with checksum and lost IGMP memberships. And when you heavy switch betreen 2 chanels there is a memory leak or what else. (i'm not an expert on that level)
Someting has changed in the new 14 Kernel.

If i find the time i will also tryout 0.31. and 0.4.1

or i will wait for a solution

mistergefahrensucher

@Rai80 said in IGMP proxy no longer works reliably after 2.7.1 update:

pkg update

gives error:

pkg: Error extracting the archive: 'Write error'
pkg: No signature found
Unable to update repository FreeBSD

what i'm doing wrong?

vjizzle

I just saw the new releases for CE and pf+ but sadly no igmp proxy fix in the release notes.

mistergefahrensucher

@vjizzle

yes but maybe a kernel fix ?
I think igmpproxy is not the problem because the same version igmpproxy is working fine with older 14er Kernel.
i see this:
Installed packages to be UPGRADED:
pfSense: 23.09 -> 23.09.1 [pfSense]
pfSense-base: 23.09 -> 23.09.1 [pfSense-core]
pfSense-boot: 23.09 -> 23.09.1 [pfSense-core]
pfSense-default-config: 23.09 -> 23.09.1 [pfSense]
pfSense-kernel-pfSense: 23.09 -> 23.09.1 [pfSense-core]
pfSense-repo: 23.09 -> 23.09.1 [pfSense]

ninin06

Has anyone already installed 2.7.2?
Does the problem still exist?
Or are there new problems?

Rai80

@ninin06 Just did the upgrade and tested. Unfortunately the issue doesn't seem te be resolved:

mistergefahrensucher

@Rai80
Yes this is because Kernel version and igmp version is the same as in 2.7.2
Kernel 1400094

long time ago as far as i remember pfsense 2.3... or 2.4 the same disaster with igmpproxy and the workaround was very crazy.
copy back an older version back. (but will give it a try again)

cheers

btw. i'm looking forwrd to a change to pimd. But ther is no understandble information for human to configure it right in a simple IPTV enviroment.
My knowledge and time to experiment is to less.

jimp

Kristof requested some information from users hitting this bug. See https://redmine.pfsense.org/issues/15043 for the full details. If anyone hitting this can check into what he's asking for it would help. Here is the text describing what to gather:

MRT_DEL_MFC; Errno(49) is interesting. error 49 is EADDRNOTAVAIL, which can only be returned (for MRT_DEL_MFC at least) if the membership igmpproxy attempted to remove does not exist.
So as far as the kernel is concerned we're not joined to that group.
(Do note that the membership lookup takes both origin and group address into account, so it's possible that the group is correct but the membership address is not for example.)

We're going to need a bit more information to track this down.
Let's start with a debug log, because igmpproxy can log the relevant information before it attempts MRT_DEL_MFC. That does require it to run with debug verbosity, which can't be set through the webui. Terminate the running process and restart it as /usr/local/sbin/igmpproxy -v -v /var/etc/igmpproxy.conf (or /usr/local/sbin/igmpproxy -d -v -v /var/etc/igmpproxy.conf to run it in the foreground and send all output to stdout).

Once the error occurs (and igmpproxy is still running!) also export the kernel's view of the multicast state with netstat -i -a -n and netstat -g.

Remie2000

@jimp said in IGMP proxy no longer works reliably after 2.7.1 update:

Kristof requested some information from users hitting this bug. See https://redmine.pfsense.org/issues/15043 for the full details. If anyone hitting this can check into what he's asking for it would help. Here is the text describing what to gather:

MRT_DEL_MFC; Errno(49) is interesting. error 49 is EADDRNOTAVAIL, which can only be returned (for MRT_DEL_MFC at least) if the membership igmpproxy attempted to remove does not exist.
So as far as the kernel is concerned we're not joined to that group.
(Do note that the membership lookup takes both origin and group address into account, so it's possible that the group is correct but the membership address is not for example.)

We're going to need a bit more information to track this down.
Let's start with a debug log, because igmpproxy can log the relevant information before it attempts MRT_DEL_MFC. That does require it to run with debug verbosity, which can't be set through the webui. Terminate the running process and restart it as /usr/local/sbin/igmpproxy -v -v /var/etc/igmpproxy.conf (or /usr/local/sbin/igmpproxy -d -v -v /var/etc/igmpproxy.conf to run it in the foreground and send all output to stdout).

Once the error occurs (and igmpproxy is still running!) also export the kernel's view of the multicast state with netstat -i -a -n and netstat -g.

Shucks I just missed that while I was downgrading...

I've downgraded to 23.05.1 for the weekend again. So my family can enjoy internet + TV again.
If somebody else can provide the logs that is still affected then it would be great.

If nobody else can find time by the end of next week I can arrange some time (when the kids are at school and my wife at work ;-) ) by recreating the situation again.

Thanks a lot @jimp for mentioning. I will also post this update on another forum where we discuss this issue.

Remie2000

@Remie2000 said in IGMP proxy no longer works reliably after 2.7.1 update:

@jimp said in IGMP proxy no longer works reliably after 2.7.1 update:

Kristof requested some information from users hitting this bug. See https://redmine.pfsense.org/issues/15043 for the full details. If anyone hitting this can check into what he's asking for it would help. Here is the text describing what to gather:

MRT_DEL_MFC; Errno(49) is interesting. error 49 is EADDRNOTAVAIL, which can only be returned (for MRT_DEL_MFC at least) if the membership igmpproxy attempted to remove does not exist.
So as far as the kernel is concerned we're not joined to that group.
(Do note that the membership lookup takes both origin and group address into account, so it's possible that the group is correct but the membership address is not for example.)

We're going to need a bit more information to track this down.
Let's start with a debug log, because igmpproxy can log the relevant information before it attempts MRT_DEL_MFC. That does require it to run with debug verbosity, which can't be set through the webui. Terminate the running process and restart it as /usr/local/sbin/igmpproxy -v -v /var/etc/igmpproxy.conf (or /usr/local/sbin/igmpproxy -d -v -v /var/etc/igmpproxy.conf to run it in the foreground and send all output to stdout).

Once the error occurs (and igmpproxy is still running!) also export the kernel's view of the multicast state with netstat -i -a -n and netstat -g.

Shucks I just missed that while I was downgrading...

I've downgraded to 23.05.1 for the weekend again. So my family can enjoy internet + TV again.
If somebody else can provide the logs that is still affected then it would be great.

If nobody else can find time by the end of next week I can arrange some time (when the kids are at school and my wife at work ;-) ) by recreating the situation again.

Thanks a lot @jimp for mentioning. I will also post this update on another forum where we discuss this issue.

I'm going to try some quick testing, time is limited don't know if I can produce any results.

Remie2000

@Remie2000 said in IGMP proxy no longer works reliably after 2.7.1 update:

@Remie2000 said in IGMP proxy no longer works reliably after 2.7.1 update:

@jimp said in IGMP proxy no longer works reliably after 2.7.1 update:

Kristof requested some information from users hitting this bug. See https://redmine.pfsense.org/issues/15043 for the full details. If anyone hitting this can check into what he's asking for it would help. Here is the text describing what to gather:

MRT_DEL_MFC; Errno(49) is interesting. error 49 is EADDRNOTAVAIL, which can only be returned (for MRT_DEL_MFC at least) if the membership igmpproxy attempted to remove does not exist.
So as far as the kernel is concerned we're not joined to that group.
(Do note that the membership lookup takes both origin and group address into account, so it's possible that the group is correct but the membership address is not for example.)

We're going to need a bit more information to track this down.
Let's start with a debug log, because igmpproxy can log the relevant information before it attempts MRT_DEL_MFC. That does require it to run with debug verbosity, which can't be set through the webui. Terminate the running process and restart it as /usr/local/sbin/igmpproxy -v -v /var/etc/igmpproxy.conf (or /usr/local/sbin/igmpproxy -d -v -v /var/etc/igmpproxy.conf to run it in the foreground and send all output to stdout).

Once the error occurs (and igmpproxy is still running!) also export the kernel's view of the multicast state with netstat -i -a -n and netstat -g.

Shucks I just missed that while I was downgrading...

I've downgraded to 23.05.1 for the weekend again. So my family can enjoy internet + TV again.
If somebody else can provide the logs that is still affected then it would be great.

If nobody else can find time by the end of next week I can arrange some time (when the kids are at school and my wife at work ;-) ) by recreating the situation again.

Thanks a lot @jimp for mentioning. I will also post this update on another forum where we discuss this issue.

I'm going to try some quick testing, time is limited don't know if I can produce any results.

I've supplied the results, downgrading again before my family returns home :-)

Cornel

@Remie2000 thank you very much for taking the time!

haraldinho

This post is deleted!

haraldinho

@vjizzle said in IGMP proxy no longer works reliably after 2.7.1 update:

Release 24.03…wow. So the fix is targeted to be here somewhere in the next 6 months orso?

Is it possible to run the igmp proxy package from 2.6 on 2.7? That would be a workaround perhaps for the time being.

@vjizzle This is typically how it is done. A separate fix is created and made available asap, usually within days or weeks. You can install the fix through the Patch option in your pfSense System-->Patches. Then, in the next big release the fix is incorporated in the main release and the patch can be dropped. Depending a bit also on where the fix should be made, not everything is under control of the pfSense team.

This was at least my experience around IGMPproxy issues, there have been some in the past. Everybody keeps saying that IGMPproxy is just a simple service passing packets along, however given the amount of issues I guess it is either more complex than everybody thinks or a good set of regression tests are missing.

vjizzle

@haraldinho
Yes, that is also how I expect things to work, however setting a delivery date that far in the future is not how you manage expectations realistically. Nonetheless, for now, running fine on 2.6.

Rai80

With the latest kernel patch from Kristof Provost this issue is solved!

See: https://redmine.pfsense.org/issues/15043

Cornel

@Rai80 great news!!
I get a file not found error on opening that next cloud link. Anyway assuming that link is restored. How do you install that image?

Rai80

@Cornel Just download the file [pfSense-kernel-pfSense-2.7.2.r.20231212.1754.pkg] to your pfSense box. When the link is working again.
Install it with: pkg add -f pfSense-kernel-pfSense-2.7.2.r.20231212.1754.pkg
Reboot

And done!

haraldinho

@Rai80 not sure what exactly the difference is, but Kristof suggested the following to install:

“Backup your device, download the pkg file to it, "pkg install -U <patchname>.pkg" and reboot.”

Rai80

Since the former link is not working anymore I uploaded the patched kernel: https://file.io/f2Xmr5QlCTnF