ipv6 broken: radvd: can't join ipv6-allrouters on <interface>

kiokoman

yeah i don't remember well but i think that the "address" in question was ff02::1 or something,
you can easily find out if you read

truss -p pidofradvd

The second setsockopt call error results in:

"can't join ipv6-allrouters on em0, failed: Too many references: can't splice(59)"

Might focus some attention on was has changed in:

sys/netinet6/in6_mcast.c

with the error codes EADDRNOTAVAIL and ETOOMANYREFS

kiokoman

my guess at that time was some kind of buffer overrun like it's opening sockets until there are so many that is unable to go on and it stop working and depending on the hardware it could happen after 4 / 8 / 24 hours

setsockopt(4,IPPROTO_IPV6,IPV6_LEAVE_GROUP,0x7fffffffd3e0,20) ERR#49 'Can't assign requested address'
setsockopt(4,IPPROTO_IPV6,IPV6_JOIN_GROUP,0x7fffffffd3e0,20) = 0 (0x0)

it is joining but not leaving everytime

this is from ip6 man page for freebsd:

IPV6_LEAVE_GROUP struct ipv6_mreq *
Drop membership from the associated multicast group. Memberships are automatically dropped when the socket is closed or when the process exits.

this is what made me think that restart radvd would temporary solve the problem
but i don't understand a damn about c/c++ or coding
but again that could be completely unrelated

w0w

I've just grabbed radvd 2.17 binary from earlier version of 2.5 and testing it on latest (2.5.0-DEVELOPMENT (amd64) built on Thu Sep 19 17:07:24 EDT 2019). At least no SPAM "IPv6 forwarding on interface seems to be disabled, but continuing anyway". Will wait another 48 hours.

A couple of thoughts:

The call to “setup_allrouters_membership” is being called several hundred times an hour. Should it be more selective when it asks to join a group rather than repeated leaving and joining? Think that is just asking for trouble we are seeing.
There are a number of upstream commits that have been applied to stable/12 since releng/12 in in6_mcast.c that suggest there are issues lurking there.

kiokoman

yes, indeed..
i'm sure it will be ok sooner or later, it's a development snapshot after all, we have our workaround in the meantime. they are aware of the problem and i'm using it at home so it's not a priority for me at the moment

Irata

@kiokoman the problem is within open source or kernel code, so who are "they" who will fix it?

w0w

If radvd 2.17_5 works well, then the issue is isolated to radvd or radvd patches, I think. Let me test it and we will know it soon, am I right/wrong.
The fact that there is no spam inspires me with hope.

kiokoman

sorry but if you check this
https://redmine.pfsense.org/issues/9577
you will see that we had 2.17 until 2019-07-22 with the same problem, it was updated to 2.18 at the end of july

@irata i don't think that netgate will release a product with a broken service, they will find a way, or they ask upstream or with a patch they made

w0w

@kiokoman
~~One thing that looks quite different for me is that radvd can't join routers right after start, not hours later as we have it now.~~

EDITED:
Oh yes, epic fail.

Irata

@kiokoman I agree, 2.5 will be left in beta until someone fixes it.

kiokoman

@rschell good work about that update on redmine
personally i had decided to wait for 2.5 to advance before trying to do anything as any patch that we come out today it will probably be lost resulting in a waste of time as it is a kernel problem and not a pfsense's fault

yon 0

Ipv6 problem also caused problems with FRR

ipv6 frr

w0w

@rschell
I see you have some progress on 12.1 builds, but you did not report back on redmine, is it working?

@w0w said in ipv6 broken: radvd: can't join ipv6-allrouters on <interface>:

@rschell
I see you have some progress on 12.1 builds, but you did not report back on redmine, is it working?

I have updated #9577, but for some reason the issue doesn't showup on the 2.5 issue list anymore. My version is running well and RADVD continues to work past 24 hours. The version is built on releng/12.1-RC2 with cherry-picked commits from 2.5 back to mid-February.

Once the pfSense is rebased on releng/12.1 or stable/12 and the message logging in RADVD is removed, these problems should be resolved.

@yon-0 said in ipv6 broken: radvd: can't join ipv6-allrouters on <interface>:

Ipv6 problem also caused problems with FRR

ipv6 frr

I haven't used FRR, so I can't speak to whether this will resolve those issues.

yon 0

@rschell

frr use with openvpn ipv6 has issue, add two ipv6 Neighbor openvpn will happen.

w0w

@jimp, any news? Did Netgate choose the FreeBSD version already? 12.1 is released a few days ago.

jimp

We are aware of the release (The FreeBSD Release Engineer is a Netgate employee :-)

12.1 is the current plan but there may be some parts picked back from stable. We'll see how things shake out.

w0w

@jimp Hmm, missed that reddit post, thanks a lot for the information!