ipv6 broken: radvd: can't join ipv6-allrouters on <interface>

[Irata]

I'm not sure how to find what's common.

I'm running pfsense within VMware ESXi and have this issue. Are others with this issue the same?

I can also confirm the latest 2.4 branch does not have this issue with the same config.

[strangegopher]

I am running it on baremetal dell box but I am using chelsio 10G sfp+ nic.

[strangegopher]

@jimp sorry for the ping but do u have any idea why ipv6 stops working?

[motific]

How is it on a fresh 2.5 install with the old config?

I’ve just had some bad behaviour with radvd not responding at all (not quite the same as your issue) and that has cleared it.

[kiokoman]

i have similar behavior on my log

Jun 6 21:32:29 	radvd 	66014 	resuming normal operation
Jun 6 21:32:29 	radvd 	66014 	attempting to reread config file
Jun 6 18:57:34 	radvd 	66014 	resuming normal operation
Jun 6 18:57:34 	radvd 	66014 	attempting to reread config file
Jun 6 14:01:49 	radvd 	65728 	version 2.17 started 
Jun 6 14:00:39 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 14:00:39 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0
Jun 6 14:00:33 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 14:00:22 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 14:00:20 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0
Jun 6 14:00:08 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 14:00:01 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0
Jun 6 13:59:53 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0
Jun 6 13:59:53 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 13:59:40 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0
Jun 6 13:59:36 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 13:59:30 	radvd 	67952 	can't join ipv6-allrouters on igb2
Jun 6 13:59:25 	radvd 	67952 	can't join ipv6-allrouters on ath0_wlan0

igb0 is wan
igb1 is lan
on igb2 i have only 1 pc, a server connected with static ipv6
on ath0_wlan0 i have a couple of notebook with windows 10 and a couple of android phone

[Irata]

A fresh install causes the same error for me, once stateless IPv6 is setup. I can't find a stable working system whatever I do, other than back to 2.4.

Because I'm using an ESXi testbed, it's easy to spin up new installs.

I do think there is something fundamental here, given the reports from wide ranging setups.

[kiokoman]

the error is generated from device-bsd44.c, a patch it's inside the port of radvd for freebsd
tracking back there was similar problem. related Bug #2878 https://redmine.pfsense.org/issues/2878 from 2013

/* XXX: See pfSense ticket #2878 */
+	setsockopt(sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP, &mreq, sizeof(mreq));
+
+	if (setsockopt(sock, IPPROTO_IPV6, IPV6_JOIN_GROUP,
+			&mreq, sizeof(mreq)) < 0) {
+		flog(LOG_ERR, "can't join ipv6-allrouters on %s", iface->props.name);
+		return (-1);
+ }

maybe it's time for a new ticket
https://redmine.pfsense.org/issues/9577

[strangegopher]

idk if we open a ticket devs might just ignore it, first we should get devs on the forums to tell us if we need to open a bug.

[motific]

@strangegopher - they're aware of it, one of the devs referred to the ticket in a thread with an issue I'm having.

[maverick_slo]

Now it hit me too:

Jun 29 07:17:29 radvd 62926 can't join ipv6-allrouters on hn0.10
Jun 29 07:15:22 radvd 62926 can't join ipv6-allrouters on hn0.10
Jun 29 07:15:00 radvd 62926 can't join ipv6-allrouters on hn0.9
Jun 29 07:13:07 radvd 62926 can't join ipv6-allrouters on hn0.7
Jun 29 07:12:47 radvd 62926 can't join ipv6-allrouters on hn0.10
Jun 29 07:11:25 radvd 62926 can't join ipv6-allrouters on hn0.8
Jun 29 07:11:23 radvd 62926 can't join ipv6-allrouters on hn0.9
Jun 29 07:10:22 radvd 62926 can't join ipv6-allrouters on hn0.10
Jun 29 07:08:10 radvd 62926 can't join ipv6-allrouters on hn0.10

[kiokoman]

welcome to the party !

[Irata]

@maverick_slo made any changes recently, or added any new IPv6 devices?

[maverick_slo]

No changes no new devices.
I just came home from vacation...

[kiokoman]

i put here a workaround that seems to work for me

[2.5.0-DEVELOPMENT][root@pfSense.localdomain]/root: crontab -l
@hourly killall radvd && sleep 5 && /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog

[Irata]

On the recent snapshot (2.5.0.a.20190728.0536) I now also get the following repeated in the routing system log:

Jul 30 16:50:49	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:42	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:41	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:34	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:29	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:24	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway
Jul 30 16:50:21	radvd	72128	IPv6 forwarding on interface seems to be disabled, but continuing anyway

[maverick_slo]

Yup same here, 2.5.0 is getting worse by every snapshot
Ah well probably Ill go back to 2.4.4...

[Irata]

Looks like it is failing within this code in device-linux.c in radvd:

int check_ip6_forwarding(void)
{
	int value;
	FILE *fp = NULL;

	fp = fopen(PROC_SYS_IP6_FORWARDING, "r");
.
.
.

Which I think is checking for /proc/sys./... but there is no /proc/sys/... directory. There is also alternative code called in device-bsd44.c. So I'm wondering if radvd in 2.5.0 is being built for the wrong platform, or it's expecting a file that is now removed. But I admit this is a wild guess.

[strangegopher]

Netgate can't possibly ship 2.5.0 with this bug that essentially disables ipv6 support in pfsense. Still no dev replies or acknowledgment of Bug #9577. Hope people that are paying for netgate products or support ask for this to be fixed.

[kiokoman]

Actually it is working for me, it's only a message log spamming out, they have time to correct it before the official release

[Irata]

It's "working" here too with a radvd restart every 6 hours. I do a radvd service restart via Cron instead of a kill, which seemed cleaner but my approach causes IPv6 sessions to be dropped when the prefix is unadvertised briefly.

Looking at the source code, that spam logging should not even be executing on a freebsd platform. So maybe it's a hint of what's broken.