DHCPv6 Client Broken in latest snapshot

[bimmerdriver]

I upgraded to the latest snapshot, built on Thu Apr 26 14:32:50 CDT 2018. The DHCPv6 client is stuck pending. I am using "Do not wait for a RA".

Apr 26 22:33:19	dhcp6c	10972	transmit failed: Input/output error
Apr 26 22:33:19	dhcp6c	10972	Sending Solicit
Apr 26 22:31:20	dhcp6c	10972	transmit failed: Input/output error
Apr 26 22:31:20	dhcp6c	10972	Sending Solicit
Apr 26 22:30:15	dhcp6c	10972	transmit failed: Input/output error
Apr 26 22:30:15	dhcp6c	10972	Sending Solicit
Apr 26 22:29:43	dhcp6c	10972	transmit failed: Input/output error
Apr 26 22:29:43	dhcp6c	10972	Sending Solicit

[bimmerdriver]

Some additional log messages:

Apr 27 07:12:09	dhcp6c	88806	transmit failed: Input/output error
Apr 27 07:12:09	dhcp6c	88806	Sending Solicit
Apr 27 07:12:08	dhcp6c	88806	transmit failed: Input/output error
Apr 27 07:12:08	dhcp6c	88806	Sending Solicit
Apr 27 07:12:07	dhcp6c	88719	skip opening control port
Apr 27 07:12:07	dhcp6c	88719	failed initialize control message authentication
Apr 27 07:12:07	dhcp6c	88719	failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory

[bimmerdriver]

Is no one else experiencing this problem? My system has had this issue since the update from when I originally posted.

[behemyth]

I have no had the issue - and i use IPv6 quite a bit. I'm using SLAAC from my provider though.

[bimmerdriver]

I reverted to 2.4.3 release using a clean install. It worked fine. I attempted to upgrade to 2.4.4 dev. The first time, it failed with a message system update failed. The dashboard was showing the upgrade was available, so I tried again. There were quite a few console errors, including errors described in https://forum.pfsense.org/index.php?topic=147415.0. To whomever took my working pfsense 2.4.4 dev, I want it back now, please.

[bimmerdriver]

I was going to download an earlier snapshot, but the earliest one is from April 30, after the problem started. Are earlier snapshots available?

[bimmerdriver]

I downloaded the latest snapshot and performed a clean installation. The problem is still present. I'm kind of blown away that no one else is experiencing this.

Here are what seem to be the indicative messages from the DHCP log:

May 21 14:51:51	dhcp6c	49073	transmit failed: Input/output error
May 21 14:51:51	dhcp6c	49073	Sending Solicit

[LostInIgnorance]

Is there any way you can provide more details on this? Who is your provider/equipment? The NIC being used? Unfortunately my provider is really behind the times and I don't have IPv6 available….thanks Chart..I mean Spectrum.

[bimmerdriver]

I'm using Telus. I have three "connections". One is directly using the router inside the modem. It's working fine. There are two others connections, which are both pfsense using the bridged port on the modem. One of these systems is running pfsense 2.4.3. It's working fine. The other is running the latest development snapshot. On this system, ipv4 is working, but ipv6 is not working, as I described above. It used to work, but some time in April it stopped working. I can only presume a change was made to pfsense that broke the ipv6 or dhcpv6 client. I am using the "do not wait for RA flag", because it's required for my connection. It's possible a change was made to pfsense that affected this flag, because without it, it's not possible to connect to Telus. The only abnormal log messages are above.

[bimmerdriver]

I'm really at a loss as to what's happening here. I'll restate the configuration and the situation. I have no idea what to look for in the logs.

My ISP is Telus. I have a bonded VDSL. There are three ipv6 prefixes. The prefix is a /56 and in order to get a prefix allocated, the do not wait for RA setting is required. I was one of the original users that tested this feature and it has been reliable for months. One prefix is for the modem, which is only used for IPTV. The other two prefixes are via a bridged port which is connected to a virtual switch on a hyper-v server using a dedicated NIC. There are two vms connected to the virtual switch. One is running pfsense 2.4.3. It has been perfectly stable for months. The other is running the latest pfsense development snapshot. It was stable for months, but in April, ipv6 stopped working after an upgrade. I tried reverting to 2.4.3 and it started working again. When I went back to the snapshot, it stopped working. I created a completely new vm and installed 2.4.3. It worked. When I updated to the development version, it stopped working. I tried reinstalling the development version from scratch on a new vm and it did not work. The only obvious log messages are posted above. I used wireshark to capture ipv6 packets from the system but there are none.

If anyone has a suggestion of where I can look or what I can do, I would appreciate it. If I can't get this working, I will move off pfsense to another system.

Are old shapshots archived somewhere? If so, I could try reinstalling them until I find the exact date where the problem started.

[bimmerdriver]

Does anyone know if "transmit failed: Input/output error" is a pfsense error or a BSD error? Is there any way to get more information about the cause? It's pretty useless without knowing what caused it.

[bimmerdriver]

I noticed that 2.4.4 development is running a different version of freebsd. Does anyone know when the change was introduced? I'm suspicious that it's related.

[w0w]

I can not find any trace when Netgate moved from FreeBSD 11.1 to 11.2 beta, but it have happened in May, I think. There are also a bunch of PHP7 related changes happened in the same period of time, but I don't see any related to DHCPv6, so your problem can be related to base system only.

[bimmerdriver]

@w0w Martin (who did a bunch of work on interfaces.inc, but since left for other pastures), took a look at interfaces.inc and didn't see any changes, so the change of freebsd is a smoking gun.

[darkfire]

I have the exact same problem with my DHCPv6 Client since the last update to 2.4.3-RELEASE-p1.

[jimp]

@bimmerdriver said in DHCPv6 Client Broken in latest snapshot:

Does anyone know if "transmit failed: Input/output error" is a pfsense error or a BSD error? Is there any way to get more information about the cause? It's pretty useless without knowing what caused it.

That would be from the dhcp6c client

[jimp]

@darkfire said in DHCPv6 Client Broken in latest snapshot:

I have the exact same problem with my DHCPv6 Client since the last update to 2.4.3-RELEASE-p1.

Are you also using "Do not wait for RA"?

[darkfire]

Yes, i do. Without “Do not wait for RA” the dhcp6c client will not start anymore.

/var/etc/dhcp6c_wan.conf

interface em0 {
	send ia-pd 0;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc pd { };

logs with “Do not wait for RA”

May 29 23:02:14 	dhcp6c 	53193 	failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
May 29 23:02:14 	dhcp6c 	53193 	failed initialize control message authentication
May 29 23:02:14 	dhcp6c 	53193 	skip opening control port
May 29 23:02:15 	dhcp6c 	53211 	Sending Solicit
May 29 23:02:15 	dhcp6c 	53211 	transmit failed: Can't assign requested address
May 29 23:02:16 	dhcp6c 	53211 	Sending Solicit
May 29 23:02:16 	dhcp6c 	53211 	transmit failed: Can't assign requested address 

process list

ps aux | grep dhcp6c
root   53211   0.0  0.1   8380  2364  -  Is   23:02       0:00.00 /usr/local/sbin/dhcp6c -d -c /var/etc/dhcp6c_wan.conf -p /var/run/dhcp6c_em0.pid em0

[jimp]

@darkfire said in DHCPv6 Client Broken in latest snapshot:

Without “Do not wait for RA” the dhcp6c client will not start anymore.

So it fails without that enabled? Or it fails both ways but with different errors?

[bimmerdriver]

My system behaves differently. If I disable do not wait for ra, dhcp6c never runs, because it's waiting for an RA, which will not be sent. Also, the error message is different.