IPv6 DHCP-PD – radvd dies after interface reset - dhcpv6 does not reaquire addr

priller

At 48 hours, the renewal time, the IPv6 LAN address disappears, RADVD vanishes from the services page, and resolv.conf picks up the IPv6 DNS from Comast (it only had the 75.75.x addresses prior to the renewal).

pfSense will only bind the LAN IPv6 prefix at boot time. Once the lease expires it will never rebind. It just keeps trying to infinity. I'd say this is definitely a Show-Stopper bug.

rebind.jpg_thumb

johnpoz

Yup seems I lost my IPs as well…

So I have updated to

2.1-BETA1 (i386)
built on Fri Apr 12 04:55:08 EDT 2013
FreeBSD 8.3-RELEASE-p7

Still not working, so turned off ipv6 on the wan - then back on. Got my 2001 on my wan, then a bit later new 2601 on my lan - but different subnet. Gawd that sucks - if the /64 handed to me is going to change every time, I will just go back to the tunnel from HE. This never changed.

Also noticed that delete virtual IPs is still not working as well. Since the virtual I put on my lan with ::1 as the address vs the mac containing ipv6 it get would not delete. Using IE gives a warning about deleting, you say yes never deletes - firefox no such warning, but might because I popups blocked. But does not delete. So once got new address on lan, I change my virtual to be in that /64 and now I have ipv6 access again. But its a pain, lets see if goes away in 48 hours.

And I looked in the config - only place that address was listed was


	 <virtualip><vip><mode>ipalias</mode>
			<interface>lan</interface>
			 <descr><type>single</type>
			<subnet_bits>64</subnet_bits>
			<subnet>2601:d:xxxx:d7::1</subnet></descr></vip></virtualip>

priller

@johnpoz:

Still not working, so turned off ipv6 on the wan - then back on. Got my 2001 on my wan, then a bit later new 2601 on my lan - but different subnet. Gawd that sucks - if the /64 handed to me is going to change every time, I will just go back to the tunnel from HE. This never changed.

That's what I've ended up doing. HE is solid.

After studying my packet captures, I see two related issues here.

When it's time to renew, pfs sends a REBIND. The response from the server is a renewal of the same prefix. That's good, unfortunately that is not processed by pfs … that's bad. The result is that the LAN IPv6 address is purged and never returns.
When pfs does an initial SOLICIT. You get a response from two Comcast DHCP servers, each with a different prefix. One is the what you had before (as long as you are within the lease window) the other is a new one. If pfs remembered it's last prefix, it should just rebind on a matching response. However, it does not. The end result is that your LAN prefix can keep changing each time you reset/reboot. Again, that is in the bad category.

shavenne

Any fix available yet? ???
My ISP (Deutsche Telekom) gives me an IPv6 address (dual stack) since I switched to IPTV. It comes via DHCP6, but it's not static (it changes everytime I reboot or go offline+online).
LAN is set to Track Interface.
When I reboot, WAN is getting the IPv6 IP, LAN is also getting another IPv6 IP, but radvd doesn't seem to work. When I go to Services and restart radvd my clients are also getting IPv6 IPs and it works perfect. But after about 10 or 20 minutes radvd kills itself and disappeares in the Services menu and the LAN interface doesn't have it's IPv6 address anymore.

Apr 25 19:18:10 radvd[58188]: version 1.9.1 started
Apr 25 19:18:41 radvd[58287]: attempting to reread config file
Apr 25 19:18:41 radvd[58287]: resuming normal operation
Apr 25 19:29:22 radvd[58287]: sendmsg: Can't assign requested address
Apr 25 19:29:23 radvd[58287]: attempting to reread config file
Apr 25 19:29:23 radvd[58287]: syntax error in /var/etc/radvd.conf, line 44:
Apr 25 19:29:23 radvd[58287]: error parsing or activating the config file: /var/etc/radvd.conf
Apr 25 19:44:31 radvd[30802]: version 1.9.1 started
Apr 25 19:44:31 radvd[30802]: syntax error in /var/etc/radvd.conf, line 2:
Apr 25 19:44:31 radvd[30802]: error parsing or activating the config file: /var/etc/radvd.conf
Apr 25 19:44:31 radvd[30802]: Exiting, failed to read config file.

Any suggestions? :-[
It's unusable for me at the moment.

mokuso

The reason the /64 allocation changes is that the /var/db/dhcp6c_duid file does not persist on reboot. Each reboot this file is recreated and a new DUID is assigned.

It would be nice to have the option to retain this file.

MaxPF

Any updates on this one?

johnpoz

I still don't see it working.. Im on

2.1-BETA1 (i386)
built on Tue May 21 20:50:09 EDT 2013
FreeBSD 8.3-RELEASE-p8

And when I updated to it the other day I tried to get it working again.. It look like it took it for a second, I saw an IPv6 in inf status, then next thing it was gone.

Now I know they just added the RC tag - so I might update tonight again, but something is still not right with this.

priller

Here's the bug for the IPv6 addressing being lost: http://redmine.pfsense.org/issues/2919

I would definitely consider this a Show Stopper bug. The ability to acquire and maintain addressing is a basic function, without that nothing else matters. I was surprised to see the change to RC status, with issues like this remaining.

MaxPF

@priller:

Here's the bug for the IPv6 addressing being lost: http://redmine.pfsense.org/issues/2919

I would definitely consider this a Show Stopper bug. The ability to acquire and maintain addressing is a basic function, without that nothing else matters. I was surprised to see the change to RC status, with issues like this remaining.

That was exactly what I thought. When I read about the RC status I went back looking for this hoping that I missed the update. Considering that the main feature of 2.1 is IPV6 support, this is pretty major, together with the broken 6RD support.

emue

Yes, this really is a critical issue. DHCPv6-PD looks to become one of the most common, if not the most common native IPv6 deployment method for home users and small businesses.

Here in Germany for example the largest ISP (Deutsche Telekom) has silently started its IPv6 rollout for end users. Millions of connections will be migrated to dual stack using DHCPv6-PD over PPPoE in the next few years.

I don't think pfsense can really claim to support IPv6 until this is working properly.

BrianPlencner

I want to echo what others have said about the broken state of IPv6 in the 2.1 beta's of pfSense.

For reference, this is the version that I am currently running: 2.1-BETA1 Built On: Thu May 9 07:05:02 EDT 2013.

When I first installed that version, I was able to fully get IPv6 to work with my ISP, which is Comcast. It worked fine for 4 days, and then just stopped working.

After 4 days, which I believe is the dhcp server's lease period for IPv6, I noticed the following things

"radvd" service is now MIA from the list.
WAN interface no longer shows anything to do with IPv6 (except the IPv6 Link Local and Gateway IPv6)
IPv6 Test sites fail with 0/10
Comcast's IPv6 Speedtest site fails to load

If I shutdown my pfSense box, as well as my modem, and then power both back up, IPv6 will then work for another 4 days, and then stop working again.

I would strongly encourage those who work on pfSense to get this resolved before moving from Beta to RC status on this project. If this is not fixed, I envision a lot of unhappy customers.

If there is anything I can to do help resolve this issue, please let me know. I can post logs, screen shots, or whatever other information may be needed by the developers at pfSense that may be helpful to resolve this.

–Brian

wallabybob

@BrianPlencner:

If there is anything I can to do help resolve this issue, please let me know. I can post logs, screen shots, or whatever other information may be needed by the developers at pfSense that may be helpful to resolve this.

Please post any reports from the system log (displayed by pfSense shell command```
clog /var/log/system.log


Any reports from radvd (output from pfSense shell command```
clog /var/log/system.log | grep radvd
```) might also be useful.

priller

There is a wealth of information, logs and other, included in this thread and attached to the bug.

If one of the devs wants to PM me, I can provide the full packet capture that is shown in my post here:
http://forum.pfsense.org/index.php/topic,59996.msg329002.html#msg329002

The bug notes also further details the exact behavior that is show in the packet capture. He it is again:

The WAN DHCP IPv6 address and LAN DHCP-PD lease time from the provider is 4 days. When the lease expires, pfSense goes through the following sequence, but never rebinds the IPv6 addressing. The result is WAN and LAN IPV6 addresses is lost and never recovers.

Initially, a DHCPv6 SOLICIT is sent. The DHCPv6 server replies with the expected addressing to be renewed.
Five seconds later, a DHCPv6 REBIND is sent. There is never a RENEW, it immediately goes to REBIND. Again, the DHCPv6 server replies with the expected addressing to be renewed.
Now pfSense keeps sending DHCPv6 SOLICITs. The DHCPv6 server is responding, but pfSense never binds.

This just continues until pfSense is rebooted.

–-

I was initially wondering if this problem was isolated to Comcast users. However, users on other providers are now reporting the same behavior.

FWIW, On the Comcast forums, m0n0wall users report no problems. Maybe somebody could ask their old buddies how they have implemented DHCP-PD client.

johnpoz

"FWIW, On the Comcast forums, m0n0wall users report no problems."

Might have to fireup m0n0wall and test that..

edit: Look at that, bing bang zoom - couple clicks and ipv6 is working ;)

Guess could always switch over to m0n0wall ;)

edit2: Well noticed one thing wrong with it as well, upon reboot getting a different lan ipv6 /64 every time.. This pretty much makes using any sort of static setup impossible. If every time your router reboots you get assigned a different /64

lookatthat.jpg_thumb

priller

As for the LAN prefix changing every time, that is because pfSense (and apparently m0n0wall) does not remember the last assigned prefix.

Whenever you reboot, you have a 50/50 chance of getting the same prefix or an entirely new one. That is because you get responses from two Comcast DHCP servers. One is giving you the last prefix you had, the other is giving an entirely new one from its pool.

Nothing wrong with that, it's pfs fault for not remembering what it had been assigned.

Somebody else had said that it was because pfs generates a new DUID on each reboot. Maybe that could happen, but that's not what I have seen on packet captures.

johnpoz

looks like m0n0wall allows you to put in your duid, so maybe could make sure get same /64 that way?

Wonder if could just block getting ip range from other server, so always get the same one.. So can not maintain the same /64 - currently if I do get one on pfsense, it is gone in a few days. Seems like I should prob go back to the tunnel - it worked!! It was stable! And always had same /64, and could get a /48 if need be so I could easy put different /64s on all my segments easy, etc.

I am BrianPlencner ipv6 I thought was the star of the 2.1 release - shouldn't making sure everything about it as stable as possible be the highest priority?

emue

Another issue I have which is preventing IPv6 from being useful:

Take a look at these log entries (IPv4 address censored):

May 27 19:09:49 router php: : rc.newwanipv6: Informational is starting pppoe0.
May 27 19:09:54 router php: : rc.newwanipv6: on (IP address: 2003:74:cc7f:821d:20d:b9ff:fe29:9698) (interface: wan) (real interface: pppoe0).
May 27 19:09:55 router UNKNOWN[16501]: Process 44678 died: No such process; trying to remove PID file. (/var/run/radvd.pid)
May 27 19:09:55 router php: : ROUTING: setting default route to 217.0.X.X
May 27 19:09:55 router php: : ROUTING: setting IPv6 default route to fe80::230:88ff:fe04:1410%pppoe0
May 27 19:09:56 router check_reload_status: Reloading filter
May 27 19:10:17 router check_reload_status: updating dyndns WAN_PPPOE,WAN_DHCP6
May 27 19:10:17 router check_reload_status: Restarting ipsec tunnels
May 27 19:10:17 router check_reload_status: Restarting OpenVPN tunnels/interfaces
May 27 19:10:17 router check_reload_status: Reloading filter
May 27 19:24:49 router php: : rc.newwanipv6: Informational is starting pppoe0.
May 27 19:24:54 router php: : rc.newwanipv6: on (IP address: 2003:74:cc7f:821d:20d:b9ff:fe29:9698) (interface: wan) (real interface: pppoe0).
May 27 19:24:55 router UNKNOWN[16501]: attempting to reread config file
May 27 19:24:55 router UNKNOWN[16501]: resuming normal operation
May 27 19:24:55 router php: : ROUTING: setting default route to 217.0.X.X
May 27 19:24:55 router php: : ROUTING: setting IPv6 default route to fe80::230:88ff:fe04:1410%pppoe0
May 27 19:24:56 router check_reload_status: Reloading filter
May 27 19:25:17 router check_reload_status: updating dyndns WAN_PPPOE,WAN_DHCP6
May 27 19:25:17 router check_reload_status: Restarting ipsec tunnels
May 27 19:25:17 router check_reload_status: Restarting OpenVPN tunnels/interfaces
May 27 19:25:17 router check_reload_status: Reloading filter

Note the repetition. It seems that exactly every 15 minutes (probably some sort of renewal/rebinding interval set by the provider) pfsense goes through some sort of restart process. This apparently includes resetting the firewall, dropping the state table in the process. In practice this means that all TCP connections time out, including IPv4.
Why is this happening? I can see no direct connection with the lease time, which is certainly greater than 15 minutes; neither the assigned prefix nor the WAN address actually change. Even if they did, there would be no reason at all to shutdown IPv4 traffic. As it is I have to completely disable IPv6 interface tracking to make the connection usable with even IPv4.

I have also noticed that radvd doesn't start directly upon configuring the interfaces but rather at the end of one of these 15 minute intervals, causing quite a delay before IPv6 becomes active (not sure whether this is in any way my provider's fault). The "no such process" error message in the log seems to be relevant to this.

razzfazz

@johnpoz:

edit: Look at that, bing bang zoom - couple clicks and ipv6 is working ;)

Is this with the stable version or with the development snapshot?

johnpoz

that was with 1.34, stable version.

priller

OK, count me in. I loaded m0n0wall (1.8.1b540) on a new VM. Just for the curiosity of seeing it survive the Comcast Day-4 DHCPv6 lease expiration problem that pfSense has.

–-

I like seeing that m0n0wall has the ability to statically configure a DUID. That can certainly be handy. I looked into the the pfSense config XML and see that there is a DUID parameter in there, but it has not been implemented in the GUI. Wonder why?