Comcast IPv6 WAN address and delegated prefix added, then removed seconds later



  • The high level symptom is that IPv6 addresses always disappear from pfsense after a few days. Sometimes a reboot brings them back, sometimes not. I've been iterating through various options (don't wait for RA, never release, etc.) with no change in behavior.

    This is on a bare-metal installation of 2.4.4-RELEASE, and has been occuring ever since upgrading from 2.4.2-RELEASE.

    I have enabled debug mode for dhcp6d, and found that it does get the assignments from Comcast, however it removes them after a few seconds and subsequent solicitations go unanswered.

    dhcpd6c a reboot, addrs acquired:

    Nov 22 22:21:33	dhcp6c	45839	extracted an existing DUID from /var/db/dhcp6c_duid: 00:01:00:01:1d:8e:f5:9f:0c:c4:7a:69:e1:1c
    Nov 22 22:21:33	dhcp6c	45839	failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
    Nov 22 22:21:33	dhcp6c	45839	failed initialize control message authentication
    Nov 22 22:21:33	dhcp6c	45839	skip opening control port
    Nov 22 22:21:33	dhcp6c	45839	<3>[interface] (9)
    Nov 22 22:21:33	dhcp6c	45839	<5>[igb0] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[send] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>[ia-na] (5)
    Nov 22 22:21:33	dhcp6c	45839	<3>[0] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>comment [# request stateful address] (26)
    Nov 22 22:21:33	dhcp6c	45839	<3>[send] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>[ia-pd] (5)
    Nov 22 22:21:33	dhcp6c	45839	<3>[0] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>comment [# request prefix delegation] (27)
    Nov 22 22:21:33	dhcp6c	45839	<3>[request] (7)
    Nov 22 22:21:33	dhcp6c	45839	<3>[domain-name-servers] (19)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[request] (7)
    Nov 22 22:21:33	dhcp6c	45839	<3>[domain-name] (11)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[script] (6)
    Nov 22 22:21:33	dhcp6c	45839	<3>["/var/etc/dhcp6c_wan_script.sh"] (31)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>comment [# we'd like some nameservers please] (35)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[id-assoc] (8)
    Nov 22 22:21:33	dhcp6c	45839	<13>[na] (2)
    Nov 22 22:21:33	dhcp6c	45839	<13>[0] (1)
    Nov 22 22:21:33	dhcp6c	45839	<13>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[id-assoc] (8)
    Nov 22 22:21:33	dhcp6c	45839	<13>[pd] (2)
    Nov 22 22:21:33	dhcp6c	45839	<13>[0] (1)
    Nov 22 22:21:33	dhcp6c	45839	<13>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[prefix] (6)
    Nov 22 22:21:33	dhcp6c	45839	<3>[::] (2)
    Nov 22 22:21:33	dhcp6c	45839	<3>[/] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[60] (2)
    Nov 22 22:21:33	dhcp6c	45839	<3>[infinity] (8)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[prefix-interface] (16)
    Nov 22 22:21:33	dhcp6c	45839	<5>[igb1] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-id] (6)
    Nov 22 22:21:33	dhcp6c	45839	<3>[0] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-len] (7)
    Nov 22 22:21:33	dhcp6c	45839	<3>[4] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[prefix-interface] (16)
    Nov 22 22:21:33	dhcp6c	45839	<5>[igb2] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-id] (6)
    Nov 22 22:21:33	dhcp6c	45839	<3>[1] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-len] (7)
    Nov 22 22:21:33	dhcp6c	45839	<3>[4] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[prefix-interface] (16)
    Nov 22 22:21:33	dhcp6c	45839	<5>[igb3] (4)
    Nov 22 22:21:33	dhcp6c	45839	<3>begin of closure [{] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-id] (6)
    Nov 22 22:21:33	dhcp6c	45839	<3>[2] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>[sla-len] (7)
    Nov 22 22:21:33	dhcp6c	45839	<3>[4] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of closure [}] (1)
    Nov 22 22:21:33	dhcp6c	45839	<3>end of sentence [;] (1)
    Nov 22 22:21:33	dhcp6c	45839	called
    Nov 22 22:21:33	dhcp6c	45839	called
    Nov 22 22:21:33	dhcp6c	45960	reset a timer on igb0, state=INIT, timeo=0, retrans=891
    Nov 22 22:21:34	dhcp6c	45960	Sending Solicit
    Nov 22 22:21:34	dhcp6c	45960	a new XID (25af49) is generated
    Nov 22 22:21:34	dhcp6c	45960	set client ID (len 14)
    Nov 22 22:21:34	dhcp6c	45960	set identity association
    Nov 22 22:21:34	dhcp6c	45960	set elapsed time (len 2)
    Nov 22 22:21:34	dhcp6c	45960	set option request (len 4)
    Nov 22 22:21:34	dhcp6c	45960	set IA_PD prefix
    Nov 22 22:21:34	dhcp6c	45960	set IA_PD
    Nov 22 22:21:34	dhcp6c	45960	send solicit to ff02::1:2%igb0
    Nov 22 22:21:34	dhcp6c	45960	reset a timer on igb0, state=SOLICIT, timeo=0, retrans=1091
    Nov 22 22:21:34	dhcp6c	45960	receive advertise from fe80::201:5cff:fe87:fe46%igb0 on igb0
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option client ID, len 14
    Nov 22 22:21:34	dhcp6c	45960	DUID: 00:01:00:01:1d:8e:f5:9f:0c:c4:7a:69:e1:1c
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option server ID, len 14
    Nov 22 22:21:34	dhcp6c	45960	DUID: 00:01:00:01:1c:e4:c0:0d:40:a8:f0:2e:8f:18
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option identity association, len 40
    Nov 22 22:21:34	dhcp6c	45960	IA_NA: ID=0, T1=1800, T2=2880
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option IA address, len 24
    Nov 22 22:21:34	dhcp6c	45960	IA_NA address: 2001:558:6045:e0:75a0:3e23:d52b:9d41 pltime=3600 vltime=3600
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option IA_PD, len 41
    Nov 22 22:21:34	dhcp6c	45960	IA_PD: ID=0, T1=1800, T2=2880
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option IA_PD prefix, len 25
    Nov 22 22:21:34	dhcp6c	45960	IA_PD prefix: 2601:642:c400:7500::/60 pltime=3600 vltime=3600
    Nov 22 22:21:34	dhcp6c	45960	get DHCP option DNS, len 32
    Nov 22 22:21:34	dhcp6c	45960	server ID: 00:01:00:01:1c:e4:c0:0d:40:a8:f0:2e:8f:18, pref=-1
    Nov 22 22:21:34	dhcp6c	45960	reset timer for igb0 to 0.964693
    Nov 22 22:21:35	dhcp6c	45960	picked a server (ID: 00:01:00:01:1c:e4:c0:0d:40:a8:f0:2e:8f:18)
    Nov 22 22:21:35	dhcp6c	45960	Sending Request
    Nov 22 22:21:35	dhcp6c	45960	a new XID (2e0926) is generated
    Nov 22 22:21:35	dhcp6c	45960	set client ID (len 14)
    Nov 22 22:21:35	dhcp6c	45960	set server ID (len 14)
    Nov 22 22:21:35	dhcp6c	45960	set IA address
    Nov 22 22:21:35	dhcp6c	45960	set identity association
    Nov 22 22:21:35	dhcp6c	45960	set elapsed time (len 2)
    Nov 22 22:21:35	dhcp6c	45960	set option request (len 4)
    Nov 22 22:21:35	dhcp6c	45960	set IA_PD prefix
    Nov 22 22:21:35	dhcp6c	45960	set IA_PD
    Nov 22 22:21:35	dhcp6c	45960	send request to ff02::1:2%igb0
    Nov 22 22:21:35	dhcp6c	45960	reset a timer on igb0, state=REQUEST, timeo=0, retrans=909
    Nov 22 22:21:35	dhcp6c	45960	receive reply from fe80::201:5cff:fe87:fe46%igb0 on igb0
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option client ID, len 14
    Nov 22 22:21:35	dhcp6c	45960	DUID: 00:01:00:01:1d:8e:f5:9f:0c:c4:7a:69:e1:1c
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option server ID, len 14
    Nov 22 22:21:35	dhcp6c	45960	DUID: 00:01:00:01:1c:e4:c0:0d:40:a8:f0:2e:8f:18
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option identity association, len 40
    Nov 22 22:21:35	dhcp6c	45960	IA_NA: ID=0, T1=1800, T2=2880
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option IA address, len 24
    Nov 22 22:21:35	dhcp6c	45960	IA_NA address: 2001:558:6045:e0:75a0:3e23:d52b:9d41 pltime=3600 vltime=3600
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option IA_PD, len 41
    Nov 22 22:21:35	dhcp6c	45960	IA_PD: ID=0, T1=1800, T2=2880
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option IA_PD prefix, len 25
    Nov 22 22:21:35	dhcp6c	45960	IA_PD prefix: 2601:642:c400:7500::/60 pltime=3600 vltime=3600
    Nov 22 22:21:35	dhcp6c	45960	get DHCP option DNS, len 32
    Nov 22 22:21:35	dhcp6c	45960	dhcp6c Received REQUEST
    Nov 22 22:21:35	dhcp6c	45960	nameserver[0] 2001:558:feed::1
    Nov 22 22:21:35	dhcp6c	45960	nameserver[1] 2001:558:feed::2
    Nov 22 22:21:35	dhcp6c	45960	make an IA: PD-0
    Nov 22 22:21:35	dhcp6c	45960	create a prefix 2601:642:c400:7500::/60 pltime=3600, vltime=3600
    Nov 22 22:21:35	dhcp6c	45960	add an address 2601:642:c400:7500:ec4:7aff:fe69:e11d/64 on igb1
    Nov 22 22:21:35	dhcp6c	45960	add an address 2601:642:c400:7501:ec4:7aff:fe69:e11e/64 on igb2
    Nov 22 22:21:35	dhcp6c	45960	add an address 2601:642:c400:7502:ec4:7aff:fe69:e11f/64 on igb3
    Nov 22 22:21:35	dhcp6c	45960	make an IA: NA-0
    Nov 22 22:21:35	dhcp6c	45960	create an address 2001:558:6045:e0:75a0:3e23:d52b:9d41 pltime=3600, vltime=15498334245746642448
    Nov 22 22:21:35	dhcp6c	45960	add an address 2001:558:6045:e0:75a0:3e23:d52b:9d41/128 on igb0
    Nov 22 22:21:35	dhcp6c	45960	executes /var/etc/dhcp6c_wan_script.sh
    

    Seconds later, addrs removed:

    Nov 22 22:21:43	dhcp6c		dhcp6c REQUEST on igb0 - running rc.newwanipv6
    Nov 22 22:21:43	dhcp6c	45960	script "/var/etc/dhcp6c_wan_script.sh" terminated
    Nov 22 22:21:43	dhcp6c	45960	removing an event on igb0, state=REQUEST
    Nov 22 22:21:43	dhcp6c	45960	removing server (ID: 00:01:00:01:1c:e4:c0:0d:40:a8:f0:2e:8f:18)
    Nov 22 22:21:43	dhcp6c	45960	got an expected reply, sleeping.
    Nov 22 22:22:11	dhcp6c	45960	Bypassing address release because of -n flag
    Nov 22 22:22:11	dhcp6c	45960	remove an IA: NA-0
    Nov 22 22:22:11	dhcp6c	45960	remove an address 2001:558:6045:e0:75a0:3e23:d52b:9d41
    Nov 22 22:22:11	dhcp6c	45960	remove an address 2001:558:6045:e0:75a0:3e23:d52b:9d41/128 on igb0
    Nov 22 22:22:11	dhcp6c	45960	reset a timer on igb0, state=INIT, timeo=0, retrans=118
    Nov 22 22:22:11	dhcp6c	45960	Bypassing address release because of -n flag
    Nov 22 22:22:11	dhcp6c	45960	remove an IA: PD-0
    Nov 22 22:22:11	dhcp6c	45960	remove a site prefix 2601:642:c400:7500::/60
    Nov 22 22:22:11	dhcp6c	45960	remove an address 2601:642:c400:7500:ec4:7aff:fe69:e11d/64 on igb1
    Nov 22 22:22:11	dhcp6c	45960	remove an address 2601:642:c400:7501:ec4:7aff:fe69:e11e/64 on igb2
    Nov 22 22:22:11	dhcp6c	45960	remove an address 2601:642:c400:7502:ec4:7aff:fe69:e11f/64 on igb3
    Nov 22 22:22:11	dhcp6c	45960	reset a timer on igb0, state=INIT, timeo=0, retrans=256
    Nov 22 22:22:11	dhcp6c	45960	removing an event on igb0, state=INIT
    Nov 22 22:22:11	dhcp6c	45960	removing an event on igb0, state=INIT
    Nov 22 22:22:11	dhcp6c	45960	executes /var/etc/dhcp6c_wan_script.sh
    Nov 22 22:22:11	dhcp6c		dhcp6c EXIT or RELEASE on igb0 running rc.newwanipv6
    Nov 22 22:22:11	dhcp6c	45960	script "/var/etc/dhcp6c_wan_script.sh" terminated
    Nov 22 22:22:11	dhcp6c	45960	exiting
    

    Steady-state behavior, solicitations sent every 10 seconds, indefinitely:

    Nov 22 23:37:42	dhcp6c	37443	Sending Solicit
    Nov 22 23:37:42	dhcp6c	37443	set client ID (len 14)
    Nov 22 23:37:42	dhcp6c	37443	set identity association
    Nov 22 23:37:42	dhcp6c	37443	set elapsed time (len 2)
    Nov 22 23:37:42	dhcp6c	37443	set option request (len 4)
    Nov 22 23:37:42	dhcp6c	37443	set IA_PD prefix
    Nov 22 23:37:42	dhcp6c	37443	set IA_PD
    Nov 22 23:37:42	dhcp6c	37443	send solicit to ff02::1:2%igb0
    Nov 22 23:37:42	dhcp6c	37443	reset a timer on igb0, state=SOLICIT, timeo=44, retrans=113136
    Nov 22 23:39:35	dhcp6c	37443	Sending Solicit
    Nov 22 23:39:35	dhcp6c	37443	set client ID (len 14)
    Nov 22 23:39:35	dhcp6c	37443	set identity association
    Nov 22 23:39:35	dhcp6c	37443	set elapsed time (len 2)
    Nov 22 23:39:35	dhcp6c	37443	set option request (len 4)
    Nov 22 23:39:35	dhcp6c	37443	set IA_PD prefix
    Nov 22 23:39:35	dhcp6c	37443	set IA_PD
    Nov 22 23:39:35	dhcp6c	37443	send solicit to ff02::1:2%igb0
    Nov 22 23:39:35	dhcp6c	37443	reset a timer on igb0, state=SOLICIT, timeo=45, retrans=123288
    

    Any input would be appreciated.



  • I had similar symptoms. I wanted to expand from single /64 to /60 with comcast. When I switch delegation size in WAN config I would see IPv6 addresses assigned to the multiple tacked LANs then disappear. Sometimes it would only allocate single /64. Rebooting the pfsense did not resolve the issue. Once I rebooted my comcast modem (zoom modem, not rented) it finally gave me the /60 I was looking for and stuck with it. Also check the DHCP6 DUID option in System/Advanced/Networking. Tweaking that value may trick it into getting a new delegated prefix.



  • I'm not having an issue getting a /60 allocated, that works fine, pfsense just removes it about 10 seconds after it receives it.

    Further investigation indicates this is related to pfsense starting multiple instances of dhcp6c. I have no idea why this is happening, since I have only one WAN interface, but the longer pfsense runs, the more instances start. When I first found this, 8 instances of dhcp6c were running, after about 1 week of uptime. Every day or so now I loose IPv6 connectivity, and check and find that 2 or 3 instances are running, kill them, restart the wan interface and have IPv6 working again. IPv6 connectivity was stable for months at a time with 2.4.2, so I'm not sure what introduced this behavior.


  • Netgate

    Please try 2.4.4-p1