Comcast dropping prefix delegation?
-
Hey all,
For the longest time I've had a stable /60 from Comcast, but lately the delegation seems to silently drop. I couldn't find anything useful in dhcp6c's logs. The delegation just becomes unusable (as if routing was dropped). If I HUP dhcp6c, I occasionally get the same prefix and the routing again just works. Sometimes it drops entirely and I get a new /60. I tried talking to Comcast, but couldn't get past the multiple tiers of people arguing with me on what IPv6 was.
Anyone else been experiencing this or have a good way to debug it? I've seen the behavior on both 2.2.6 and 2.3-RC1. 99% sure the problem is entirely on Comcast's side.
Thanks!
-
Just curious if your modem happens to be rebooting when you experience the issue. It could be that they're doing maintenance in your area (they're still upgrading CMTS units around the country in preparation for DOCSIS 3.1).
-
I'm having the exact same problem -except I don't think my IPv6 has ever been stable… Everything works fine for a day or so and then just quietly quits "as if routing was dropped" -exactly. If I do a release/renew on my WAN I get a new prefix and everything works fine again for a day or so... then silently quits. I also called Comcast... I guess you know how that went.
My only other thought was maybe my modem was dropping the ball somewhere? I'm not totally up to speed on how the routing prefixes get passed around or updated from time to time but something is amiss somewhere. I'm running 2.3_1 here.
I don't think the modem is rebooting. IPv4 works without interruption.
-
traceroute out via IPv6, what does that look like when it isn't working? Then also try a traceroute in via IPv6, when it's working and when it's not.
-
@cmb:
traceroute out via IPv6, what does that look like when it isn't working? Then also try a traceroute in via IPv6, when it's working and when it's not.
When it breaks, I can't get beyond pfSense. After HUPing dhcp6c, a new traceroute successfully completes. (Most of the time on the same /60.)
-
@JC:
When it breaks, I can't get beyond pfSense. After HUPing dhcp6c, a new traceroute successfully completes. (Most of the time on the same /60.)
That doesn't necessarily narrow it down, though what's probably happening is Comcast dropped your PD route and the renewal brings it back. If you traceroute in from the Internet to your LAN IPv6 IP when it's working and when it's not, that'll be more telling. Can do so at https://lg.he.net/ if you don't have an outside server somewhere to use.
-
Same here. Except I always get a new prefix with each renew. I traced it backward and forward and it traces everything except the last hop. It's the one step beyond pfsense that's the problem.
Out of curiosity, what modem do you use JC Denton?
-
Out of curiosity, what modem do you use JC Denton?
I have a Motorola SB6141.
cmb, the traceroute ends one hop before pfSense (i.e., my IPv6 upstream gateway) when tracing externally. Do a renew and it's back to normal.
-
And it barfed again shortly within an hour or two. Renewed and got a different /60. Very frustrating.
I think my current workaround will be to have cron HUP dhcp6c every few hours until I can find a Comcast rep that can speak v6.
-
"until I can find a Comcast rep that can speak v6"
I gave up on that ;) I just run a tunnel with HE until comcast is ready for primetime on their ipv6.. Rock solid… One thing for sure I always have the same /48 never have to worry about that changing..
-
I'm pretty sure that Comcast's IPv6 is done and ready, considering they gave a talk last month in France at a Cisco symposeum about moving to IPv6-only (don't worry, there's still at least a few years before that's a reality; though "IPv4 as a service" could be sooner).
I can honestly say that I've had very minimal issues with pfSense and Comcast's IPv6. Usually they appear when the modem reboots, then comes back online (like during maintenance)… sometimes IPv6 comes back, sometimes it doesn't, sometimes it does but not completely. But I've had pfSense uptime stretches of over a month with no IPv6 issues.
-
Maybe in your area it is.. But it for sure not primetime in all their areas… Shoot try calling and asking for help with, good luck talking to anyone with a clue.. So great that they tell the world they are ready, and not saying they are not ahead of the game compared to other isps.
So their ipv6 is 100% and ready, just like they tell the world their customer service is great... <rofl>I wouldn't trust everything that comes out of the corp PR mouth ;)
So they plan on going ipv6 only internally... Which is interesting, but they sure an the hell can not just give their customers ipv6 only..</rofl>
-
Maybe in your area it is.. But it for sure not primetime in all their areas… Shoot try calling and asking for help with, good luck talking to anyone with a clue.. So great that they tell the world they are ready, and not saying they are not ahead of the game compared to other isps.
So their ipv6 is 100% and ready, just like they tell the world their customer service is great... <rofl>I wouldn't trust everything that comes out of the corp PR mouth ;)
So they plan on going ipv6 only internally... Which is interesting, but they sure an the hell can not just give their customers ipv6 only..</rofl>
Maybe in my area… which is still running an old CMTS and only has 8 downstream channels, while there are other areas that are upgraded and ready for DOCSIS 3.1 and have 24 or more channels. My modem supports 16, so that's how I know I only have 8 available.
As for phone support... I don't expect them to know a THING about IPv6. I will only ever call as a last resort, and so far in over a year and a half I've only needed to call twice... once was to confirm an outage with their automatic system (the outage was fixed less than an hour later), the second was to activate a modem that wouldn't activate through their "walled garden".
-
@JC:
And it barfed again shortly within an hour or two. Renewed and got a different /60. Very frustrating.
Ugly. Definitely sounds like they have something screwed up there. The one thing beyond what you've tried that I'd do is run a ping to your IPv6 LAN IP from the Internet, and packet capture on WAN filtered on ICMPv6. The traceroute makes it sound like it's definitely not getting to you, but that'll 100% confirm or deny that. Assuming you likely won't see the ICMPv6 on WAN, that means they did drop your PD route. That's probably the likely scenario since everything else you've noted matches the reports of about a handful of other people who are having that problem.
-
@cmb:
@JC:
And it barfed again shortly within an hour or two. Renewed and got a different /60. Very frustrating.
Ugly. Definitely sounds like they have something screwed up there. The one thing beyond what you've tried that I'd do is run a ping to your IPv6 LAN IP from the Internet, and packet capture on WAN filtered on ICMPv6. The traceroute makes it sound like it's definitely not getting to you, but that'll 100% confirm or deny that. Assuming you likely won't see the ICMPv6 on WAN, that means they did drop your PD route. That's probably the likely scenario since everything else you've noted matches the reports of about a handful of other people who are having that problem.
Yep, no packets. I don't suppose anyone here would have an intelligent contact at Comcast? FWIW, I tried again and they offered to roll a truck to connect my "version 6 server" :-X
In the meantime, I've brought my Tunnelbroker tunnel back up for reliable v6.
-
@JC:
Yep, no packets. I don't suppose anyone here would have an intelligent contact at Comcast? FWIW, I tried again and they offered to roll a truck to connect my "version 6 server" :-X
Man, they really need to train their support people on IPv6…
But your mention of an intelligent contact at Comcast made me remember, John Brzozowski himself has an account on this site. You'd be hard-pressed to find a more intelligent IPv6 contact anywhere in the world.
I don't want to inundate him with people's support issues, but this is definitely a problem that's impacting multiple Comcast customers and it's pretty clearly something in their network. It's not that many relative to how many people use pfSense with Comcast IPv6, maybe a handful that I've heard from recently. But it might be something they're unaware of, or are working on and could use a technically competent customer to help. I emailed John with a link to this thread, maybe he'll be able to chip in.
-
Hello all,
I have posted here before in the past. I run the IPv6 program at Comcast. Quick clarifying question, at some point those who had a /60 are now getting a /64 for PD? Or no PD at all?
John
-
Quick clarifying question, at some point those who had a /60 are now getting a /64 for PD? Or no PD at all?
Hi John,
I get a /60 again if I force a renew. There's about a 20-25% chance I get the /60 I was using back.
I know there's no guarantee I'll have the same PD for any length of time. What's getting me though, is the PD's route seems to drop silently (and prematurely) on the Comcast side. So IPv6 traffic on my side grinds to a halt until I manually intervene or dhcp6c goes out and attempts a renew when its timer expires. I've been able to replicate this using two different routers in two vastly different Comcast service locations (two different states).
Happy to help troubleshoot this any way I can. Just let me know what you need :)
Thanks!
-
OK this helps. When we perform moves and splits for capacity addresses (for IPv4 and IPv6) and prefixes for IPv6 change unless of course you are a commercial static customer. Commercial static customer IP blocks (IPv4 and IPv6) are static across moves and splits. NOTE - commercial static has NOT yet launched, targeting this summer.
One thing that we did find is that after a mode or a split RENEWs for /60s are responded to with a /64. I think this is what folks are seeing. This is a bug in the DHCP server that I believe has been fixed. I need to check on the rollout of the same.
If the router in use does not deactivate the /60 or parts of the same this will properly result in traffic for the same not being routed. Remember the router was just moved from one CMTS to another. The new /64 should be properly routed. I understand this does not to the crux of the issue a /60 is needed/wanted.
Once you have the /64, the best way to get a /60 back is to change the DUID of your DHCPv6 client. This will result in a new DHCPv6 from your router which will in turn be responded to with a /60 (make sure you request a /60 in the DHCPv6 SOLICIT). This is the best self service approach I can offer, otherwise, the only way for me to help is to manually request the deactivation of the target /64 which in turn prevent the same from being RENEWed. The latter has some scaling issues, i.e. you need me to be involved with each and every request.
Let me know if the above does or does not make sense. And more importantly if what I suggested addresses the issue.
John
-
So what it sounds like might be happening is that since pfSense is requesting renewal of a /60, and Comcast is responding with a /64, pfSense is ignoring the /64 (not necessarily a bad practice if you have set up multiple networks using prefix IDs of a /60). Eventually, the /60 lease expires, but pfSense isn't showing that the /60 isn't valid anymore.
So pfSense might need to take action if the IPv6 prefix renewal doesn't complete as expected and/or the lease received previously expires… though it does sound like the root issue here is on Comcast's side, with the server responding with incorrect prefix size.
