Comcast ipv6 routing? problem
-
I'll lead with I'm ipv4 experienced but a ipv6 newbie.
I've got a box currently running pfSense 2.1.4-RELEASE that has been stable in all things since we originally brought it up ~12 months ago. We originally had some problems keeping ipv6 stable (would loose the LAN ipv6 address and would have to release/new) on the WAN interface to make it work again. But, after a while it stabilized (I assume Comcast change/fixed something). Recently, however, without any changes on our end, ipv6 is no longer routing on the clients. I can ping from the pfSense box (via diagnostics) to ipv6.google.com without a problem but fail from all the workstations. Looking at the RRD Graphs/Traffic show's a precipitous drop in ipv6 traffic (~5MB per day out of ~9GB overall traffic when we used to see ~25% ipv6 traffic). And as far as I can tell from various browser extensions we're getting no pages via ipv6.
From what I can see everything looks good address wise. The wan has ipv6 starting with 2001:558:6007:1 and it appears to be /128 (though I think that was a /64 originally). The lan interface and and all the clients get 2701:4:2500:117 based addresses that appear to be /64. The default gateways on the clients include fe80::1:1 (which is also the pfSense lan ipv6 link local as well).
ping -6 on the clients gives what appears to be a valid ipv6 address for ipv6.google.com but constantly times out. This is happening across all the clients (mostly Win7, but a couple Ubuntu servers and a couple Linux Mint boxes). I'm moderately positive it's not a client problem (unless it's fed down via DHCP) considering some of the dozen clients haven't seen config changes in years.
We've gone through various internet guide on ipv6 (specifically with Comcast sometimes) setup with pfSense and as far as we can tell everything is setup fine. Using DHCP6 on the WAN and track interface on the LAN, pointing at the WAN.
Can anyone suggest where to go looking for this problem? Any agreement on my assessment it's routing related? If there's any details missing to help diagnose, please let me know.
Mike
-
From what I can see everything looks good address wise. The wan has ipv6 starting with 2001:558:6007:1 and it appears to be /128 (though I think that was a /64 originally). The lan interface and and all the clients get 2701:4:2500:117 based addresses that appear to be /64.
WAN must hold on to /64 I think.
What exactly do you try to say with "all the clients get 2701:4:2500:117" ?
How do the client boxes get their IPv6 ?
SLAAC ? -
Ok, then any idea how I can get it back to a /64? I thought I read somewhere else on this forum that /128 was ok for WAN. I'll have to look again. Does the WAN DHCP to Comcast get logged in one of the logs? I haven't seen it.
All of the clients get IPv6 addresses with the same prefix and this matches the prefix of the pfSense LAN IPv6. All the clients are set to use DHCP locally so whatever magic makes the DHCP pass through work from client through pfSense to Comcast seems to be working (Track Interface on the LAN setup?). My (limited) understanding based on the Comcast/pfSense guides I've read is that the normal configuration is to allow Comcast to assign addresses to all the local clients rather than having something on pfSense do that.
Based on the IPv6 addresses all having the same prefix I assume they're all being assigned addressed by the same server/method and that they are correct. Might be a bad assumption.
I have no idea what SLAAC is, unfortunately, or how it would/wouldn't be used here.
Some additional ping info: pinging ipv6.google.com specifically from the pfSense WAN works, but does NOT from the pfSense LAN. This would, I think, mean that either the IPv6 LAN address is not good (ie, didn't come from Comcast) or the internal routing on pfSense is not working correctly.
I also see in the routing log:
Aug 13 08:58:20 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:20 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:20 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stopping
Aug 13 08:58:20 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:20 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:20 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stoppingand
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: sendto(s_udp=14): No route to host
Aug 13 08:58:25 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:25 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stopping
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: sendto(s_udp=14): No route to host -
Ok, then any idea how I can get it back to a /64? I thought I read somewhere else on this forum that /128 was ok for WAN. I'll have to look again. Does the WAN DHCP to Comcast get logged in one of the logs? I haven't seen it.
Works fine here with a /128 on the WAN interface. In any case, this prefix length is determined by the upstream router, so there's not way for you to force it (nor is there any point in doing so).
All of the clients get IPv6 addresses with the same prefix and this matches the prefix of the pfSense LAN IPv6. All the clients are set to use DHCP locally so whatever magic makes the DHCP pass through work from client through pfSense to Comcast seems to be working (Track Interface on the LAN setup?). My (limited) understanding based on the Comcast/pfSense guides I've read is that the normal configuration is to allow Comcast to assign addresses to all the local clients rather than having something on pfSense do that.
No, the clients do not get their addresses from Comcast's DHCP server; Comcast delegates a prefix to your pfSense box, and the clients get addresses either through stateless autoconfic (SLAAC) or DHCPv6 from the pfSense box.
Based on the IPv6 addresses all having the same prefix I assume they're all being assigned addressed by the same server/method and that they are correct. Might be a bad assumption.
If they all match the prefix on the pfSense box's LAN interface (and that is set to track the WAN), they should be ok.
Some additional ping info: pinging ipv6.google.com specifically from the pfSense WAN works, but does NOT from the pfSense LAN. This would, I think, mean that either the IPv6 LAN address is not good (ie, didn't come from Comcast) or the internal routing on pfSense is not working correctly.
Assuming you're talking about doing "ping6 -I [lan interface] www.google.com" on your pfSense box, I don't think you'd expect that to work, as there's no default router on that interface. I get "no route to host" when doing that, but IPv6 on my LAN clients works just fine.
I also see in the routing log:
Aug 13 08:58:20 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:20 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:20 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stopping
Aug 13 08:58:20 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:20 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:20 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stoppingand
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: sendto(s_udp=14): No route to host
Aug 13 08:58:25 miniupnpd[3841]: ioctl(s, SIOCGIFADDR, …): Can't assign requested address
Aug 13 08:58:25 miniupnpd[3841]: Failed to get IP for interface em0
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stopping
Aug 13 08:58:25 miniupnpd[3841]: SendNATPMPPublicAddressChangeNotification: sendto(s_udp=14): No route to hostWhat interface is em0? The error means that miniupnpd was unable to determine an IPv4 address for that interface.
-
Also, did you recently make any changes to the DHCP6 client configuration options on the WAN interface page? I know that e.g. changing prefix delegation size has lead to (transient, but very slow to resolve) issues for folks in the past.
-
Ok, thanks for the clarification on those points.
No changes had been made to the IPv6 settings since it went live months (maybe a year?) ago. Recently we've poked around in it some, but returned it to previous settings. Under DHCP for IPv4 we (recently - weeks ago) added a secondary WINS server. It's possible IPv6 has been down that long but I don't think so. It's not something we check normally, we're mostly interested in keeping it up for the future so as the the world wide transition happens we're ready.
Sorry, em0 is the WAN adapter and em1 is the LAN.
I actually did the ping via the web server interface and selected LAN from the combo box. I assumed it would work, but I guess looking at the interface there's no default gateway for the LAN adapter.
Interesting that em0 would have a IPv4 problem, since that's been problem free. Is miniupnpd strictly UPnP? If so, it's possible that's not working, it's not something we use very often.
And suggestions on what to look at for this?
-
Interesting that em0 would have a IPv4 problem, since that's been problem free. Is miniupnpd strictly UPnP? If so, it's possible that's not working, it's not something we use very often.
Yes, that's for UPnP (and NAT-PMP) only, so probably doesn't matter for you. Could just be a timing-related issue.
And suggestions on what to look at for this?
I'd fire up packet captures on your pfSense box's WAN and LAN interface and then try to ping an external address from a LAN client. I'm assuming you do have a firewall rule that allows IPv6 traffic from "LAN network" (as opposed to something hard-coded)?