Receiving /59 PD results in tracking interfaces using /63


  • Hello All -

    I am fighting with Comcast Business to sort out some issues with IPv6. They really don't understand how it works but I can live with that if I wasn't having what looks like a problem with pfSense, also.

    In my case, the modem they foisted on me will only hand out a /59. If pfSense asks for a /60, it will accept the /59 but then assign /63s to the tracking interfaces. This causes slaac not to work.

    Why would pfSense assign /63s to those interfaces configured to track the WAN?


  • @slykens

    Do you understand how DHCPv6-PD works? The /59 or /60 is the prefix assigned to pfsense, which is then split into /64s for each interface. A /60 gives you 16 /64s and /59, 32. You many or may not have a /128 WAN address.


  • @jknott Yes, I understand how IPv6 works. Did you read my post?

    To be absolutely clear -

    The WAN interface is configured for PD with a hint of /60. The modem responds with a /59 which pfSense then accepts. Ostensibly, this should not be a problem - why should getting more than we asked for be a problem?

    The LAN and OPT interfaces are configured to track the WAN interface with appropriate Prefix IDs. When pfSense sees the above, the TRACKING interfaces (LAN, OPT, etc) are assigned /63s from the /59 assigned to pfSense by the modem. Since these interfaces are /63, slaac does not work and the devices on those networks do not auto configure as they would if the interface had a /64 assigned.

    The problem is that when pfSense receives the /59 on the WAN interface, the inside interfaces (LAN, OPT, etc) that are configured to track WAN are then configured with /63 instead of the appropriate /64.

  • LAYER 8 Global Moderator

    @slykens said in Receiving /59 PD results in tracking interfaces using /63:

    are assigned /63s from the /59

    That makes zero sense.. And if that was the case I would think the forums would be on fire with people complaining..

    Please post up this - your saying the status of your interface after is picked prefix out of what was delegated to you is /63? Please post up screen shot of that.

    Your running what version of pfsense 2.4.5p1?


  • @johnpoz Screen Shot 2020-12-24 at 4.13.27 PM.png

    Attached is a screen shot showing what I see. I apologize for the bit of potato redaction but I think the information needed is clear.

    In this scenario pfSense asks for /60, the Comcast modem offers /59, so pfSense takes it.

    edited to add - I have set pfSense to request a /59 and it will properly assign /64s to the tracking interfaces.

  • LAYER 8 Global Moderator

    @slykens said in Receiving /59 PD results in tracking interfaces using /63:

    I have set pfSense to request a /59 and it will properly assign /64s to the tracking interfaces.

    Really - yeah this needs to be reported. My ISP doesn't provide IPv6, so I have no way to even try to test this other than setting up a downstream pfsense..

    Lets call in @jimp and @Derelict and see if they have some info or what info you could post up about this.. I will have a quick look see in redmine to see if already reported.

    But from what you posted and what you say you can fix it by just requesting /59 does seem like something really weird going on..


  • @johnpoz I concur, doctor, something isn't right, thus my posting about it. :)

    I'm happy to provide what other information I can so, anyone, please ask.

    To provide a little more insight - this is on Comcast's business service with their CGA4131COM modem and static IPv4. The modem will only give me a /59 as best as I can tell and initially it wanted to give me a /59 that started with the /64 it had assigned to its LAN interface - something that is obviously braindead and won't work. It is now at least offering me the ba40:: network as a /59 (instead of ba00::) but there is apparently something in the modem that is dropping traffic destined for the delegated network that is preventing it from working for me. I have a tier 2 tech helping figure this out and right now they seem to agree it is a problem on their end and likely in the modem itself.

    My home residential service with an SB8200 but purely dynamic works perfectly with pfSense requesting a /60 and distributing /64s to my inside interfaces as I desire.


  • While I don't disagree about the problem, I also have to ask... if you know that pfSense is getting a /59, why don't you adjust the setting in WAN to show that you're getting a /59? Then most likely all will begin working as expected.

  • LAYER 8 Global Moderator

    Yeah he did that..

    • I have set pfSense to request a /59 and it will properly assign /64s to the tracking interfaces.

    But there is still something wrong if a slightly different PD gets sent that the lan side mask is not a /64

  • LAYER 8 Netgate

    @johnpoz @slykens

    It would be really nice if Comcast fixed their IPv6 deployment. You should be getting what you ask for. Seems all of their IPv6 modem/router code is hopelessly broken. /59 is just plain silly. Calling them to try to get them to fix it is like calling this guy: 🤡

    As I understand the problem, there is no good way to get the issued PD out of dhcp6c. It doesn't matter what they actually delegate. All the math done in pfSense assumes you receive what you asked for.

    This same issue holds a lot of IPv6 things that would be nice like an automatic "Alias" for the delegated prefix and things that would be much more sane if there was an interface to get the delegated prefix from dhcp6.


  • It would be interesting to see what a packet capture of the DHCPv6-PD sequence shows.


  • @jknott dhcpv6-60-59.pcap Attached please find a capture of the dhcp conversation.

    In this cap, pfsense asks for 60, the modem responds with a 59, pfsense then requests the 59 and the modem assigns it.


  • @derelict said in Receiving /59 PD results in tracking interfaces using /63:

    @johnpoz @slykens

    It would be really nice if Comcast fixed their IPv6 deployment. You should be getting what you ask for. Seems all of their IPv6 modem/router code is hopelessly broken. /59 is just plain silly. Calling them to try to get them to fix it is like calling this guy: 🤡

    Comcast's residential IPv6 works great. Business is a total mess. Now they are chasing a pre-routing firewall in their modem that appears to be dropping packets destined for the delegated network on my side. At least the tier 2 guy I am working with acknowledges there is a problem and is trying to get it corrected. As mentioned, I can happily live with the /59 and pfSense will work properly with it when the configuration matches, but I can't yet pass traffic due to the modem's wonkiness.

    As I understand the problem, there is no good way to get the issued PD out of dhcp6c. It doesn't matter what they actually delegate. All the math done in pfSense assumes you receive what you asked for.

    Perhaps not a perfect solution but the prefix acquired does show up in the dhcp.log so it should be parseable to get the info:

    Dec 24 11:19:28 gateway dhcp6c[53540]: get DHCP option IA_PD prefix, len 25
    Dec 24 11:19:28 gateway dhcp6c[53540]: IA_PD prefix: 2603:3007:x❌::/59 pltime=344481 vltime=344481

    This same issue holds a lot of IPv6 things that would be nice like an automatic "Alias" for the delegated prefix and things that would be much more sane if there was an interface to get the delegated prefix from dhcp6.

    The folks pushing Netplan on Ubuntu have finally fixed token support. Small steps are making it better all the time. I agree overall on the sentiment - it would be awful nice if everything "just worked" the way it has been advertised to. I don't know that adoption would finally become universal but it wouldn't hurt. So many of my IT peers are ambivalent to anti-IPv6 that it feels like I'm the odd one out.


  • @virgiliomi said in Receiving /59 PD results in tracking interfaces using /63:

    if you know that pfSense is getting a /59, why don't you adjust the setting in WAN to show that you're getting a /59? Then most likely all will begin working as expected.

    I said in the edit to my third post in this thread that that is what I have done. That does not mean that pfSense is behaving properly, however.


  • @slykens

    1. I see the /60 solicit
    2. I see the /59 offer
    3. I see the /59 request
    4. I see the /59 provided

    So, Comcast is giving a /59 when a /60 is requested, but pfsense then accepts it. I have no idea why pfsense is messing up a prefix that is accepted, but you should check with Comcast about why they're providing a /59, when a /60 is requested.

  • LAYER 8 Global Moderator

    @jknott said in Receiving /59 PD results in tracking interfaces using /63:

    I have no idea why pfsense is messing up a prefix that is accepted

    Yeah that is what I think should be fixed.. I hear ya the ISP is messed up.. But if you get /X as your prefix no matter if its what you asked for or not as far as size... I would hope pfsense should create your /64s from this - as long as its large enough to create your prefixes off of, etc..

    Its quite possible the coding of this is more involved than my noncoder brain thinks it is - but it doesn't seem like a optimal result setting /63s - which are not good..

    It very well could be a very low priority thing as well.. Since you would hope 999999/1000000 times the ISP would hand you the prefix you request..

    Maybe logging the mismatched prefix request vs assigning a bad prefix on the tracking interfaces would be better than assigning a /63 or /whatever that is not going to be good.


  • @johnpoz said in Receiving /59 PD results in tracking interfaces using /63:

    It very well could be a very low priority thing as well.. Since you would hope 999999/1000000 times the ISP would hand you the prefix you request..

    I agree it’s not a dealbreaker but in this situation fixing pfSense is more agile than expecting Comcast to resolve their rectal cranial inversion quickly and likely benefits others who may have simply given up since IPv6 isn’t a requirement in many deployments. I’m very lucky to be working with a tech at Comcast who has not yet told me to pound sand.

    I also think it would be good if pfSense would respond to brain dead situations by doing the best it can - like you said, if there’s enough space to make /64s then do what it can. This isn’t a work around just for Comcast, per se, it would be a fix so pfSense would behave as expected.


  • @jknott said in Receiving /59 PD results in tracking interfaces using /63:

    @slykens

    1. I see the /60 solicit
    2. I see the /59 offer
    3. I see the /59 request
    4. I see the /59 provided

    So, Comcast is giving a /59 when a /60 is requested, but pfsense then accepts it. I have no idea why pfsense is messing up a prefix that is accepted, but you should check with Comcast about why they're providing a /59, when a /60 is requested.

    I don’t mean to be rude but I don’t think you read what I’ve posted. This is the second post in the thread from you where you either misunderstand what has been posted or you try to direct the conversation away from the issue posted about, in this case that pfSense mishandles assigning tracking interfaces /64s when it receives an “unexpected” delegation.

    To go down that road, yes you are correct that Comcast should handle it correctly, especially as a /60 is smaller than a /59, but Comcast has insisted to its customers for years that their modems work fine and when they finally get cornered have been telling customers to get bent. In my case the tech I am working with has taken it seriously and I (and other Comcast customers) can live with a /59 instead of a /60. PfSense properly handling the unexpected delegation would make the whole thing work more smoothly. I suggest this is important to fix as Comcast is a technology leader and their methods are likely to be replicated by other ISPs in the US and it’s not like it is a workaround only for Comcast - it’s fixing it so it behaves properly.

  • LAYER 8 Netgate

    @jknott They send a /59. It's really stupid. So are they.

  • LAYER 8 Netgate

    @slykens Why should pfSense have to code around Comcast's long-term inability to provide the service you are paying for? Comcast Business IPv6 is a complete clown car.

    I have already explained the technical limitations surrounding this issue from the pfSense perspective. If you are supposed to get a /56 or a /60 and explicitly request a /56 or /60, and get a /59 (!) instead it is Comcast's provisioning that is broken - not the firewall. Should pfSense assume the technical debt in forking and maintaining their own dhcp6c because Comcast is hopelessly broken, continues to be broken for years, and tells you your third-party firewall is beyond our demarc and we can't help you with it despite multiple customers all providing the same data that Comcast's gear is at fault?

    Comcast doesn't care. Use an HE.net tunnel if Comcast is your only choice.


  • @slykens

    If Comcast is anything like my ISP (Rogers), it has nothing to do with the modem. This is handled at the head end office, by the CMTS, which talks to the modem at the customer site. The modem, in bridge mode, as needed by pfsense for DHCPv6-PD to work, should be transparent. The alternative is for it to be in gateway mode, where a single /64 is provided it. In this case, you cannot use another router beyond it, to distribute a prefix.

    The hard part may be getting the right people to work on the problem. A couple of years ago, I had a problem and was even able to identify the failing CMTS by host name. Even though both tier two support and a senior tech could see the problem was back at the head end, the appropriate people refused to work on the problem because I was running my own router (pfsense), ignoring the fact it was also happening to my next door neighbour, who was using the modem in gateway mode. It was only after the senior tech took his modem to the head end and tried it with 4 different CMTS, with it failing only on the one I was connected to, did those guys get off their butt to fix their problem.

    Bottom line, Comcast should not be providing a /59, when a /60 is requested and that leaves the issue of pfsense not properly handling a prefix that it was given, even if incorrectly.

    Incidentally, I have done work inside my ISP's offices in a few cities, though not the one I'm connected to and on equipment connected to the CMTS.


  • @derelict said in Receiving /59 PD results in tracking interfaces using /63:

    Why should pfSense have to code around Comcast's long-term inability to provide the service you are paying for? Comcast Business IPv6 is a complete clown car.

    Because it is not a fix "just for Comcast," even though they are the trouble in the instant scenario.

    Expecting pfSense to behave logically is reasonable. Do you think it is reasonable that pfSense mishandles assigning /64s when it receives a larger delegation than expected? Do you believe this is the only potential or possible scenario where this could occur?


  • Comcast or not, this is not the first time I’ve seen pfSense’s inability to handle a prefix size different from what has been requested. It really should be fixed, if for no reason than to provide a better user experience.

    Does it require forking dhcpd just for pfSense? I would be surprised if it needed that drastic of a measure. But I’m not a developer, so I don’t know.


  • @jknott said in Receiving /59 PD results in tracking interfaces using /63:

    Bottom line, Comcast should not be providing a /59, when a /60 is requested and that leaves the issue of pfsense not properly handling a prefix that it was given, even if incorrectly.

    I absolutely agree.

    I also believe, however, it is reasonable to expect pfSense to properly handle what it does get as long as it can reasonable meet the need - in this case a /59 in place of a /60 obviously exceeds the need.


  • @virgiliomi

    My take on this is if there actually is a problem within pfsense, then it's in the FreeBSD it's based on and any such problem and possible fix should be sent upstream.


  • @jknott said in Receiving /59 PD results in tracking interfaces using /63:

    If Comcast is anything like my ISP (Rogers), it has nothing to do with the modem. This is handled at the head end office, by the CMTS, which talks to the modem at the customer site. The modem, in bridge mode, as needed by pfsense for DHCPv6-PD to work, should be transparent. The alternative is for it to be in gateway mode, where a single /64 is provided it. In this case, you cannot use another router beyond it, to distribute a prefix.

    Comcast's business offering in regards to IPv6 is a mess as others here have astutely pointed out.

    If you have static IPv4 service, you must use one of their modems. They use some flavor of RIP on the cable side to route your block to you and a "consumer" modem, like an SB8200, apparently won't do what they want. Again, if I was fully dynamic, both IPv4 and IPv6, as I am at my home this entire situation would not even have come up and would be handled upstream as you suggest.

    The modem Comcast provides doesn't operate in bridge mode when static IPv4 is involved. It is the gateway on my /29 in IPv4. Comcast assigns the modem a /56, then modem assigns a /64 to its LAN interface, and the modem is responsible for handling the rest of it on its LAN interface side. In the instant scenario, I desire to ask it for a /60 but it will only give me a /59 - that's a limitation in the modem firmware as the PD request never goes beyond the modem as it has a /56 to give me address space out of. Now, I have configured pfSense to ask for the /59 and it properly handles assigning /64s internally but there's another problem in Comcast's modem where that traffic is being dropped by the modem - Comcast has told me this much - so I'm still waiting for them to suss that but expect everything to work happily once they do.

    I suspect if I paid for a metro E delivered service I could have things done very differently but 200 Mbps DOCSIS is $99/mo whilst I suspect the ethernet service would start at $1000 and go up from there. Yes, I know, you get what you pay for. :)

  • LAYER 8 Netgate

    @slykens Pull requests will be evaluated for appropriateness.


  • In the end, pfSense handling of IPv6 isn't ideal, for example not showing what the isp is providing. The router (with modem) I had to buy for my new dsl line shows this and even can delegate prefixes to pfSense. You got what you paid for, I guess.


  • Hmm... look... another ISP (in Germany this time) with the same issue. I guess Comcast isn't the only one broken. Can this be looked into now to see where the problem lies as far as pfSense's handling of prefix size received being different from prefix size requested?

    https://forum.netgate.com/topic/159463/ipv6-not-working-wan-and-lan-interface-getting-an-ip-adress-not-any-client