IPv6 with track interface on LAN stopped working

[Dudleydogg]

@derelict Yes its strange, I have DHCP6 setup on WAN, I was not asking for an IP just PD, which has been working. I have 2 LAN nic's on 2 separate Vlan,s so one PD is 0 the other was dd.
One morning My gig network was down, switch was hosed so I rebooted the switch. Now I can no longer get any ipv6 info. to the Interfaces.
But if I look in the DHCP logs I see a PD being assigned I can see a WAN IP assigned. but they never make it to my Interfaces.
Snippet from the Logs:
Sep 2 15:45:24 dhcp6c 48184 IA_NA address: 2603:9000:ff00:b5:1da7:7fff:a000:b95a pltime=604706 vltime=604706
Sep 2 15:45:24 dhcp6c 48184 get DHCP option IA_PD, len 41
Sep 2 15:45:24 dhcp6c 48184 IA_PD: ID=0, T1=302353, T2=483764
Sep 2 15:45:24 dhcp6c 48184 get DHCP option IA_PD prefix, len 25
Sep 2 15:45:24 dhcp6c 48184 IA_PD prefix: 2603:9000:b591:7800::/56 pltime=604706 vltime=604706
so my Request for PD is being answered but Track interface never gets an assigned Prefix.
also FYI I wiped my config back to Factory, just as a test, setup WAN and LAN and it still does not work.
If I hook laptop directly to the Modem I get WAN ipv6 address.
so something is hosed with Tracking on the interfaces. or my DUID is banned, (but I changed it) I also changed my wan MAC address and get a new ipv4 address now.
but no Ipv6 makes it to the interface.
What would you need to see my issue more clearly.

[Derelict]

What is in /var/etc/dhcp6c_wan.conf ??

[Dudleydogg]

more dhcp6c_wan.conf
interface vmx0 {
send ia-na 0; # request stateful address
send ia-pd 0; # request prefix delegation
request domain-name-servers;
request domain-name;
script "/var/etc/dhcp6c_wan_script.sh"; # we'd like some nameservers please
};
id-assoc na 0 { };
id-assoc pd 0 {
prefix ::/56 infinity;
prefix-interface vmx1 {
sla-id 0;
sla-len 8;
};
prefix-interface vmx2 {
sla-id 187;
sla-len 8;
};
};

[Derelict]

Looks like what should be there.

What is after that in the DHCP logs? There should be information about creating and updating prefixes below the part where it gets the IA_PD response. The create a prefix and add an address in this log is this node's track interface LAN with an id of 1. Just post a whole sequence if you can.

Sep/04/2018 00:52:24: Sending Solicit
Sep/04/2018 00:52:24: set client ID (len 14)
Sep/04/2018 00:52:24: set identity association
Sep/04/2018 00:52:24: set elapsed time (len 2)
Sep/04/2018 00:52:24: set option request (len 4)
Sep/04/2018 00:52:24: set IA_PD prefix
Sep/04/2018 00:52:24: set IA_PD
Sep/04/2018 00:52:24: send solicit to ff02::1:2%re1
Sep/04/2018 00:52:24: reset a timer on re1, state=SOLICIT, timeo=4, retrans=16326
Sep/04/2018 00:52:24: receive advertise from fe80::208:a2ff:fe0a:593f%re1 on re1
Sep/04/2018 00:52:24: get DHCP option identity association, len 40
Sep/04/2018 00:52:24:   IA_NA: ID=0, T1=0, T2=0
Sep/04/2018 00:52:24: get DHCP option IA address, len 24
Sep/04/2018 00:52:24:   IA_NA address: 2001:dabb:ad00:7fff::ed96:eec5 pltime=4500 vltime=7200
Sep/04/2018 00:52:24: get DHCP option IA_PD, len 41
Sep/04/2018 00:52:24:   IA_PD: ID=0, T1=0, T2=0
Sep/04/2018 00:52:24: get DHCP option IA_PD prefix, len 25
Sep/04/2018 00:52:24:   IA_PD prefix: 2001:dabb:ad00:fc00::/56 pltime=4500 vltime=7200
Sep/04/2018 00:52:24: get DHCP option client ID, len 14
Sep/04/2018 00:52:24:   DUID: 00:01:00:xx:xx:xx:xx:xx:fe:e0:54:6e:79:49
Sep/04/2018 00:52:24: get DHCP option server ID, len 14
Sep/04/2018 00:52:24:   DUID: 00:01:00:01:21:6c:b6:e4:00:08:a2:0a:59:3f
Sep/04/2018 00:52:24: get DHCP option DNS, len 16
Sep/04/2018 00:52:24: server ID: 00:01:00:01:21:6c:b6:e4:00:08:a2:0a:59:3f, pref=-1
Sep/04/2018 00:52:24: reset timer for re1 to 0.997431
Sep/04/2018 00:52:25: picked a server (ID: 00:01:00:01:21:6c:b6:e4:00:08:a2:0a:59:3f)
Sep/04/2018 00:52:25: Sending Request
Sep/04/2018 00:52:25: a new XID (777d7b) is generated
Sep/04/2018 00:52:25: set client ID (len 14)
Sep/04/2018 00:52:25: set server ID (len 14)
Sep/04/2018 00:52:25: set IA address
Sep/04/2018 00:52:25: set identity association
Sep/04/2018 00:52:25: set elapsed time (len 2)
Sep/04/2018 00:52:25: set option request (len 4)
Sep/04/2018 00:52:25: set IA_PD prefix
Sep/04/2018 00:52:25: set IA_PD
Sep/04/2018 00:52:25: send request to ff02::1:2%re1
Sep/04/2018 00:52:25: reset a timer on re1, state=REQUEST, timeo=0, retrans=955
Sep/04/2018 00:52:25: receive reply from fe80::208:a2ff:fe0a:593f%re1 on re1
Sep/04/2018 00:52:25: get DHCP option identity association, len 40
Sep/04/2018 00:52:25:   IA_NA: ID=0, T1=0, T2=0
Sep/04/2018 00:52:25: get DHCP option IA address, len 24
Sep/04/2018 00:52:25:   IA_NA address: 2001:dabb:ad00:7fff::ed96:eec5 pltime=4500 vltime=7200
Sep/04/2018 00:52:25: get DHCP option IA_PD, len 41
Sep/04/2018 00:52:25:   IA_PD: ID=0, T1=0, T2=0
Sep/04/2018 00:52:25: get DHCP option IA_PD prefix, len 25
Sep/04/2018 00:52:25:   IA_PD prefix: 2001:dabb:ad00:fc00::/56 pltime=4500 vltime=7200
Sep/04/2018 00:52:25: get DHCP option client ID, len 14
Sep/04/2018 00:52:25:   DUID: 00:01:00:xx:xx:xx:xx:xx:fe:e0:54:6e:79:49
Sep/04/2018 00:52:25: get DHCP option server ID, len 14
Sep/04/2018 00:52:25:   DUID: 00:01:00:01:21:6c:b6:e4:00:08:a2:0a:59:3f
Sep/04/2018 00:52:25: get DHCP option DNS, len 16
Sep/04/2018 00:52:25: dhcp6c Received REQUEST
Sep/04/2018 00:52:25: nameserver[0] 2001:dabb:ad00:7fff::1
Sep/04/2018 00:52:25: make an IA: PD-0
Sep/04/2018 00:52:25: create a prefix 2001:dabb:ad00:fc00::/56 pltime=4500, vltime=7200
Sep/04/2018 00:52:25: add an address 2001:dabb:ad00:fc01:fce0:54ff:fe6e:7949/64 on re0
Sep/04/2018 00:52:25: T1(2250) and/or T2(3600) is locally determined
Sep/04/2018 00:52:25: make an IA: NA-0
Sep/04/2018 00:52:25: create an address 2001:dabb:ad00:7fff::ed96:eec5 pltime=4500, vltime=14320663271269473312
Sep/04/2018 00:52:25: add an address 2001:dabb:ad00:7fff::ed96:eec5/128 on re1
Sep/04/2018 00:52:25: T1(2250) and/or T2(3600) is locally determined
Sep/04/2018 00:52:25: executes /var/etc/dhcp6c_wan_dhcp6withoutra_script.sh
Sep/04/2018 00:52:27: script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh" terminated
Sep/04/2018 00:52:27: removing an event on re1, state=REQUEST
Sep/04/2018 00:52:27: removing server (ID: 00:01:00:01:21:6c:b6:e4:00:08:a2:0a:59:3f)
Sep/04/2018 00:52:27: got an expected reply, sleeping.

[Dudleydogg]

This was working perfectly, then all the sudden it stopped working. I can see in Logs where a PD is coming down the pipe but the LAN Track interface never populates with the PD.
for Testing purposes I built a new Pfsense box, No config just blank, setup WAN LAN, enabled Ipv6 and same result.
Sorry if my Logs are upside down, box was checked to have recent at top. This is what confuses it looks like I am getting correct responses.

Sep 3 21:09:46 dhcp6c 49426 reset a timer on vmx0, state=REQUEST, timeo=1, retrans=1845
Sep 3 21:09:46 dhcp6c 49426 send request to ff02::1:2%vmx0
Sep 3 21:09:46 dhcp6c 49426 set IA_PD
Sep 3 21:09:46 dhcp6c 49426 set IA_PD prefix
Sep 3 21:09:46 dhcp6c 49426 set option request (len 4)
Sep 3 21:09:46 dhcp6c 49426 set elapsed time (len 2)
Sep 3 21:09:46 dhcp6c 49426 set identity association
Sep 3 21:09:46 dhcp6c 49426 set IA address
Sep 3 21:09:46 dhcp6c 49426 set server ID (len 14)
Sep 3 21:09:46 dhcp6c 49426 set client ID (len 14)
Sep 3 21:09:46 dhcp6c 49426 Sending Request
Sep 3 21:09:45 dhcp6c 49426 reset a timer on vmx0, state=REQUEST, timeo=0, retrans=911
Sep 3 21:09:45 dhcp6c 49426 send request to ff02::1:2%vmx0
Sep 3 21:09:45 dhcp6c 49426 set IA_PD
Sep 3 21:09:45 dhcp6c 49426 set IA_PD prefix
Sep 3 21:09:45 dhcp6c 49426 set option request (len 4)
Sep 3 21:09:45 dhcp6c 49426 set elapsed time (len 2)
Sep 3 21:09:45 dhcp6c 49426 set identity association
Sep 3 21:09:45 dhcp6c 49426 set IA address
Sep 3 21:09:45 dhcp6c 49426 set server ID (len 14)
Sep 3 21:09:45 dhcp6c 49426 set client ID (len 14)
Sep 3 21:09:45 dhcp6c 49426 a new XID (cd1ea5) is generated
Sep 3 21:09:45 dhcp6c 49426 Sending Request
Sep 3 21:09:45 dhcp6c 49426 server ID: 00:01:00:01:21:c2:86:9d:00:50:56:97:d7:cd, pref=255
Sep 3 21:09:45 dhcp6c 49426 preference: 255
Sep 3 21:09:45 dhcp6c 49426 get DHCP option preference, len 1
Sep 3 21:09:45 dhcp6c 49426 IA_PD prefix: 2603:9000:b505:bb00::/56 pltime=567719 vltime=567719
Sep 3 21:09:45 dhcp6c 49426 get DHCP option IA_PD prefix, len 25
Sep 3 21:09:45 dhcp6c 49426 IA_PD: ID=0, T1=283859, T2=454175
Sep 3 21:09:45 dhcp6c 49426 get DHCP option IA_PD, len 41
Sep 3 21:09:45 dhcp6c 49426 IA_NA address: 2603:9000:ff00:b5:5cc7:f677:e6e4:6a7e pltime=566005 vltime=566005
Sep 3 21:09:45 dhcp6c 49426 get DHCP option IA address, len 24
Sep 3 21:09:45 dhcp6c 49426 IA_NA: ID=0, T1=283002, T2=452804
Sep 3 21:09:45 dhcp6c 49426 get DHCP option identity association, len 40
Sep 3 21:09:45 dhcp6c 49426 DUID: 00:01:00:01:21:c2:86:9d:00:50:56:97:d7:cd
Sep 3 21:09:45 dhcp6c 49426 get DHCP option server ID, len 14
Sep 3 21:09:45 dhcp6c 49426 DUID: 00:01:00:01:22:e4:18:42:00:50:56:b7:ee:fb
Sep 3 21:09:45 dhcp6c 49426 get DHCP option client ID, len 14
Sep 3 21:09:45 dhcp6c 49426 receive advertise from fe80::2bc:60ff:fe93:1419%vmx0 on vmx0
Sep 3 21:09:45 dhcp6c 49426 reset a timer on vmx0, state=SOLICIT, timeo=1, retrans=2083
Sep 3 21:09:45 dhcp6c 49426 send solicit to ff02::1:2%vmx0
Sep 3 21:09:45 dhcp6c 49426 set IA_PD
Sep 3 21:09:45 dhcp6c 49426 set IA_PD prefix
Sep 3 21:09:45 dhcp6c 49426 set option request (len 4)
Sep 3 21:09:45 dhcp6c 49426 set elapsed time (len 2)
Sep 3 21:09:45 dhcp6c 49426 set identity association
Sep 3 21:09:45 dhcp6c 49426 set client ID (len 14)
Sep 3 21:09:45 dhcp6c 49426 Sending Solicit
Sep 3 21:09:44 dhcp6c 49426 reset a timer on vmx0, state=SOLICIT, timeo=0, retrans=1091
Sep 3 21:09:44 dhcp6c 49426 send solicit to ff02::1:2%vmx0
Sep 3 21:09:44 dhcp6c 49426 set IA_PD
Sep 3 21:09:44 dhcp6c 49426 set IA_PD prefix
Sep 3 21:09:44 dhcp6c 49426 set option request (len 4)
Sep 3 21:09:44 dhcp6c 49426 set elapsed time (len 2)
Sep 3 21:09:44 dhcp6c 49426 set identity association
Sep 3 21:09:44 dhcp6c 49426 set client ID (len 14)
Sep 3 21:09:44 dhcp6c 49426 a new XID (f6776d) is generated
Sep 3 21:09:44 dhcp6c 49426 Sending Solicit

[Dudleydogg]

Also like to mention that during boot I see WAN Syslodg: Bind : Can't assign requested address.
sorry typed that from Memory, but it hangs the box for a few minutes, have read its FE80 Addresses that are stuck on the interface.

[Derelict]

That is not representative of actually getting a response. All I see there is Send Solicit. Look at the log I posted. See the send request and receive reply.

[Dudleydogg]

so I am not receiving a reply or is their anyway possible I'm Blocking it? When I hook laptop to Modem reboot everything I get WAN ipv6 address, so I am not blaming the ISP I have to be missing something.

Also I do have a Tunnel to HE.net and I do not forward all traffic out the HE gateway, I have a Rule on specific Vlan to route that traffic.

[Dudleydogg]

Log finally posted this Says No responses were received?

Sep 3 21:18:35 dhcp6c 86742 removing server (ID: 00:01:00:01:21:c2:86:9d:00:50:56:97:d7:cd)
Sep 3 21:18:35 dhcp6c 86742 removing an event on vmx0, state=REQUEST
Sep 3 21:18:35 dhcp6c 86742 no responses were received
Sep 3 21:18:07 dhcp6c 86742 reset a timer on vmx0, state=REQUEST, timeo=9, retrans=27750
Sep 3 21:18:07 dhcp6c 86742 send request to ff02::1:2%vmx0
Sep 3 21:18:07 dhcp6c 86742 set IA_PD
Sep 3 21:18:07 dhcp6c 86742 set IA_PD prefix
Sep 3 21:18:07 dhcp6c 86742 set option request (len 4)
Sep 3 21:18:07 dhcp6c 86742 set elapsed time (len 2)
Sep 3 21:18:07 dhcp6c 86742 set identity association

Found this also in the Log, is this .Key thing a problem?

Sep 3 21:15:15 dhcp6c 5912 <5>[vmx0] (4)
Sep 3 21:15:15 dhcp6c 5912 <3>[interface] (9)
Sep 3 21:15:15 dhcp6c 5912 skip opening control port
Sep 3 21:15:15 dhcp6c 5912 failed initialize control message authentication
Sep 3 21:15:15 dhcp6c 5912 failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
Sep 3 21:15:15 dhcp6c 5912 extracted an existing DUID from /var/db/dhcp6c_duid: 00:01:00:01:22:e4:18:42:00:50:56:b7:ee:fb

[Derelict]

Packet capture for IPv6 ICMPv6 on WAN and see what's actually going on out there. Maybe they don't like the DUID since they already think they have a lease out to the laptop. Are you requesting a PD on the laptop? I doubt it so that isn't a comparable test.

Have you called them? Maybe they need to clear something in their DHCP server.

No, that control port is not an issue.

| Sep 3 21:18:35 dhcp6c 86742 no responses were received

Exactly.

[Dudleydogg]

Correct on the laptop did not request a PD not really a valid test, other than It does get a WAN ipv6 address, so the least I should get one on the Wan IP on the pfsense box.
I will generate a custom unique UUID and force that out, you have a good point as pfsense spoofs the MAC and UUID

[Derelict]

I would get rid of any spoofed MAC addresses unless you know you need them. That's really hacky. It's almost always better to just have the ISP do what they need to do (if anything) so you can use the new MAC address.

DHCPv6 doesn't use MAC as an identifier. It uses the DUID. Thought part of the DUID generation input might be a link-layer (MAC) address.

[Dudleydogg]

I did that early today took out all the MAC fields, made sure they were Actually the MAC of that Interface card. SO im all clean there. the DUID or UUID is generated using the WAN's mac address.

Running packet capture now but its not finding the conversation just some icmp stuff.

[Dudleydogg]

I setup Syslog and can filter for dhcpd6, so I caught this in the Log when saving WAN/LAN forcing dhcpd6 to update:

reset a timer on vmx0
transmit failed: Can't assign requested address
set IA_PD
set IA_PD prefix
set option request (len 4)
set elapsed time (len 2)
set identity association
set client ID (len 18)
Sending Solicit
reset a timer on vmx0
transmit failed: Can't assign requested address
set IA_PD
set IA_PD prefix
set option request (len 4)
set elapsed time (len 2)
set identity association
set client ID (len 18)
Sending Solicit
reset a timer on vmx0
transmit failed: Can't assign requested address

[dxmaster]

@Derelict you live in Vegas so I’m guessing you have Cox as well. Any tips to get ipv6 working? I just redid my install from scratch, set lan to track, nothing checked on WAN. Still only getting link-local ipv6 address. It’s driving me crazy!

[Derelict]

This is what I use with Cox and a Netgear CM600:

[dxmaster]

so these are the logs from dhcp, any hints from them as to what is going on? I even tried swapping interfaces just now and still no good.

[Derelict]

Anything else ever happen? It's not uncommon to take a few solicits before getting a response. Not sure why they delay.

With Cable it is strange because you are often actually talking to your modem, which is obtaining the address information from upstream via whatever method (I have never seen a Cable ISP any further up than the modem itself so it's a "black box" to me). Have you verified with Cox that your modem will work with IPv6 and that they don't have to enable something?

[dxmaster]

@derelict Yeah, if I go direct into my laptop from the modem I get an IPv6 address right away. I just switched back over to my edgerouter and boom, just like that it has an IPv6 address.

[dxmaster]

Ok, so if I plug the edgerouter into the modem then feed pfsense from the edgerouter it gets an ipv6 address but not direct to the modem. What the heck?!