No IPv6 after upgrade to 23.01

brmoon

I'm a Verizon FIOS customer for home internet. Like many others, IPv6 stopped working after the upgrade to 23.01. I tried a great many things to get IPv6 working again after the upgrade to 23.01. While I can't possibly recount all the things, I will share the last thing I did that resulted in delegated IPv6 being reassigned to my LAN network.

From the pfSense console I used the 2) Set interface(s) IP address option. Both IPv4 and IPv6 were again set to DHCP. (What I would consider to be no change.)

When the console menu came back up, my IPv6 was present on LAN as I had previous experienced before the upgrade.

Hope this helps someone else.

maverickws

@forresgeek Hi

I tried this: set my MTU to 1480, but unfortunately didn't change the situation.

mhillmann

For what it's worth. We've found a permanent solution for our issue for now (meaning no M+O flags set in RA announcements), testing it for long term validity:
We configure the interface as static with the fixed GUA address next hop gateway extracted from Wireshark capture. Then we change the IPV6 interface from static to dynamic and don't delete the static gateway (in ROUTING tab) to use it as default gateway for the interface. Rtsold() doesn't deliver any gateway for the interface BUT we've got one from the static definition, so we can route IPV6 packtes normally. As the unassigned dynamic gateway doesn't interfere with routing but dhcp6c() renews NA+PD delegations on expiration, we think we've got a stable configuration that will endure over time as long as the defined (fixed) next hop gateway with a GUA address stays valid, as it apperently does.
I acknowledge that this is a VERY nonstandard configuration we get from our ISP, but apparently their whole network is configured this way and their own (mainly Huawei) routers respect this arrangement when set to router mode as configured by default, delivering a correctly working IPV6 network on the internal side, RA-wise, meaning the RA sets the M+O flags as the RFC demands and pfSense can work as intended. The whole issue only becomes relevant if trying to use a different router, needing the ONT to be put into bridge mode.
Nonetheless it would be usefull to acknowledge this use case and see if is possible to force a fixed gateway even with DHCPv6.

maverickws

@mhillmann hi there,

based on your answer I thought of applying this to our setup and see how it would go.

The approach was slightly different, before when working I knew what was the GW address assigned to the IPv6.

So I simply added a new gateway on the routing tab, used that GW address and selected it as gateway.

So this did change something as now I can ping google ipv6 from Diagnostics > Ping

16 bytes from 2a00:1450:4003:806::200e, icmp_seq=0 hlim=59 time=15.372 ms
16 bytes from 2a00:1450:4003:806::200e, icmp_seq=1 hlim=59 time=15.177 ms
16 bytes from 2a00:1450:4003:806::200e, icmp_seq=2 hlim=59 time=15.382 ms
16 bytes from 2a00:1450:4003:806::200e, icmp_seq=3 hlim=59 time=22.012 ms
16 bytes from 2a00:1450:4003:806::200e, icmp_seq=4 hlim=59 time=15.506 ms

--- ipv6.l.google.com ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 15.328/18.726/30.600/5.971 ms

My issue now is that, for some reason, I don't get IPv6 connectivity on local machines. I get IPv6 address but no route.

On my machine netstat gives me a default v6 gateway which is fe80::abcd:ef01:fe03:bd10 being that this is my pfSense LAN interface local-link address. So I can't really figure out now why I have connectivity on the pfSense LAN interface but not on local hosts.

If I ping that GW my machine has configured:

# ping6 fe80::abcd:ef01:fe03:bd10%en0
PING6(56=40+8+8 bytes) fe80::8af:1f71:bffc:afc1%en0 --> fe80::abcd:ef01:fe03:bd10%en0
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=0 hlim=64 time=1.323 ms
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=1 hlim=64 time=1.381 ms
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=2 hlim=64 time=1.302 ms
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=3 hlim=64 time=1.269 ms
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=4 hlim=64 time=2.170 ms
16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=5 hlim=64 time=2.981 ms
^C
--- fe80::abcd:ef01:fe03:bd10%en0 ping6 statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.269/1.738/2.981/0.638 ms

EDIT: This is a temporary (not totally working still) fix not a solution.
DHCP6 should take care of this without the need of having to find out the GW for the next hop and adding static gateways.

I am appalled how no one from Netgate is getting involved in this. This was a breaking change as MANY ISP's (and I mean many) will have similar setups and hardware.
Sweep it under the rug, don't talk about it, and it's gone, is that it?

@jimp @Derelict

SteveITS

@maverickws Is the mask correct? There was another post recently where the person was saying devices were getting a /128 mask not /64.

IPv6 is working fine here.

maverickws

@steveits I'm attributed a /56 prefix;

WAN configured as DHCP6
LAN, IoT and DOM (all local networks, IoT and DOM are VLANS) configured as tracking WAN interface for IPv6

Each interface gets a /64 prefix delegation size.

This is the same configuration (exactly the same configuration) that has been working for years with IPv6.

The IPv6 addresses are correctly attributed to all the internal interfaces. Actually, on the above test (after creating a static gateway as suggested) pinging ipv6.google.com you can see the pfSense pinging from its IPv6 LAN address.

I would refer to Mr. @mhillmann 's post here: https://forum.netgate.com/post/1088842

As this seems the most on point explanation of what might be happening.

SteveITS

@maverickws If it can be replicated (seems so), I suggest opening an issue at redmine.pfsense.org, with the details and any logs, referencing this thread.

And/or ask the ISP, if they are not following an RFC. Is everyone with this issue using the same ISP?

23.01 skipped from FreeBSD 12 to 14 so it's likely a lot of internal things changed.

Derelict

@maverickws said in No IPv6 after upgrade to 23.01:

DHCP6 should take care of this without the need of having to find out the GW for the next hop and adding static gateways.

IPv6 does not get its default next-hop from DHCP[6] like IPv4 does. It relies on router advertisements. To reiterate, there is nothing built into DHCP6 to propagate a next-hop gateway. It must be in an acceptable router advertisement.

If the router advertisements are non-standard then the problem is at the ISP.

I, personally, find it pretty appalling that an ISP would deploy IPv6 in such a manner, breaking customer routers that adhere to the standards and locking customers into ISP gear that works around the non-standard behavior.

maverickws

@derelict The ISP you're talking about is actually the #1 ISP around here, with the biggest fibre optics infrastructure, most clients, both in the B2C as B2B.

They work with all kinds of OEMs: Cisco, Huawei, Ericsson/Alcatel and others.

The way they implement their solution worked before the update. It still works for all customers who do not use this solution, which will likely be over 95% of the customers. So yeah you can find it appalling, I'd say it isn't them (the ISP) but the OEM's yet many customers around the world will face these issues, but never enough to force them.

If I plug a Cisco where the pfSense is now, will IPv6 work? [YES, it does].

Now, what do you think it will happen? An infinitely small percentage of the customers will make the OEM's and the ISP's to change, or people will just move to something that works?

I believe you know the answer. It's not even a matter of personal opinion, it's a very simple business decision.

nighthawk1967

@maverickws This ipv6 problem started a few updates ago and before that everything worked for years now its just problems .

mhillmann

@maverickws
If you only define a fixed gateway and don't use DHCPv6 for prefix delegation too, the ISP's GW probably will answer pings from pfSense, but as it has no associaton between the delegated internal prefix and the routing address of the pfSense appliance it refuses to route anything else coming from the public address of your router. This is the whole point of keeping the DHCPV6 configuration (even if it doesn't work correctly): IA-NA to IA-PD binding. The routing infrastructure of your ISP must know which is YOUR local GUA that has the delegated prefix behind itself for the packets to get there...
This gets even worse if the particular ISP does not keep the delegated prefix fixed (as in our case) changing it randomly when our router issues a DHCP REFRESH message. To resolve this latter issue we use ULA + NPt for our internal network, with the added benefit of easy IPv6 gateway failover (we use several ISP's on the same router) without mandatory internal renumbering of hosts as the prefix changes. Such a configuration mimicks NAT of IPv4 but only changing the first 64 bits of your local host to the public delegated prefix on the pfSense router.

jimp

https://redmine.pfsense.org/issues/14072

maverickws

@jimp I really appreciate the report, thank you. You guys address the issue description way better than me.

mhillmann

@jimp We have another twist to the already acknowledged regression: our ISP refuses to delegate a prefix shorter than a /64, but grants as many(?) /64 prefix delegations as we request as long as the IA-PD identifier is unique. This gives roughly the same result as requesting a shorter PD and splitting it up on the track interface, but regrettably there is no way to inform unique IAID´s from the IA-PD request to individual track interfaces in order to create an association between the granted PD and the local interfaces onto which these PD´s should be assigned. The implied logic in pfSense is that we only get one PD of a certain length and associate it to the routing interface via its name, not the IAID. It might be useful to consider this use case too, as I have read somewhere that even ATT has this use policy.
The following should be possible and not restricted/blocked by the pfSense configuration (0 and 1 are the IAID's):
interface ix2 {
send ia-na 0;
send ia-pd 0;
send ia-pd 1;
script "/var/etc/dhcp6c_opt13_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
prefix-interface ix2.666 {
sla-id 0;
sla-len 0;
};
};
id-assoc pd 1 {
prefix-interface ix2.667 {
sla-id 0;
sla-len 0;
};
};

jimp

That is completely unrelated to the problem here and wouldn't have changed in 23.01. It belongs in a separate thread.

mhillmann

@jimp sorry, my mistake. I´ve started a new thread already.

maverickws

@jimp said in No IPv6 after upgrade to 23.01:

https://redmine.pfsense.org/issues/14072

I was thinking as a solution, what about having an option that when selected executes the script even if the ISP doesn't have a correct config, and when unselected has the supposed to be correct behaviour? Would that work?

mhillmann

@maverickws I believe this would be equivalent to always run the script, as it uses the same script for both flags already.

maverickws

@mhillmann yeah but I mean if we assume the current behaviour is the correct behaviour and by default should be so, it would make sense, imo, the option to override the default behaviour and then run the script regardless. (so when that option is selected, always run the script)

mhillmann

@maverickws I agree, this may be the best option.