No IPv6 after upgrade to 23.01
-
I'm a Verizon FIOS customer for home internet. Like many others, IPv6 stopped working after the upgrade to 23.01. I tried a great many things to get IPv6 working again after the upgrade to 23.01. While I can't possibly recount all the things, I will share the last thing I did that resulted in delegated IPv6 being reassigned to my LAN network.
From the pfSense console I used the 2) Set interface(s) IP address option. Both IPv4 and IPv6 were again set to DHCP. (What I would consider to be no change.)
When the console menu came back up, my IPv6 was present on LAN as I had previous experienced before the upgrade.
Hope this helps someone else.
-
@forresgeek Hi
I tried this: set my MTU to 1480, but unfortunately didn't change the situation.
-
For what it's worth. We've found a permanent solution for our issue for now (meaning no M+O flags set in RA announcements), testing it for long term validity:
We configure the interface as static with the fixed GUA address next hop gateway extracted from Wireshark capture. Then we change the IPV6 interface from static to dynamic and don't delete the static gateway (in ROUTING tab) to use it as default gateway for the interface. Rtsold() doesn't deliver any gateway for the interface BUT we've got one from the static definition, so we can route IPV6 packtes normally. As the unassigned dynamic gateway doesn't interfere with routing but dhcp6c() renews NA+PD delegations on expiration, we think we've got a stable configuration that will endure over time as long as the defined (fixed) next hop gateway with a GUA address stays valid, as it apperently does.
I acknowledge that this is a VERY nonstandard configuration we get from our ISP, but apparently their whole network is configured this way and their own (mainly Huawei) routers respect this arrangement when set to router mode as configured by default, delivering a correctly working IPV6 network on the internal side, RA-wise, meaning the RA sets the M+O flags as the RFC demands and pfSense can work as intended. The whole issue only becomes relevant if trying to use a different router, needing the ONT to be put into bridge mode.
Nonetheless it would be usefull to acknowledge this use case and see if is possible to force a fixed gateway even with DHCPv6. -
@mhillmann hi there,
based on your answer I thought of applying this to our setup and see how it would go.
The approach was slightly different, before when working I knew what was the GW address assigned to the IPv6.
So I simply added a new gateway on the routing tab, used that GW address and selected it as gateway.
So this did change something as now I can ping google ipv6 from Diagnostics > Ping
16 bytes from 2a00:1450:4003:806::200e, icmp_seq=0 hlim=59 time=15.372 ms 16 bytes from 2a00:1450:4003:806::200e, icmp_seq=1 hlim=59 time=15.177 ms 16 bytes from 2a00:1450:4003:806::200e, icmp_seq=2 hlim=59 time=15.382 ms 16 bytes from 2a00:1450:4003:806::200e, icmp_seq=3 hlim=59 time=22.012 ms 16 bytes from 2a00:1450:4003:806::200e, icmp_seq=4 hlim=59 time=15.506 ms --- ipv6.l.google.com ping6 statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 15.328/18.726/30.600/5.971 ms
My issue now is that, for some reason, I don't get IPv6 connectivity on local machines. I get IPv6 address but no route.
On my machine
netstat
gives me a default v6 gateway which isfe80::abcd:ef01:fe03:bd10
being that this is my pfSense LAN interface local-link address. So I can't really figure out now why I have connectivity on the pfSense LAN interface but not on local hosts.If I ping that GW my machine has configured:
# ping6 fe80::abcd:ef01:fe03:bd10%en0 PING6(56=40+8+8 bytes) fe80::8af:1f71:bffc:afc1%en0 --> fe80::abcd:ef01:fe03:bd10%en0 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=0 hlim=64 time=1.323 ms 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=1 hlim=64 time=1.381 ms 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=2 hlim=64 time=1.302 ms 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=3 hlim=64 time=1.269 ms 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=4 hlim=64 time=2.170 ms 16 bytes from fe80::abcd:ef01:fe03:bd10%en0, icmp_seq=5 hlim=64 time=2.981 ms ^C --- fe80::abcd:ef01:fe03:bd10%en0 ping6 statistics --- 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 1.269/1.738/2.981/0.638 ms
EDIT: This is a temporary (not totally working still) fix not a solution.
DHCP6 should take care of this without the need of having to find out the GW for the next hop and adding static gateways.I am appalled how no one from Netgate is getting involved in this. This was a breaking change as MANY ISP's (and I mean many) will have similar setups and hardware.
Sweep it under the rug, don't talk about it, and it's gone, is that it? -
@maverickws Is the mask correct? There was another post recently where the person was saying devices were getting a /128 mask not /64.
IPv6 is working fine here.
-
@steveits I'm attributed a /56 prefix;
WAN configured as DHCP6
LAN, IoT and DOM (all local networks, IoT and DOM are VLANS) configured as tracking WAN interface for IPv6Each interface gets a /64 prefix delegation size.
This is the same configuration (exactly the same configuration) that has been working for years with IPv6.
The IPv6 addresses are correctly attributed to all the internal interfaces. Actually, on the above test (after creating a static gateway as suggested) pinging
ipv6.google.com
you can see the pfSense pinging from its IPv6 LAN address.I would refer to Mr. @mhillmann 's post here: https://forum.netgate.com/post/1088842
As this seems the most on point explanation of what might be happening.
-
@maverickws If it can be replicated (seems so), I suggest opening an issue at redmine.pfsense.org, with the details and any logs, referencing this thread.
And/or ask the ISP, if they are not following an RFC. Is everyone with this issue using the same ISP?
23.01 skipped from FreeBSD 12 to 14 so it's likely a lot of internal things changed.
-
@maverickws said in No IPv6 after upgrade to 23.01:
DHCP6 should take care of this without the need of having to find out the GW for the next hop and adding static gateways.
IPv6 does not get its default next-hop from DHCP[6] like IPv4 does. It relies on router advertisements. To reiterate, there is nothing built into DHCP6 to propagate a next-hop gateway. It must be in an acceptable router advertisement.
If the router advertisements are non-standard then the problem is at the ISP.
I, personally, find it pretty appalling that an ISP would deploy IPv6 in such a manner, breaking customer routers that adhere to the standards and locking customers into ISP gear that works around the non-standard behavior.
-
@derelict The ISP you're talking about is actually the #1 ISP around here, with the biggest fibre optics infrastructure, most clients, both in the B2C as B2B.
They work with all kinds of OEMs: Cisco, Huawei, Ericsson/Alcatel and others.
The way they implement their solution worked before the update. It still works for all customers who do not use this solution, which will likely be over 95% of the customers. So yeah you can find it appalling, I'd say it isn't them (the ISP) but the OEM's yet many customers around the world will face these issues, but never enough to force them.
If I plug a Cisco where the pfSense is now, will IPv6 work? [YES, it does].
Now, what do you think it will happen? An infinitely small percentage of the customers will make the OEM's and the ISP's to change, or people will just move to something that works?
I believe you know the answer. It's not even a matter of personal opinion, it's a very simple business decision.
-
@maverickws This ipv6 problem started a few updates ago and before that everything worked for years now its just problems .
-
@maverickws
If you only define a fixed gateway and don't use DHCPv6 for prefix delegation too, the ISP's GW probably will answer pings from pfSense, but as it has no associaton between the delegated internal prefix and the routing address of the pfSense appliance it refuses to route anything else coming from the public address of your router. This is the whole point of keeping the DHCPV6 configuration (even if it doesn't work correctly): IA-NA to IA-PD binding. The routing infrastructure of your ISP must know which is YOUR local GUA that has the delegated prefix behind itself for the packets to get there...
This gets even worse if the particular ISP does not keep the delegated prefix fixed (as in our case) changing it randomly when our router issues a DHCP REFRESH message. To resolve this latter issue we use ULA + NPt for our internal network, with the added benefit of easy IPv6 gateway failover (we use several ISP's on the same router) without mandatory internal renumbering of hosts as the prefix changes. Such a configuration mimicks NAT of IPv4 but only changing the first 64 bits of your local host to the public delegated prefix on the pfSense router. -
-
-
@jimp I really appreciate the report, thank you. You guys address the issue description way better than me.
-
@jimp We have another twist to the already acknowledged regression: our ISP refuses to delegate a prefix shorter than a /64, but grants as many(?) /64 prefix delegations as we request as long as the IA-PD identifier is unique. This gives roughly the same result as requesting a shorter PD and splitting it up on the track interface, but regrettably there is no way to inform unique IAIDยดs from the IA-PD request to individual track interfaces in order to create an association between the granted PD and the local interfaces onto which these PDยดs should be assigned. The implied logic in pfSense is that we only get one PD of a certain length and associate it to the routing interface via its name, not the IAID. It might be useful to consider this use case too, as I have read somewhere that even ATT has this use policy.
The following should be possible and not restricted/blocked by the pfSense configuration (0 and 1 are the IAID's):
interface ix2 {
send ia-na 0;
send ia-pd 0;
send ia-pd 1;
script "/var/etc/dhcp6c_opt13_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
prefix-interface ix2.666 {
sla-id 0;
sla-len 0;
};
};
id-assoc pd 1 {
prefix-interface ix2.667 {
sla-id 0;
sla-len 0;
};
}; -
That is completely unrelated to the problem here and wouldn't have changed in 23.01. It belongs in a separate thread.
-
@jimp sorry, my mistake. Iยดve started a new thread already.
-
@jimp said in No IPv6 after upgrade to 23.01:
I was thinking as a solution, what about having an option that when selected executes the script even if the ISP doesn't have a correct config, and when unselected has the supposed to be correct behaviour? Would that work?
-
@maverickws I believe this would be equivalent to always run the script, as it uses the same script for both flags already.
-
@mhillmann yeah but I mean if we assume the current behaviour is the correct behaviour and by default should be so, it would make sense, imo, the option to override the default behaviour and then run the script regardless. (so when that option is selected, always run the script)
-
@maverickws I agree, this may be the best option.
-
@maverickws said in No IPv6 after upgrade to 23.01:
https://forum.netgate.com/post/1088842
Well, it took me 22 days to realize that ipv6 is no longer working since upgrading.
It seems that a problem has been identified. Has a workaround been proposed? All of my google home devices (which use SLAAC) are no longer working reliably. If I turn off IPV6 on that VLAN, it works.
Thanks,
Devan
-
Take with a grain of salt... I was suffering from this issue and enabled the DHCPv6 server on my LAN interface and that seemed to fix it.
I thought I had the DHCPv6 server enabled on my LAN interface before upgrading to 23.01, but I found it not enabled when troubleshooting this issue.
Just thought I would share in case this helps anyone.
-
@jpwoodbu
Thanks.I checked on DSL reports and a few NJ users are reporting IPv6 outages. I just removed all traces of IPv6 from pfsense and called it a day. I'll try again in a year. IPv6 really didn't offer any noticeable benefits but introduced much more instability, mostly from Verizon.
-
@ddbnj Its not just NJ FiOS users. I'm in NoVA and IPv6 isn't working for me either.
-
I will say it's not your Isp because i can use the ips router and ipv6 works .Pfsense version 22 works fine just can't install packages because the package manager is out of date ,So is there going to going to be fix any time this year ?
The no help is not good for anyone
The ipv6 problem is not funny !! -
@nighthawk1967 I disagree with that. I have a Mikrotik on another FiOS network that should support IPv6. I'm pretty sure that's doesn't have ipv6 working either...
-
@compuguy said in No IPv6 after upgrade to 23.01:
@ddbnj Its not just NJ FiOS users. I'm in NoVA and IPv6 isn't working for me either.
Yep... after some overnight maintenance early Tuesday morning, I've lost my IPv6 connectivity in Northern VA. Clearly Verizon has changed something, but it sounds like pfSense may need to change something also... looking forward to whenever the patch is available for the fix!!
-
@compuguy said in No IPv6 after upgrade to 23.01:
@nighthawk1967
I disagree with that. I have a Mikrotik on another FiOS network that should support IPv6. I'm pretty sure that's doesn't have ipv6 working either...Update: Looks like the Mikrotik router is having no issues and is able to pull a IPv6 prefix from Verizon. This seems to be a PFsense 23.01 bug/issue only....
-
@compuguy said in No IPv6 after upgrade to 23.01:
This seems to be a PFsense 23.01 bug/issue only...
My setup / background :
Before January 2023, my ISP - my Internet connection was just IPv4. I was a static IP, so that's fine to me.
But no IPv6what so ever.
So, around 2014 ( ? ) I decided to create a he.net tunnelbroker account, as they give a free /48 for "live" with local access points all over the planet.
I tend to think these guys somewhat created the IPv6 RFCs.January 2023 : fiber was installed. That was the end of 24 Mbits/sec VDSL : here comes 800+ Mbits up/down.
And a new fiber ONT integrated ISP router.
That didn't route the '6in4' protocol, so that was the end of my he.net free IPv6 access.
The positive side of this was that their local POP in paris, for me, would not route that kind of bandwith to me, so their Ipv6 would cripple my overall Internet access.
And the new USP box offers IPv6 for it's LAN devices.
And even better : it hands over /64 prefixs (delegations) to router type LAN (ISP router point of view) devices, like pfSense.And yes : my pfSense DHCP6 client does receive a (one) /64 for my pfSense LAN.
Not really perfect yet, as I have several LAN using IPv6 and my ISP router is buggy : it can gve more prefixes (64) but it then fails to route over IPv6. So only one it will be.I had some insight in what actually happens between my ISP router and pfSense with some severe tcpdump sessions.
Then some one one a french forum ( ! ) pointed me to Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5 so I crafted my own dhcp6 client config file, put the dhcp6 client into debug mode, and saw what happened written on the screen (logs).
Ok, nice, the router is somewhat buggy. The ISP made huge progress : a /56 for every user, and prefixes are handed over to devices that asks for one, or more of them.
But : not perfect yet, for me.Of course, in France, about 16 million users use the same ISP (Orange) and 10 million have fiber. And I'm the only one using multiple IPv6 LANs ..... ?
Now, to join the subject of this thread : our ISPs are not perfect yet.
But I use 23.01 on a 4100, and IPv6 works for me, Phones, PC's, printers etc do get a routable IPv6 from pfSense. I'm not using the tracker mode, but the DHCP6 server with static IPv6. -
@maverickws said in No IPv6 after upgrade to 23.01:
@steveits I'm attributed a /56 prefix;
WAN configured as DHCP6
LAN, IoT and DOM (all local networks, IoT and DOM are VLANS) configured as tracking WAN interface for IPv6Each interface gets a /64 prefix delegation size.
How did you get all the local networks tracking the LAN? I can get one network doing this but when I try to setup another network/VLAN I get an error that tracking is already established and in use (or something along these lines).
-
@jasonreg You have to use another prefix ID. And they don't track LAN but your WAN.
-
This :
doesn't mean you will get more then one prefix.
--- I think ---With that (image) setting, I get 1 prefix, so I can select one (from 0 to 0) as an interface to track (on one LAN interface).
I wanted to have several prefixes, so I used the forum thread sited above as a guideline.
I created this file /root/att-rg-dhcpv6-pd.conf :
interface ix3 { send ia-na 0; send ia-pd 0; send ia-pd 1; send ia-pd 2; send ia-pd 3; send ia-pd 4; send ia-pd 5; send ia-pd 6; send ia-pd 7; request domain-name-servers; request domain-name; script "/var/etc/dhcp6c_wan_script.sh"; }; id-assoc na 0 { }; id-assoc pd 0 { prefix-interface igc0 { sla-id 0; sla-len 0; }; }; id-assoc pd 1 { prefix-interface igc2 { sla-id 0; sla-len 0; }; }; id-assoc pd 2 { }; id-assoc pd 3 { }; id-assoc pd 4 { }; id-assoc pd 5 { }; id-assoc pd 6 { }; id-assoc pd 7 { };
where ix3 is my WAN interface.
igc0 is my LAN interface
igc1 is my second LAN interfaceI've used this file like this :
A then, looking at thedhcp6client log lines, the magic happend :
I saw 2 /64 prefixes handed over buy my ISP upstream router, and could use Tracking on two LAN interfaces, the first had the index 0 and the other the index 1.
( and then things still failed for me, but that's another story )Btw : I do not pretend that I fully understand what the config file does.
-
@bob-dig said in No IPv6 after upgrade to 23.01:
@jasonreg You have to use another prefix ID. And they don't track LAN but your WAN.
Yes, sorry typo. Is it as simple as changing the default "0" here:
to a "1" for a separate VLAN or network? -
@jasonreg
Exact !
When tracking works (aka : you received on or more prefixes), that is how it should be set up. -
@jasonreg 0 to ff in your case, look at the screen(shot).
-
@bob-dig said in No IPv6 after upgrade to 23.01:
@jasonreg 0 to ff in your case, look at the screen(shot).
OK, so I am a bit lacking here. "0 to ff" would mean what exactly for the next 5 interfaces? Apologize for the (probably) basic question ...
And, do I enable the DHCP6 server and RA on each interface or on just the LAN?
-
@jasonreg OK - have that one. numbers until I needed 10 or more than I need to use hex. Got it.
So I now have IPV6 on all networks. The only thing I do not understand is why the WAN_DHCP6 Gateway shows Red "Offline, Packetloss"
That said, it looks like the IPV6 addresses are now being handed out. Verified on iPad/iPhones etc. changing to the various VLANs.
-
@jasonreg
Sometimes after doing a lot of configuration changes, giving a reboot on the firewall gets everything running perfectly at last.
It could also mean your IPv6 upstream gateway doesn't respond to monitoring. In that case you either mark the GW always up, or get a valid IPv6 address for the Gateway monitor to reach and define its state.About my previous attempts to figure this out and following up on @mhillmann's conversation;
I have data from over a year back where I can see my IPv6 gateway with this ISP. My IPv6 address is always one of a /40 pool, and the upstream gateway is always the same. I checked this extensively.
So the WAN interface was using DHCPv6 but in Routing the selected IPv6 gateway was the static one I created. I actually went a step further and disabled the dynamic gateway on the routing table.
So the static gateway was up but no traffic from the machines which had an IPv6 correctly attributed by tracking WAN interface setup.
So I left this at a standstill waiting for a fix to the issue here. I resigned to "no IPv6" for a while.
The other day I opened a site which accused me of being using IPv6.Made an IPv6 test: IPv6 working.
Tested on other machines: IPv6 working.Checked the firewall, have the Static IPv6 gateway selected, the same configuration I left before when IPv6 wasn't working. And wasn't for many days.
Currently 17 days uptime, have no idea how the issue resolved by itself.
So in the meanwhile, tried just rebooting the firewall to see how'd it would go.
After reboot, no IPv6 again. -
Thanks for the redmine report, I'll disable IPv6 until a fix is released.
-
@shadowking
Yes, thank you everyone for the conversation. I didn't realize this thread existed and had created my own which appears to be a result of the same issue. I'm gonna disable ipV6 until we have a resolution. -
-
-
-
-