IPv6 IP Alias prevents Track Interface from working with DHCPv6 and RA
-
@johnpoz said in Cannot Ping pfSense ULA From Subnet Without ULA Assigned to Firewall:
I noticed you have a /128 prefix - that is WRONG ;)
I have a /128 on my WAN interface. Works fine, including pinging. Packet Capture shows the pings coming from that /128 address.
-
Dude see the other thread your talking in - I hear you /128 is viable address for a tunnel and loopback - but its not really a global address mask.. Talk to your isp if you have problem with it, etc..
Min viable prefix in ipv6 is /64 --- your the ipv6 is better than sliced bread guy around here.. You should know this ;)
-
Either way, I still can't ping the firewall address regardless of the prefix length. 53::1 works fine, fd53::1 doesn't work - and the 53::1 address works whether the prefix is /64 or /128. I've tested from Ubuntu, Windows, and Android, to rule out an OS related issue, but they're all a no-go.
Is anyone able to test this from a interface IPv6 address set to Track WAN Prefix? That's the only different variable that I can see here. If not, I'll likely spin up a clean pfSense VM to see whether this is occurring on a fresh install. Is it possible that a plugin could be causing this, and if so, is there a consistent way to disable them without complete removal?
-
In case anyone encounters the same problem, I happened upon the solution by chance months later while configuring NTP on a multi-homed Ubuntu server and wanted to post it here for posterity's sake.
My main LAN subnet obtains an IPv6 address via WAN DHCPv6 prefix tracking and advertises it via RA's. Because most of my other VLAN's use ULA's and eschew dynamic GUA's, I have the firewall set up with static IPv6 addresses which get advertised to each subnet. For consistency's sake, I wanted devices on my main LAN to obtain a ULA as well as a GUA, and had configured the RA's to include a ULA prefix of fd00:172:28:198::/64. Unfortunately, since pfSense doesn't currently support multiple IPv6 prefixes per tracked interface ([https://redmine.pfsense.org/issues/5999]) this means that the client ping was going out through it's ULA, hitting it's destination, and then being routed back through the firewall. Since the firewall didn't itself have a ULA on the LAN interface, it had no route back to the client, and the traffic was dropped. Removing the ULA prefix from the RA's on that interface resolved this issue.
If anyone has a solution for allowing both static and dynamic IPv6 addresses on the same interface while avoiding the issues I encountered here, I'd appreciate the suggestions.
-
Unfortunately, since pfSense doesn't currently support multiple IPv6 prefixes per tracked interface
I am doing exactly this.
DHCP6 serves the delegated prefix based on track interface settings:
RA Serves BOTH the tracked interface prefix and the configured ULA:
ETA: Forgot about the VIPs:
My Mac's LAN interface looks like this:
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 options=3<RXCSUM,TXCSUM> ether a8:60:b6:19:15:fe inet6 fe80::14ea:9daa:af44:6b3d%vlan0 prefixlen 64 secured scopeid 0xb inet6 2600:beef:cafe:ab01:1092:8d9e:537d:5207 prefixlen 64 autoconf secured inet6 fd8c:9857:66db:1:4c9:e132:ee:396a prefixlen 64 autoconf secured inet 192.168.223.6 netmask 0xffffff00 broadcast 192.168.223.255 inet6 2600:beef:cafe:ab01::145a prefixlen 64 dynamic inet6 2600:beef:cafe:ab01:edf8:e8af:4a5c:da72 prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:edf8:e8af:4a5c:da72 prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:b95b:1aa2:a178:32c9 prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:b95b:1aa2:a178:32c9 prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:b0f0:e7fb:d671:c1e prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:b0f0:e7fb:d671:c1e prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:91f5:785f:2283:21fc prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:91f5:785f:2283:21fc prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:b1d6:45a3:c20:1d15 prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:b1d6:45a3:c20:1d15 prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:4c01:bf7b:6079:299b prefixlen 64 deprecated autoconf temporary inet6 fd8c:9857:66db:1:4c01:bf7b:6079:299b prefixlen 64 deprecated autoconf temporary inet6 2600:beef:cafe:ab01:347d:50ed:27e3:2de9 prefixlen 64 autoconf temporary inet6 fd8c:9857:66db:1:347d:50ed:27e3:2de9 prefixlen 64 autoconf temporary nd6 options=201<PERFORMNUD,DAD> vlan: 223 parent interface: en0 media: autoselect (1000baseT <full-duplex>) status: activeThe IPv6 stack in the Mac sources from a global address to global destinations and a ULA address to ULA destinations.
$ ping6 -c1 www.google.com PING6(56=40+8+8 bytes) 2600:beef:cafe:ab01:347d:50ed:27e3:2de9 --> 2607:f8b0:4005:80b::2004 16 bytes from 2607:f8b0:4005:80b::2004, icmp_seq=0 hlim=55 time=21.027 ms $ ping6 -c1 nc.mydomain.com PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> fd8c:9857:66db:4::c0a8:e012 16 bytes from fd8c:9857:66db:4::c0a8:e012, icmp_seq=0 hlim=63 time=0.695 msI use split DNS to give ULA addresses to inside queries. It's been working great. And now my inside network no longer breaks in the unlikely event Cox changes my PD.
One thing I have not tried is pinging ULA from an interface with only global addresses and pinging global from interfaces with only ULA addresses. A lot of the burden there is on the client's stack. Considering, with a /56, I don't have any problem assigning GUA and ULA to every IPv6 interface it has never come up. Let me try something real quick:
From LAN to the LAN GUA
$ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab01:208:a2ff:fe0a:593f PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab01:208:a2ff:fe0a:593f 16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=0 hlim=64 time=0.352 ms 16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=1 hlim=64 time=0.391 ms 16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=2 hlim=64 time=0.325 msFrom LAN to another interface's GUA:
$ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab02:208:a2ff:fe0a:593f PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab02:208:a2ff:fe0a:593f 16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=0 hlim=64 time=0.455 ms 16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=1 hlim=64 time=0.292 ms 16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=2 hlim=64 time=0.298 ms ^C --- 2600:beef:cafe:ab02:208:a2ff:fe0a:593f ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 0.292/0.348/0.455/0.075 msFrom LAN to a host on DMZ GUA:
$ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab04::c0a8:e012 PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab04::c0a8:e012 16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=0 hlim=63 time=0.775 ms 16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=1 hlim=63 time=0.612 ms 16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=2 hlim=63 time=0.629 ms ^C --- 2600:beef:cafe:ab04::c0a8:e012 ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 0.612/0.672/0.775/0.073 msSo it looks like it's all working as expected. There are GUA and ULA addresses on the configured interfaces.
Rebooting based on the trouble description in that redmine:
Yeah. it all just works here. GUA and ULA /64s on all interfaces configured as such. All the above pings still work. Seems fine.
For completeness, this is how I did one of the linux servers (nextcloud - accessible on GUA from the outside and ULA from the inside):
auto ens18 iface ens18 inet static address 192.168.224.18 netmask 255.255.255.0 gateway 192.168.224.1 dns-search example.com admin.example.com dns-nameservers 192.168.223.1 iface ens18 inet6 auto privext 2 accept-ra 2 up ip -6 addr add 2600:beef:cafe:ab04::192.168.224.18/64 dev $IFACE label $IFACE:0 down ip -6 addr del 2600:beef:cafe:ab04::192.168.224.18/64 dev $IFACE label $IFACE:0 up ip -6 addr add fd8c:9857:66db:4::192.168.224.18/64 dev $IFACE label $IFACE:0 down ip -6 addr del fd8c:9857:66db:4::192.168.224.18/64 dev $IFACE label $IFACE:0(Please do not use ULA prefix fd8c:9857:66db::/48. Please generate your own.)
-
Hi @Derelict,
Thank you for your extremely detailed reply! I apologize that it took me so long to try your suggested approach, but my wife and I brought home a baby girl around the same time that I original posted this, so I've been slightly distracted :D.
That said, I was able to replicate your configuration and unfortunately, I'm still encountering the same issue. Per the bug report, "This setup works perfectly until I reboot the pfSense machine or reset the LAN interface. This causes the IP Alias / CARP address to appear as the primary interface route and the tracked interface to appear as a secondary route (only visible by running netstat -nr). Both DHCPv6 and RA give out ULA routes only."
It's hard to tell what the differences might be between our setups, and I'm sure you're much more intimately familiar with the codebase than I am as a Netgate employee, but the bug report does indeed seem to be valid for at least some firewall configurations. Any idea what I might try to troubleshoot this, or should I just accept this as a bug until it's addressed? As you can imagine, it's difficult to develop an internally stable IPv6 infrastructure with this bug plus IPv6 NPT not tracking the dynamically assigned prefix for customers on dynamic IPs. It looks like Chris Linstruth also updated that bug (#5999) with some additional info since my last reply.
-
Yes. There is a problem there. I have been looking at a way to fix it but "I am not a programmer."
I only have ULA on two interfaces (LAN and DMZ where statically-addressed servers are) so when I reboot (which is rare) I:
Remove the VIPs
Edit/Save WAN
Add the VIPs back.And it's generally good until the next reboot which is generally weeks/months.
-
@chewie198 Just ran into this bug as well. Let me know if you need any info from my system.
-
@apearson I am not a Netgate developer, so it's not entirely up to me whether or not this gets fixed. That said, I am a software developer and I do have a vested interest in seeing this issue get resolved. If enough people are interested in seeing this fixed and other independent developers are willing to lend a hand, we may be able to submit a pull request in an attempt to resolve the issue.
I suspect it will be an uphill fight since pfSense's paying customers are largely businesses and this issue would not affect them because a business would almost certainly be using static IP ranges with either NPT or advertising their own PI address space via BGP. Nevertheless, I have added a link to this thread in the comments for https://redmine.pfsense.org/issues/5999 in case other users or developers encounter this issue and are willing to help out in some way.
I'm happy to help you troubleshoot if you need help working around the problem. Just know that you can only host internal services via IPv4 addresses for the time being so long as you have a dynamic public IP and expect your routes to persist after a reboot (which is probably the majority of individual end users).
-
I'm trying to understand the problem here, as the thread goes back quite a while and I've forgotten the details. Is the problem that ULA addresses are not working properly? I have both ULA and GUA addresses on my network and the local DNS points to the ULA address, not GUA. This means that even if my GUA prefix should change, my local network will keep on working fine. I use tracking for my GUA prefix and configured the ULA prefix on the Router Advertisements page.
Dude see the other thread your talking in - I hear you /128 is viable address for a tunnel and loopback - but its not really a global address mask.. Talk to your isp if you have problem with it, etc..
Min viable prefix in ipv6 is /64 --- your the ipv6 is better than sliced bread guy around here.. You should know this ;)
I don't see that I replied to this, so here it goes. Yes, I have a /128 for my WAN address, but the only /64 on my WAN interface is for my link local address. On IPv6, link local addresses are often used for routing.
-
@JKnott The original problem was that I couldn't ping between VLANs using ULA's because I didn't have the ULA prefix in my router advertisements so there was no route back to the source. Once I added the prefix then I was able to ping successfully. That was just an overlooked mistake on my end.
However, once that was rectified the problem became that, per the bug report, "This setup works perfectly until I reboot the pfSense machine or reset the LAN interface. This causes the IP Alias / CARP address to appear as the primary interface route and the tracked interface to appear as a secondary route (only visible by running netstat -nr). Both DHCPv6 and RA give out ULA routes only."
Like you, I use tracking for my GUA prefix and configured the ULA prefix on the Router Advertisements page. As you can imagine, it's difficult to maintain global IPv6 connectivity without a globally unique prefix/address being handed out via DHCP or RA's.
Does your GUA prefix continue to be advertised after a reboot? If so, it would appear that the behavior is inconsistent. In my case, @Derelict's case, and others, we're seeing the VIP address "take over" the interface for DHCP and RA purposes.
-
@chewie198 said in Cannot Ping pfSense ULA From Subnet Without ULA Assigned to Firewall:
Does your GUA prefix continue to be advertised after a reboot? If so, it would appear that the behavior is inconsistent. In my case, @Derelict's case, and others, we're seeing the VIP address "take over" the interface for DHCP and RA purposes.
Yes, both my GUA and ULA prefixes remain after rebooting. The only problem I've experienced was when I first started using pfSense was my GUA prefix would change even for something as simple as disconnecting/reconnecting the Ethernet cable. However, a setting was added, so that pfSense would retain the existing prefix.
Here's the relevant part of a recent RA:
ICMPv6 Option (Prefix information : fd48:1a37:2160::/64)
Type: Prefix information (3)
Length: 4 (32 bytes)
Prefix Length: 64
Flag: 0xe0, On-link flag(L), Autonomous address-configuration flag(A), Router address flag(R)
Valid Lifetime: 86400
Preferred Lifetime: 14400
Reserved
Prefix: fd48:1a37:2160::And for the GUA:
ICMPv6 Option (Prefix information : 2607:fea8:abcd:dcba::/64)
Type: Prefix information (3)
Length: 4 (32 bytes)
Prefix Length: 64
Flag: 0xe0, On-link flag(L), Autonomous address-configuration flag(A), Router address flag(R)
Valid Lifetime: 86400
Preferred Lifetime: 14400
Reserved
Prefix: 2607:fea8:abcd:dcba::(Prefix has been changed to protect the guilty)
-
@JKnott As I haven't reviewed the code, I can't tell whether the behavior is merely nondeterministic or whether there's something specific to your setup that is causing it to consistently advertise the correct prefix. Given that Chris Buechler commented on the Redmine ticket that "find_interface_ipv6 returns the first IP on the interface in that case, which might not be the intended one in that situation," I would speculate that the first address in the list is random and you've just gotten lucky. Would you be willing to reboot 4-5 times in succession and confirm that ifconfig is showing the GUA first each time? If so, maybe there's something helpful we could derive from your configuration as a workaround.
I'm still concerned, however, that unless the code is specifically written to use the GUA as the basis for DHCP and RA advertisements, achieving the correct result is just going to be a lucky break based off of implementation details. The best case scenario, I think, would be to provide a dropdown for the user to select what the primary address prefix should be for DHCP solicitations. Are you seeing the correct GUA prefix for DHCP as well, or are you only using RA's? I didn't see you mention DHCPv6 in your last reply.
-
@chewie198 I believe the "quick fix" is to have a checkbox somewhere that says "ignore ULA" and, somewhere in the code (ianap) honor that as the addresses on each interface are looped through.
A more-perfect solution is likely much more complicated.
-
@Derelict I agree, solving this thoroughly is not easy at all. If it were, it likely would have been done already. Right now, these sections of code have been written with the assumption that there will only be one primary IP address per interface, and with IPv6, that is no longer the case. That means we now have to maintain a list of all the addresses on a given interface and modify the UI to reflect a separate list of settings for each one, or at the very least, to allow for choosing a primary. That's not a trivial amount of work.
I like your quick fix idea though because that would almost certainly be less code to write and test and it should cover a basic use case for a large percentage of end-user configurations. Then we could choose whether to allocate DHCP addresses under the GUA or ULA ranges per user preference. Given @JKnott's reply, maybe we're already in the clear regarding RA's. I don't recall seeing this behavior the last time I assigned a VIP, but I can retest if we're getting conflicting reports.
-
I've updated the thread title to better reflect the current discussion.
-
@chewie198 Was there ever a bug opened in Redmine for this?
-
@IsaacFL Yes, https://redmine.pfsense.org/issues/5999 was opened several years ago but the development work is still pending. @jimp mentioned that the fix was difficult and that they would be willing to accept a pull request to fix the issue, but i haven't seen much interest from other developers or users in trying to resolve this - at least not on this particular thread or within the issue comments. If you're seriously interested in resolving this then I would suggest either starting a bounty or, if you're a developer, teaming up to tackle this, preferably with some feedback from Netgate as they would be the ones approving the pull requests.
-
@chewie198 Thanks, I was just wanting to see what had happened since I do not use ULA myself.
I had through some of my own testing found that what is happening, is that during the boot process the radvd.conf is being incorrectly configured whenever a ipv6 VIP is present.
Looking at the Issue 5999, not sure I would have any additional information since interest doesn't seem to be there.
-
I've got the exact same issue as this, but it's described perfectly in this thread, not mine.
Hoping that this bug gets some priority.
@JKnott and I have been bantering back and fourth on my thread, which was originally about Apple TV's taking too much NDP space, which has evolved into a thrilling tale of IPv6 ULA's and WAN track interfaces.
Anyway... I've got to delete the ULA IPv6 VIP every time I pull the WAN interface, push it up/down, or reboot the box, because the VIP takes over.
I think the answer would be some sort of interface priority on the LAN for which address is primary (GUA or ULA) or something, but I am also not a programmer. It just seems like pfSense doesn't appropriately deal with multiple addresses correctly on IPv6.
I also get some funky activity with some clients not pinging to other clients correctly, which is likely due to the client stacks (Mac, Windows, pihole, DNScrypt, etc). Windows seems to ALWAYS work for pings, regardless what I do (Mac not so much). But it's likely again due to priority in which interface (GUA or ULA) is sending the ICMP packets, or possibly which it get's first or something.
Anyway - if any diagnostics are required from my end or anything let me know... happy to help, but can't really contribute much more not being a programmer. Just wanted to add that it would be great to see the IPv6 multiple-addresses side of things ramped up in pfSense. Once it's adopted more and more I am sure all of the OS stacks will catch up.
Hoping maybe the Netgate folks get to it in a future release... properly getting track interface to work with multiple IP addresses on a LAN interface including GUA and ULA. Definitely some funky routing and "which interface gets priority or sends the traffic and can route" going on... both on the pfSense side (which they can control), as well as the various client OS's (Windows, Mac, Linux, etc). All of them do it differently. Windows machines here always seem to ping everything just fine... Mac's not so much.
If anyone finds a fix / workaround (possibly a script to pull and add the ULA VIP after 5-10 seconds whenever the WAN goes up/down)... let me know and I'd be happy to test it.
Best Regards,
dg6464
