ipv6: can ping GUA address in different VLAN, but not ULA.

Tanya

Hi,

As the title says: I have trouble reaching ULA addresses on other VLANs.

Short version:

Test machine 1 can ping Test machine 2 (same VLAN) on its GUA and ULA.
Test machine 1 can ping Test machine 3 (different VLAN) on its GUA, but NOT on its ULA.

AFAIK I have made no changes to the firewall rules that explain this behaviour.

In a further test I have created an additional (dummy) VLAN with a fixed IPv6 on its interface.
If that fixed IPv6 is one from the global range Test Machine 1 (on other VLAN) can ping it. If I change the address to a ULA, it can no longer ping it.

I would expect ULA addresses to reachable.
Do I misunderstand something, or is this a bug, or an error in my configuration?

For details on what, why and how, please see my next post.

Tanya

Long version with all details:

pfSense version: 2.4.3-RELEASE-p1 (amd64)
built on Thu May 10 15:02:52 CDT 2018
FreeBSD 11.1-RELEASE-p10

I have set up a few VLANs on our pfSense (= also main router) and a few virtual machine to prepare for future rollout of ipv6.

I get a 56bit prefix from my ISP and use track interface on VLAN20 and VLAN40. I use the VLAN number as ipv6 prefix ID.
Apart from that I added an extra subnet (ULA) to the RA for both VLANs.
For the ULA I used a tool to generate a 48bit part and added the VLAN number again to complete the 64bits.

[The reason I also want the ULA addresses is that I want to use static ULAs for our servers.
We will probably move to another building (first) and change ISP (later, when fiber is available) in the near future, so probably that means at least one and
maybe even 2 changes to our prefix - and thus GUA addresses. Hence, ULA seems the safest option.]

The effect is that the clients get 6 IP-addresses. For example, a VM in VLAN20:

ipv4 = 192.168.20.101
ipv6 GUA = 2a02:xxxx:yyyy:zz20:743b:f963:8017:9a63
ipv6 ULA = fdfd:d8e5:340d:0020:743b:f963:8017:9a63
ipv6 LL = fe80::743b:f963:8017:9a63
ipv6 GUA temporary = 2a02:xxxx:yyyy:zz20:254e:c44a:cdbb:64c9
ipv6 ULA temporary = fdfd:d8e5:340d:0020:254e:c44a:cdbb:64c9

All my test VMs are Windows 7 Pro machines.
I have disabled the Windows Firewall on all test machines to make sure that doesn't affect testing.

A Test Machine on VLAN20 can ping another test machine on VLAN20 on all 6 IP addresses.
Likewise a test machine on VLAN40 can ping another test machine on VLAN40 on all 6 IP addresses.

However, when I ping from a test machine on VLAN20 to a test machine on VLAN40 I do get some unexpected results.

ping ipv4 = success
ping ipv6 GUA = success
ping ipv6 ULA = failed [unexpected]
ping ipv6 LL = failed [expected]
ping ipv6 GUA temporary = success
ping ipv6 ULA temporary = failed [unexpected]

While the LL failure is of course totally expected, I'm surprised the ULA addresses don't work.
If I understand correctly, ULA address should be accessible on other VLANs, but not pass to the outside world.

I have not blocked anything in the firewall rules on VLAN20.
I tried to explicitly allow anything ipv6 related (added the middle rule) but that made no difference.

There are a few rules on VLAN40, but only IPV4-related.

Tanya

For a further test, I added a extra VLAN (70). There are no machines in this VLAN.
Instead of Track Interface, I have given it a static IP address.

I gave it a GUA-style address, 1111:2222:3333:4444::1

I can ping it from my Test Machine in VLAN20.
I cannot ping it from my Test Machine in VLAN 40 [that's normal I guess as VLAN40 as no ipv6 rules]

Next I change the static IP address to an ULA-style address fdfd:d8e5:340d:0070::1

I can no longer ping it from my Test Machine in VLAN20.

That seems to be the same effect as pinging an ULA client address in other VLANs.
Maybe it has the same cause too?

bug?
misconfiguration?
... ?

chpalmer

Being Im not anywhere close to knowledgeable with IPv6 I had a theory and it seems to be true..

From https://en.wikipedia.org/wiki/Unique_local_address

A unique local address (ULA) is an Internet Protocol version 6 (IPv6) address in the address range fc00::/7. ... They are routable only within the scope of such private networks, but not in the global IPv6 Internet.

I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

I may be totally wrong but it may give you a clue or two to look at.. :)

Tanya

I must admit that is a bit confusing.

I was under the impression that ULA is the equivalent of the IPv4 private addresses (192.168.x.x, 10.x.x.x) and that Link Local (fe80::xxx) was the equivalent of APIPA (Automatic Private IP, i.e. 169.254.x.x)

The main difference between ULA and Link Local is - as far as I understood - that ULA could be routed (locally) and LL could not.

That is how I interpreted what I have read about it, but of course I may be wrong, or my source may have had it wrong.

I can't find the docs I based my interpretations on at the moment. I'll try to find it back and have a closer look at it.

chpalmer

Yep-true.. IPv4 local addresses are in deed routable internally and one would think that there would be a similar IPv6 solution.

This page wants you to subscribe but you can copy and paste..
https://community.infoblox.com/t5/IPv6-CoE-Blog/3-Ways-to-Ruin-Your-Future-Network-with-IPv6-Unique-Local/ba-p/5663

http://biplane.com.au/blog/?p=12

Looks like your right according to these. IMO it should be routable accross the platform.

JKnott

@chpalmer said in ipv6: can ping GUA address in different VLAN, but not ULA.:

I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

That is incorrect. They are routeable, but not over the public Internet. They're used just like the RFC1918 addresses on IPv4.

chpalmer

@jknott said in ipv6: can ping GUA address in different VLAN, but not ULA.:

@chpalmer said in ipv6: can ping GUA address in different VLAN, but not ULA.:

I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

That is incorrect. They are routeable, but not over the public Internet. They're used just like the RFC1918 addresses on IPv4.

Yep- like I said in my last post..
Looks like your right according to these. IMO it should be routable accross the platform.

Tanya

OK, so basically we all agree and think it should work.
That's good, that would mean that the concept of what I'm trying to achieve seems OK.

The bad thing is that - currently - it does not work.
No idea yet if that is caused my a bug, a misconfiguration on my part (missing or wrong firewall rules?) or both?

Does anyone have a clue?

awebster

Make sure that these aren't checked in the Interface configuration each VLAN.
0_1531495088117_3b804dff-bc8a-4ef2-beee-16fef5b07464-image.png

Tanya

@awebster

Thanks for the suggestion.
I'll check that when I'm back at work on Monday (it's 18:20 over here, working day is done :) )

I'm quite convinced the first option is unchecked, otherwise the ipv4 ping 192.168.x.x would't pass either (it does).
I'm not so sure about the second option. If the default setting is OFF then it is probably still off. I can't remember turning it on, anyway.

But both need doublechecking on Monday. I'll get back with the results when that's done.

Tanya

@awebster said in ipv6: can ping GUA address in different VLAN, but not ULA.:

Make sure that these aren't checked in the Interface configuration each VLAN.

I have just double-checked and can confirm that both options are unchecked on all of the VLAN interfaces

xianic

I was writing a post asking for help with the same think but I thing though I might have solved it. The setup I have is:

GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.
A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11
fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface
A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

Then I can ping the host on VLAN11 from a host on VLAN10. I'm having trouble allowing TCP through at the moment, some more debugging will be required there. Hopefully this is helpful to you.

Tanya

@xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

Then I can ping the host on VLAN11 from a host on VLAN10. I'm having trouble allowing TCP through at the moment, some more debugging will be required there. Hopefully this is helpful to you.

I'm certainly going to check that firewall rule and try if I can get something done that way.
If I understand correctly, the rule you added grants access from (all) your machines on VLAN10 to a single machine on VLAN11?

In my situation, I would need all my machines from the first VLAN to be able to access all of the hosts in the 2nd VLAN. Vice versa, all machines in VLAN2 should only be able to reach a very limited number of machines in VLAN1.

That's how the ip4-equivalent is set up.

@xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.

A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11

fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface

Do the Virtual IP and the static GUA still work after you reboot pfsense?
IIRC there is a problem with track interface and virtual IPs where after the reboot the interface only has the virtual IP and no longer the GUA.
That's something I read on the forums and also experienced as such when I was seeing if a virtual IP could help me out with this problem.

Anyhow, you gave me a few things to try. I'll get back with the results soon.

Tanya

To do some further testing, I have made a setup with 3 virtual machines

a new pfSense (version 2.4.3)
a Windows client (Win7, firewall disabled)
a 2nd Windows client (Win7, firewall disabled)

These 3 are totally isolated from the rest of my network.
I tried to setup pfSense with as little changes as possible (no VPN, ...), just the ipv6 stuff.

The WAN is set to auto (DHCP) but not connected (so track interface won't work)
LAN with Static IPv6: fddd:666:666:90::1
VLAN91 with Static IPv6: fddd:666:666:91::1
VLAN92 with Static IPv6: fddd:666:666:92::1
LAN: RA = assisted
VLAN92: RA = Stateless DHCP

I put 1 Win client on LAN and one on VLAN92.

The client on LAN:

got ping replies from the client on VLAN92

The client on VLAN92:

got ping timeout from the client on LAN
got ping timeout from the pfSense's interface ipv6 address on LAN

When I copied the "Default Allow LAN IPv6 to any rule" from LAN to VLAN92, both ping commands from VLAN92 started giving replies (I left them running while applying the firewall rule)

...

And then ... I rebooted the firewall.

After the reboot, none of the ping commands seem to work anymore, all just give timeout. (I left them running while rebooting the firewall)

Strange thing is:

no configuration was changed on the firewall
no configuration was changed on any of the virtual machines

I can see no changes in the config before the reboot, and the config after the reboot.
Still, it worked before and it doesn't after.
Whatever I do, I can no longer get it to work.

Tanya

@xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

I was writing a post asking for help with the same think but I thing though I might have solved it. The setup I have is:

GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.

A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11

fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface

A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

That doesn't work for me, as - like I more or less suspected - the GUA disappears from the interface after a reboot, and the Virtual IP is assigned to the interface instead.
The net result is that the clients now only get ULA addresses and no longer a GUA.
Hence: no internet access.

In your case, maybe the GUA stays in place because you have defined them as static rather than track interface?

Tanya

Anyone with an idea / suggestion / recommendation / something I could try ?
I have run out of ideas, and - alas - made little to no progress at all.

chpalmer

@tanya said in ipv6: can ping GUA address in different VLAN, but not ULA.:

In your case, maybe the GUA stays in place because you have defined them as static rather than track interface?

Your using "Track Interface"? If so what interface are you tracking?

Tanya

@chpalmer

That's the WAN interface I'm tracking.
Even if I wanted to, the dropdown list doesn't let me select anything else. It's WAN or nothing.
The Track Interface part in itself is working.
It's just that when I add an IP alias (and reboot) that it stops working: the GUA is no longer assigned to the interface, but the virtual IP is.
(even though the configuration it itself is unchanged)

That is IIRC a known bug or issue, I've seen it in a few other posts as well.

Tanya

Any other suggestions?

I have tried everything that was suggested and everything I could think of, but no success so far.

Two weeks ago I have created a bug report for the scenario I described above (post 15) where a working setup with ULA addresses stopped functioning after only reboot of the firewall.
Sadly, that too has generated little interest (none in fact) and has been shuffled back to page 2 of the bugtracker.