ipv6: can ping GUA address in different VLAN, but not ULA.



  • Hi,

    As the title says: I have trouble reaching ULA addresses on other VLANs.

    Short version:

    Test machine 1 can ping Test machine 2 (same VLAN) on its GUA and ULA.
    Test machine 1 can ping Test machine 3 (different VLAN) on its GUA, but NOT on its ULA.

    AFAIK I have made no changes to the firewall rules that explain this behaviour.

    In a further test I have created an additional (dummy) VLAN with a fixed IPv6 on its interface.
    If that fixed IPv6 is one from the global range Test Machine 1 (on other VLAN) can ping it. If I change the address to a ULA, it can no longer ping it.

    I would expect ULA addresses to reachable.
    Do I misunderstand something, or is this a bug, or an error in my configuration?

    For details on what, why and how, please see my next post.



  • Long version with all details:

    pfSense version: 2.4.3-RELEASE-p1 (amd64)
    built on Thu May 10 15:02:52 CDT 2018
    FreeBSD 11.1-RELEASE-p10

    I have set up a few VLANs on our pfSense (= also main router) and a few virtual machine to prepare for future rollout of ipv6.

    I get a 56bit prefix from my ISP and use track interface on VLAN20 and VLAN40. I use the VLAN number as ipv6 prefix ID.
    Apart from that I added an extra subnet (ULA) to the RA for both VLANs.
    For the ULA I used a tool to generate a 48bit part and added the VLAN number again to complete the 64bits.

    [The reason I also want the ULA addresses is that I want to use static ULAs for our servers.
    We will probably move to another building (first) and change ISP (later, when fiber is available) in the near future, so probably that means at least one and
    maybe even 2 changes to our prefix - and thus GUA addresses. Hence, ULA seems the safest option.]

    The effect is that the clients get 6 IP-addresses. For example, a VM in VLAN20:

    • ipv4 = 192.168.20.101
    • ipv6 GUA = 2a02:xxxx:yyyy:zz20:743b:f963:8017:9a63
    • ipv6 ULA = fdfd:d8e5:340d:0020:743b:f963:8017:9a63
    • ipv6 LL = fe80::743b:f963:8017:9a63
    • ipv6 GUA temporary = 2a02:xxxx:yyyy:zz20:254e:c44a:cdbb:64c9
    • ipv6 ULA temporary = fdfd:d8e5:340d:0020:254e:c44a:cdbb:64c9

    All my test VMs are Windows 7 Pro machines.
    I have disabled the Windows Firewall on all test machines to make sure that doesn't affect testing.

    A Test Machine on VLAN20 can ping another test machine on VLAN20 on all 6 IP addresses.
    Likewise a test machine on VLAN40 can ping another test machine on VLAN40 on all 6 IP addresses.

    However, when I ping from a test machine on VLAN20 to a test machine on VLAN40 I do get some unexpected results.

    • ping ipv4 = success
    • ping ipv6 GUA = success
    • ping ipv6 ULA = failed [unexpected]
    • ping ipv6 LL = failed [expected]
    • ping ipv6 GUA temporary = success
    • ping ipv6 ULA temporary = failed [unexpected]

    While the LL failure is of course totally expected, I'm surprised the ULA addresses don't work.
    If I understand correctly, ULA address should be accessible on other VLANs, but not pass to the outside world.

    I have not blocked anything in the firewall rules on VLAN20.
    I tried to explicitly allow anything ipv6 related (added the middle rule) but that made no difference.

    0_1531409871801_VLAN20.PNG

    There are a few rules on VLAN40, but only IPV4-related.



  • For a further test, I added a extra VLAN (70). There are no machines in this VLAN.
    Instead of Track Interface, I have given it a static IP address.

    I gave it a GUA-style address, 1111:2222:3333:4444::1

    • I can ping it from my Test Machine in VLAN20.
    • I cannot ping it from my Test Machine in VLAN 40 [that's normal I guess as VLAN40 as no ipv6 rules]

    Next I change the static IP address to an ULA-style address fdfd:d8e5:340d:0070::1

    • I can no longer ping it from my Test Machine in VLAN20.

    That seems to be the same effect as pinging an ULA client address in other VLANs.
    Maybe it has the same cause too?

    • bug?
    • misconfiguration?
    • ... ?


  • Being Im not anywhere close to knowledgeable with IPv6 I had a theory and it seems to be true..

    From https://en.wikipedia.org/wiki/Unique_local_address

    A unique local address (ULA) is an Internet Protocol version 6 (IPv6) address in the address range fc00::/7. ... They are routable only within the scope of such private networks, but not in the global IPv6 Internet.

    I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

    I may be totally wrong but it may give you a clue or two to look at.. :)



  • I must admit that is a bit confusing.

    I was under the impression that ULA is the equivalent of the IPv4 private addresses (192.168.x.x, 10.x.x.x) and that Link Local (fe80::xxx) was the equivalent of APIPA (Automatic Private IP, i.e. 169.254.x.x)

    The main difference between ULA and Link Local is - as far as I understood - that ULA could be routed (locally) and LL could not.

    That is how I interpreted what I have read about it, but of course I may be wrong, or my source may have had it wrong.

    I can't find the docs I based my interpretations on at the moment. I'll try to find it back and have a closer look at it.



  • Yep-true.. IPv4 local addresses are in deed routable internally and one would think that there would be a similar IPv6 solution.

    This page wants you to subscribe but you can copy and paste..
    https://community.infoblox.com/t5/IPv6-CoE-Blog/3-Ways-to-Ruin-Your-Future-Network-with-IPv6-Unique-Local/ba-p/5663

    http://biplane.com.au/blog/?p=12

    Looks like your right according to these. IMO it should be routable accross the platform.



  • @chpalmer said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

    That is incorrect. They are routeable, but not over the public Internet. They're used just like the RFC1918 addresses on IPv4.



  • @jknott said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    @chpalmer said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    I think their use of the word "routable" is misleading as I believe they mean it has to be within the same subnet. Your operating across interfaces thus routing between the addresses.

    That is incorrect. They are routeable, but not over the public Internet. They're used just like the RFC1918 addresses on IPv4.

    Yep- like I said in my last post..
    Looks like your right according to these. IMO it should be routable accross the platform.



  • OK, so basically we all agree and think it should work.
    That's good, that would mean that the concept of what I'm trying to achieve seems OK.

    The bad thing is that - currently - it does not work.
    No idea yet if that is caused my a bug, a misconfiguration on my part (missing or wrong firewall rules?) or both?

    Does anyone have a clue?



  • Make sure that these aren't checked in the Interface configuration each VLAN.
    0_1531495088117_3b804dff-bc8a-4ef2-beee-16fef5b07464-image.png



  • @awebster

    Thanks for the suggestion.
    I'll check that when I'm back at work on Monday (it's 18:20 over here, working day is done :) )

    I'm quite convinced the first option is unchecked, otherwise the ipv4 ping 192.168.x.x would't pass either (it does).
    I'm not so sure about the second option. If the default setting is OFF then it is probably still off. I can't remember turning it on, anyway.

    But both need doublechecking on Monday. I'll get back with the results when that's done.



  • @awebster said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    Make sure that these aren't checked in the Interface configuration each VLAN.

    I have just double-checked and can confirm that both options are unchecked on all of the VLAN interfaces



  • I was writing a post asking for help with the same think but I thing though I might have solved it. The setup I have is:

    • GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.
    • A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11
    • fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface
    • A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

    Then I can ping the host on VLAN11 from a host on VLAN10. I'm having trouble allowing TCP through at the moment, some more debugging will be required there. Hopefully this is helpful to you.



  • @xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    • A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

    Then I can ping the host on VLAN11 from a host on VLAN10. I'm having trouble allowing TCP through at the moment, some more debugging will be required there. Hopefully this is helpful to you.

    I'm certainly going to check that firewall rule and try if I can get something done that way.
    If I understand correctly, the rule you added grants access from (all) your machines on VLAN10 to a single machine on VLAN11?

    In my situation, I would need all my machines from the first VLAN to be able to access all of the hosts in the 2nd VLAN. Vice versa, all machines in VLAN2 should only be able to reach a very limited number of machines in VLAN1.

    That's how the ip4-equivalent is set up.

    @xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    • GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.
    • A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11
    • fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface

    Do the Virtual IP and the static GUA still work after you reboot pfsense?
    IIRC there is a problem with track interface and virtual IPs where after the reboot the interface only has the virtual IP and no longer the GUA.
    That's something I read on the forums and also experienced as such when I was seeing if a virtual IP could help me out with this problem.

    Anyhow, you gave me a few things to try. I'll get back with the results soon.



  • To do some further testing, I have made a setup with 3 virtual machines

    • a new pfSense (version 2.4.3)
    • a Windows client (Win7, firewall disabled)
    • a 2nd Windows client (Win7, firewall disabled)

    These 3 are totally isolated from the rest of my network.
    I tried to setup pfSense with as little changes as possible (no VPN, ...), just the ipv6 stuff.

    • The WAN is set to auto (DHCP) but not connected (so track interface won't work)

    • LAN with Static IPv6: fddd:666:666:90::1

    • VLAN91 with Static IPv6: fddd:666:666:91::1

    • VLAN92 with Static IPv6: fddd:666:666:92::1

    • LAN: RA = assisted

    • VLAN92: RA = Stateless DHCP

    I put 1 Win client on LAN and one on VLAN92.

    The client on LAN:

    • got ping replies from the client on VLAN92

    The client on VLAN92:

    • got ping timeout from the client on LAN
    • got ping timeout from the pfSense's interface ipv6 address on LAN

    When I copied the "Default Allow LAN IPv6 to any rule" from LAN to VLAN92, both ping commands from VLAN92 started giving replies (I left them running while applying the firewall rule)

    ...

    And then ... I rebooted the firewall.

    After the reboot, none of the ping commands seem to work anymore, all just give timeout. (I left them running while rebooting the firewall)

    Strange thing is:

    • no configuration was changed on the firewall
    • no configuration was changed on any of the virtual machines

    I can see no changes in the config before the reboot, and the config after the reboot.
    Still, it worked before and it doesn't after.
    Whatever I do, I can no longer get it to work.



  • @xianic said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    I was writing a post asking for help with the same think but I thing though I might have solved it. The setup I have is:

    • GUAs on the VLAN interface are statically assigned, tracked should work just as well and would probably be less work for me.
    • A /64 ULA prefix for each VLAN on the RA page, e.g fdxx:xxxx:xxxx:a::/64 on VLAN10 and fdxx:xxxx:xxxx:b::/64 on VLAN11
    • fdxx:xxxx:xxxx:a::1 and fdxx:xxxx:xxxx:b::1 virtual addresses assigned to each interface
    • A firewall rule on VLAN10 interface to allow all traffic from fdxx:xxxx:xxxx:a::/64 to a specific host on VLAN11

    That doesn't work for me, as - like I more or less suspected - the GUA disappears from the interface after a reboot, and the Virtual IP is assigned to the interface instead.
    The net result is that the clients now only get ULA addresses and no longer a GUA.
    Hence: no internet access.

    In your case, maybe the GUA stays in place because you have defined them as static rather than track interface?



  • Anyone with an idea / suggestion / recommendation / something I could try ?
    I have run out of ideas, and - alas - made little to no progress at all.



  • @tanya said in ipv6: can ping GUA address in different VLAN, but not ULA.:

    In your case, maybe the GUA stays in place because you have defined them as static rather than track interface?

    Your using "Track Interface"? If so what interface are you tracking?



  • @chpalmer

    That's the WAN interface I'm tracking.
    Even if I wanted to, the dropdown list doesn't let me select anything else. It's WAN or nothing.
    The Track Interface part in itself is working.
    It's just that when I add an IP alias (and reboot) that it stops working: the GUA is no longer assigned to the interface, but the virtual IP is.
    (even though the configuration it itself is unchanged)

    That is IIRC a known bug or issue, I've seen it in a few other posts as well.



  • Any other suggestions?

    I have tried everything that was suggested and everything I could think of, but no success so far.

    Two weeks ago I have created a bug report for the scenario I described above (post 15) where a working setup with ULA addresses stopped functioning after only reboot of the firewall.
    Sadly, that too has generated little interest (none in fact) and has been shuffled back to page 2 of the bugtracker.



  • Sadly pfsense is atm not fully IPv6 compliant imho, the freeBSD underlying parts are, but not the management parts of pfsense.

    It's like in any other linux project, u still get only answers if the developers are interested. :/

    For the rest of us, we need to invent the wheel every day again and again.



  • Just to check some more, in my test setup (3 virtual machines) I have replaced pfSense with one of its competing products, and applied the same minimal configuration.
    Unlike pfSense, this setup keeps working after a reboot of the firewall/router.

    This once again confirms that what I intend to do should just work, doesn't it?

    Finally, I removed the competing product, and once more installed pfSense from scratch, and again applied the minimal config.
    Sadly, it again stops working once the firewall has been rebooted.

    I really would like to continue using pfSense, but I do need some help to make this work.



  • U have to setup the routes statically since pfsense does all over gui wrappers.

    What happens after an reboot is, that the IP ordering GUA/ULA changes and, after all, so some of the needed routes get "lost" this way.

    imo i tracked down the error to the interfaces.inc, but since netgate remains silent, i do not see to fix that(have similar IPv6 problems)



  • @ulfmerbold

    You mean: this is a bug/feature and unless someone solves it, I will never get it to work?



  • It isnt a feature, but yeah if u cant solve the ordering issue for self u have to wait and someone of the developers has to fix that bug. And its surely a bug when one setup has different results ->before reboot/after reboot.

    Btw, what was the competing product if i may ask?



  • Ah ok, know that too but i still tried pfsense to get to work in this case. I need pfblocker for example, so it isnt so easy to change for me.

    In your case i would choose the one which does job best, at the end all are a fork of something else.

    So vote with your foots and dont stay silent if the support is crap for u, not only the paying customers are customers.



  • to the interfaces.inc file:
    The specific parts of the script just checks for link local and an interface ipv6, but since IPv6 knows more than one type of an interface IP (GUA and ULA handled by a single function and stops if an matching IP is found)

    This could be the reason for the behavior i ve seen for my problem and at the end for ur's too.

    For me an ifconfig in a console, i ll get all IPv6 IP's of an specific interface...if i do same in the gui i ll just only get two IP's
    So u get for example in GUI an LL+GUA or an LL+ULA, but NOT ULA+GUA+LL

    Since most configs generate from the pfsense scripts, the underlying "real" IP's are ignored in this case.

    At the end u have missing routes, cause the routes are build from only the half of informations needed

    But my programming skills are not so deep to evaluate my thinking, im an hardware guy. :/