No traffic gets past HE ipv6 tunnel


  • Hello,

    I am trying to setup a HE ipv6 tunnel.
    GIF Interface works and comes up. Clients in LAN get proper ipv6 addresses but I can not ping anything on the far end of the tunnel.

    I do have a XG7100 with two WAN interfaces up. The tunnel is connected via the PPPoE interface, I already set the MTU size to 1452 on the Hurricane Electric advanced tab.

    The GW is up
    ec319b12-4ee5-4a67-b907-cc680478dbae-image.png

    The LAN interface is set up:
    290e08a1-82aa-4497-b3c8-92f4eeec127d-image.png

    DHCPv6 looks like so:
    eede6c7e-06ab-4e66-8c43-86fb95966f0a-image.png

    Clients do get an IPv6 Address:
    f78ad614-4e3d-4b7f-98be-1ab86a58ae85-image.png

    Firewall rule to allow IPv6 traffic from LAN to any is setup.

    However, nothing get's past the pfsense.
    Any ideas would be very welcome.

    Cheers,
    Markus


  • @toskium

    Two thoughts, firewall rules and routing. You need both to be properly configured. Without knowing more about what you've done, that's all I can offer.


  • @JKnott don't know if the following is sufficient, let me know if you need to know more.

    ipv6 Routes on the pfsense look like so:

    c0b966ad-3b1d-4ad7-8542-b301243fad07-image.png

    the only ipv6 rule on the LAN interface is that:

    c6cf3d62-2391-4f26-9f36-6d841b3a3f5f-image.png


  • @toskium

    Then you'll have to do some testing. For example you say you can't ping anything at the other end. Do you mean the tunnel end point? Or something on the LAN that the end point is connected to? The first is some hard error. The other is a routing issue. So, start with pinging end point to end point and if that's successful, go further. Also, is it a DNS issue, where pinging by host name fails? What about by IP address?


  • Probably not related, but this firewall rule won't introduce any issues :

    9675447f-0b28-4a0b-b059-22816407ce9e-image.png

    I'm using tunnel.he.net myself.

    I compared your routing table with mine.
    I found 3 differences :

    1 = I'm using pfBlockerNG(latest) : that' where my ::10.10.10.1 comes from.
    3 = I'm exposing a /64 out of the he.net's /46 for my VPN server, so I can use IPv4 and IPv6 for my road warriors.

    It sure 1 & 3 are actually not related.

    2=> Mine is attached to an interface 'em1'.
    Your is a lagg ?? Now where did that lagg came from ? ( and what is it ? ^^)
    I advise you to go by the books : use an (one unique) LAN interface and make IPv6 work. When done, go lagg ...

    53b3f877-503b-4f5f-bb11-232911fde714-image.png


  • @Gertjan Mine is on a lagg interface since the xg7100 does not have separate physical interfaces. The current config for the xg7100 is the result of a conversion from a SG3100. This was done by netgate support and they asked me in the process which switch port is supposed to be mapped on what interface, so that's the reason the interface looks as it does :-).


  • I was kidding, I know what lagg is, but never used it - don't need it.

    The thing is : for that small detail, everything look good.

    Just to be sure :

    The gateway tab :

    1fc5fc1a-c1cf-4a99-8c94-253af2d00317-image.png

    and

    f53e866e-e7f3-4a3b-8dc8-73ff597818f7-image.png

    edit : and because he;net is some kind of ISP for you and mle, always keep an eye on https://tunnelbroker.net/status.php
    and the he.net users forum : the POP's stop working ones in a while.


  • @Gertjan thanks for your input. Unfortunately it all looks correct.
    The only major difference I seem to have is my multi-wan setup.

    My gateways tab looks like so:
    73bebbca-6e1b-4666-8cb9-1b29f56b27ae-image.png

    You seem to be using a different monitoring IP though, but that shouldn't be an issue.

    What I tried so far:

    • ruled out name resolution, by directly pinging ipv6 address.
    • ruled out wrong routes from client by pinging directly from pfsense to ipv6 address.

    Since you mentioned the tunnelbroker.net status page, all tunnel servers are up and running at the time of my testing.


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    different monitoring IP though

    I've use one of my own servers to ping-reply for dpinger.

    25d92eb0-90cf-4a56-9aae-6d6402291fc5-image.png

    Using the console / SSH access, option 8, you can ping6 to some host ?

  • LAYER 8

    i have pppoe and he.net myself, to make it work for me I had to set mtu to 1472 and mss to 1440 on the gif interface
    on tunnelbroker.net the mtu is set to 1472


  • Same thing here :

    ade87f5a-75c5-4719-8e66-7b822b075278-image.png

    Although my WAN is connected to my ISP router, I know this routers is doing pppoe on the ADSL side.

    edit : and yes, on the he.net side MTU is set to 1472.


  • @Gertjan @kiokoman
    I tried what you just suggested, unfortunately no change in the behaviour.
    I can't even ping the tunnel server ipv6 endpoint address even though the tunnel is up.
    In theory I would need to be able to ping the ipv6 tunnel server address from my pfsense when selecting the HE tunnel interface.

  • LAYER 8

    but the tunnel show it's up, can you ping from pfsense that 2001:470:6c ::1 and ::2 ?


  • @kiokoman I can ping myself -> ::2 but I can not ping the tunnel end at HE with -> ::1.
    I am mystified :-)


  • If you can't ping6 the remote part of the tunnel, the he.net POP, the one ending with ::1 then I advise you to use ping6 with some parameters, like :

    ping6 -I gif0 2001:470:1f12:5c0::1
    

    To force it to use the correct interface.

    You can get the interface name with

    ifconfig
    

    Btw : starts to looks like a routing issue.

    You have no IPv6 activated on your WAN_DHCP and WAN2_PPPOE as these are IPv4 only (are they ?).


  • @Gertjan

    ping6 -I gif0 -c 3 2001...
    

    works fine. So the tunnel by itself is working.

    what I do not yet understand is why that doesn't work from the GUI diagnostic ping when I specifically set the interface to the HE gif0 interface. In theory it should deliver the same result.

    WAN:
    0ebe47a9-0ed3-4287-944f-1db7d88c33e0-image.png

    WAN2:
    84b93276-8524-4a07-8f68-aa6fabf87cc4-image.png

    I do not see any other ipv6 related interfaces in the whole config.


  • This post is deleted!

  • That's the current ipv6 routing table:

    Destination	Gateway	Flags	Use	Mtu	Netif	Expire
    default	2001:470:6c:aaaa::1	UGS	151	1500	gif0	
    ::1	link#19	UH	1747	16384	lo0	
    2001:470:20::2	2001:470:6c:aaaa::1	UGHS	0	1500	gif0	
    2001:470:6c:aaaa::1	link#43	UH	23260	1280	gif0	
    2001:470:6c:aaaa::2	link#43	UHS	6	16384	lo0	
    2001:470:6d:aaaa::/64	link#32	U	6249	1500	lagg0.4088	
    2001:470:6d:aaaa::1	link#32	UHS	0	16384	lo0	
    2001:4860:4860::8888	2001:470:6c:aaaa::1	UGHS	0	1500	gif0	
    fe80::%igb0/64	link#1	U	0	1500	igb0	
    fe80::8261:5fff:fe04:ea3f%igb0	link#1	UHS	0	16384	lo0	
    fe80::%lo0/64	link#19	U	0	16384	lo0	
    fe80::1%lo0	link#19	UHS	0	16384	lo0	
    fe80::%lagg0/64	link#23	U	0	1500	lagg0	
    fe80::208:a2ff:fe11:5f66%lagg0	link#23	UHS	0	16384	lo0	
    fe80::%igb0.7/64	link#24	U	0	1500	igb0.7	
    fe80::8261:5fff:fe04:ea3f%igb0.7	link#24	UHS	0	16384	lo0	
    fe80::%lagg0.4081/64	link#25	U	0	1500	lagg0.4081	
    fe80::208:a2ff:fe11:5f66%lagg0.4081	link#25	UHS	0	16384	lo0	
    fe80::%lagg0.4082/64	link#26	U	21	1500	lagg0.4082	
    fe80::208:a2ff:fe11:5f66%lagg0.4082	link#26	UHS	0	16384	lo0	
    fe80::%lagg0.4083/64	link#27	U	0	1500	lagg0.4083	
    fe80::208:a2ff:fe11:5f66%lagg0.4083	link#27	UHS	0	16384	lo0	
    fe80::%lagg0.4084/64	link#28	U	0	1500	lagg0.4084	
    fe80::208:a2ff:fe11:5f66%lagg0.4084	link#28	UHS	0	16384	lo0	
    fe80::%lagg0.4085/64	link#29	U	0	1500	lagg0.4085	
    fe80::208:a2ff:fe11:5f66%lagg0.4085	link#29	UHS	0	16384	lo0	
    fe80::%lagg0.4086/64	link#30	U	0	1500	lagg0.4086	
    fe80::208:a2ff:fe11:5f66%lagg0.4086	link#30	UHS	0	16384	lo0	
    fe80::%lagg0.4087/64	link#31	U	0	1500	lagg0.4087	
    fe80::208:a2ff:fe11:5f66%lagg0.4087	link#31	UHS	0	16384	lo0	
    fe80::%lagg0.4088/64	link#32	U	619	1500	lagg0.4088	
    fe80::208:a2ff:fe11:5f66%lagg0.4088	link#32	UHS	0	16384	lo0	
    fe80::%lagg0.4000/64	link#33	U	0	1500	lagg0.4000	
    fe80::208:a2ff:fe11:5f66%lagg0.4000	link#33	UHS	0	16384	lo0	
    fe80::%lagg0.20/64	link#34	U	0	1500	lagg0.20	
    fe80::208:a2ff:fe11:5f66%lagg0.20	link#34	UHS	0	16384	lo0	
    fe80::%lagg0.30/64	link#35	U	0	1500	lagg0.30	
    fe80::208:a2ff:fe11:5f66%lagg0.30	link#35	UHS	0	16384	lo0	
    fe80::%lagg0.40/64	link#36	U	0	1500	lagg0.40	
    fe80::208:a2ff:fe11:5f66%lagg0.40	link#36	UHS	0	16384	lo0	
    fe80::%lagg0.50/64	link#37	U	0	1500	lagg0.50	
    fe80::208:a2ff:fe11:5f66%lagg0.50	link#37	UHS	0	16384	lo0	
    fe80::%lagg0.60/64	link#38	U	0	1500	lagg0.60	
    fe80::208:a2ff:fe11:5f66%lagg0.60	link#38	UHS	0	16384	lo0	
    fe80::%lagg0.70/64	link#39	U	0	1500	lagg0.70	
    fe80::208:a2ff:fe11:5f66%lagg0.70	link#39	UHS	0	16384	lo0	
    fe80::%lagg0.80/64	link#40	U	0	1500	lagg0.80	
    fe80::208:a2ff:fe11:5f66%lagg0.80	link#40	UHS	0	16384	lo0	
    fe80::%lagg0.90/64	link#41	U	0	1500	lagg0.90	
    fe80::208:a2ff:fe11:5f66%lagg0.90	link#41	UHS	0	16384	lo0	
    fe80::%pppoe0/64	link#42	U	0	1492	pppoe0	
    fe80::208:a2ff:fe11:5f66%pppoe0	link#42	UHS	0	16384	lo0	
    fe80::%gif0/64	link#43	U	0	1500	gif0	
    fe80::8261:5fff:fe04:ea3f%gif0	link#43	UHS	0	16384	lo0
    

    for better readability:
    e9f7ec0f-f645-4cac-afba-590cef7d8ff6-image.png

  • LAYER 8

    @toskium said in No traffic gets past HE ipv6 tunnel:

    2001:470:20::2

    what is it? ^
    and why do you have google dns there ?

    ah i see it's he net dns


  • @kiokoman this comes from my general DNS settings, the howto on docs.netgate.com stated to add google DNS servers in System > General Setup like so:

    b0cc0d12-6847-46cd-98b7-8350b8d61754-image.png

  • LAYER 8

    but i don't have any dns server on my routing table (i'm also using the /48 from he net)

    Internet6:
    Destination                       Gateway                       Flags     Netif Expire
    default                           2001:470:25:xxx::1            UGS        gif0
    ::1                               link#4                        UH          lo0
    2001:470:25:xxx::1                link#9                        UH         gif0
    2001:470:25:xxx::2                link#9                        UHS         lo0
    2001:470:26:xxx::/64              link#2                        U           em1
    2001:470:26:xxx::1                link#2                        UHS         lo0
    2001:470:b4e1:xxx::/64           link#3                        U           em2
    2001:470:b4e1:xxx::1             link#3                        UHS         lo0
    fe80::%em0/64                     link#1                        U           em0
    fe80::5054:ff:fe3d:64cc%em0       link#1                        UHS         lo0
    fe80::%em1/64                     link#2                        U           em1
    fe80::5054:ff:fe91:db46%em1       link#2                        UHS         lo0
    fe80::%em2/64                     link#3                        U           em2
    fe80::5054:ff:fe27:556a%em2       link#3                        UHS         lo0
    fe80::%lo0/64                     link#4                        U           lo0
    fe80::1%lo0                       link#4                        UHS         lo0
    fe80::%em1.10/64                  link#8                        U        em1.10
    fe80::5054:ff:fe91:db46%em1.10    link#8                        UHS         lo0
    fe80::%gif0/64                    link#9                        U          gif0
    fe80::6097:dd62:2e35:991d%gif0    link#9                        UHS         lo0
    fe80::6097:dd62:2e35:991d%ovpnc1  link#10                       UHS         lo0
    

  • @kiokoman fair enough, but how did they end up there? (I guess that's a rhetorical question...)
    Removing them from System > General Setup does not purge them from the routing table.

    Edit:
    okay, restarting the gif0 interface purges them. It seems like they are added to the routing table when being entered in System > General Setup as a DNS server.

    Now that I am able to ping the ipv6 address of the tunnel server over at HE (2001:470:....::1) using:

    ping6 -I gif0 2001...
    

    I should also be able to ping other ipv6 hosts, but I can't. For instance ipv6.google.com

    ping6 -I gif0 2a00:1450:4005:803::200e
    

    leads to 100% package loss

  • LAYER 8

    manually delete it
    route -6 del 2001:470:20::2 2001:470:6c:aaaa::1
    route -6 del 2001:4860:4860::8888 2001:470:6c:aaaa::1
    ok sorry i'm at work, i was too late on answering

    i think you have discovered a bug there ^ ...
    i have one of my pfsense with a route that appear at boot out of nowhere, i have setup an earlyshellscript to remove everytime that offending route, since 2.4.4-p3
    https://forum.netgate.com/topic/147254/lost-ipv6-connectivity-from-one-interface


  • Discovering bugs is fine :-) where can I report that properly so it has a chance of being fixed?

  • LAYER 8


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    @kiokoman this comes from my general DNS settings, the howto on docs.netgate.com stated to add google DNS servers in System > General Setup like so:

    b0cc0d12-6847-46cd-98b7-8350b8d61754-image.png

    A bug, maybe -I'll add some @home and see what happens.

    Why did you add all these DNS servers ?
    You are aware that you don't need them ?? The resolver, out of the box is close to perfect. [ and then people start forwarding because ... / [ we never know why ] /..... and things go downhill ]

    edit :
    When I add these :

    881fd212-7933-46b5-a277-9f863d3b0fc5-image.png

    ...the IPv6 of the DNS of he.net, I wind up seeing this :

    3d4fe86a-9119-4a37-9b9f-d40e6fdd292d-image.png

    in the routing table.
    Which doesn't look 'wrong' to me, as 2001:470:20::2 should be reached over the interface gif0 = he.net = my (their) 2001:470:1f12:5xx::1

    edit : and my IPv6 still works ....

  • LAYER 8

    because i use bind on another server and not unbound nor forwarder for example ^^


  • @Gertjan I added the DNS servers because the howto says so.


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    the howto says so

    Source ?

    Read again the initial setup instruction : you'll find https://docs.netgate.com/pfsense/en/latest/config/general.html where it says :

    Note
    The DNS Resolver is active by default and uses resolver mode (DNS Resolver). When set this way the DNS Resolver does not need forwarding DNS servers as it will communicate directly with root DNS servers and other authoritative DNS servers.

    I don't want to say that forwarding - using tiers DNS servers, is bad.
    It was somewhat mandatory in the early ages. But not anymore.
    You use DNS : use the source aka "Internet itself".


  • @Gertjan according to the recipe for setting up a ipv6 tunnel. You will need to go to docs.netgate.com and search vor Hurricane Electric. Askismet doesn't allow me to post the direct link. It's the first result, if you look in the DNS chapter of that recipe you will find the reference.


  • You mean :

    f42fadd6-28f1-498f-a6a7-b5d1e050084a-image.png

    Or https: slash slash docs dot netgate dot com slash /pfsense/en/latest/recipes slash ipv6-tunnel-broker.html#setup-ipv6-dns ?
    (Askismet can be circumvented so easily)

    Because the doc is somewhat old.

    This :

    If the DNS Resolver is used in non-forwarding mode, it will talk to IPv6 root servers automatically once IPv6 connectivity is functional.

    is not an "if" any more.
    The DNS Resolver is used out of the box.
    pfSense used a forwarder in the past, it's still there : the lightweight forwarder (dnsmasq), mutual exclusive with the functionality of the resolver (unbound).

    Way back, code had to be mean, lean and small, as devices had limited resources.
    The Internet start with these, Internet IS https://en.wikipedia.org/wiki/Root_name_server.
    So : why take your info from "some one" if you can tap into the source ?

    But ...... the damage has been done.
    people like to use VPN 'for protection' and '8.8.8.8' for their DNS, and antivirals for their safety. They haven't figured out yet that it's all - and only - a "€/$" thing.

    To gain some milli seconds, it could be useful to use a close by DNS server. he.net has a (their) own DNS servers close at every POP.
    You don't need them, they are just optional, as all the others.


  • @gertjan @kiokoman
    Thank you all for providing support but in the end it turns out it's a routing issue on Hurricane Electrics end.

    What did I do?
    I first registered a tunnel on the nearest server here in Germany (Berlin), I followed the setup but it didn't work.

    After doing some debugging with the packet filter it became clear that absolutely no traffic was routed from HEs end to my router.

    Since desperate times call for desperate measures I deleted the endpoint and reregistered a new one but in Frankfurt instead of Berlin.

    The Frankfurt tunnel works as expected, unfortunately the latency is significantly higher.
    I'll write an email to HEs support since the not working Berlin tunnel had been the second attempt, so I guess there is something up with the Berlin endpoint.


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    The Frankfurt tunnel works as expected, unfortunately the latency is significantly higher.

    You should be able to use any tunnel site. Try another to see if it works any better.


  • Berlin is working fine here. But it was down some days ago.


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    I'll write an email to HEs support since the not working Berlin tunnel had been the second attempt, so I guess there is something up with the Berlin endpoint.

    Use their forum. They are reactive. https://forums.he.net/
    Do the save approach : if thousands are connected to "Berlin / Düsseldorf or Frankfurt" but you can't, then where would lie the issue ? ;)
    If a tunnel goes bad, you will find notification about it on the forum after a minute or so, as it impact many users.

    https://www.tunnelbroker.net/usage/tunnels_by_country.php : 5000 (german) users !?


  • @gertjan I have read in the internets that this sometimes happens and it is advisable to just delete the tunnel and request a new prefix.

    I tried that, but then I realized that I got the same tunnel prefix assigned again.

    For the life of me, I did not change anything other than the GIF interface and the ipv6 net on my LAN interface and ping from the router to ipv6.google.com instantly worked.

    Yes, I know how that sounds 😅


  • @toskium said in No traffic gets past HE ipv6 tunnel:

    I have read in the internets that this sometimes happens and it is advisable to just delete the tunnel and request a new prefix.

    Read that where ? ? ?
    Deleting your tunnel will discard your IPv6 /64 and IPv6/48 and you start with new ones.
    I'm using tunnel.he.net nearly 10 years now, and never had to do that.
    The POP in Paris (France) goes down ones in a while : not an issue as IPv4 and IPv6 are complementary for now : one can replace the other.
    If my IPv6 network prefix is not working well, then throwing away a prefix and taking another one on the same POP won't help much : the issue stays.
    If the issue happens often, the I tend to say : it's a local issue.

  • LAYER 8 Moderator

    @gertjan said in No traffic gets past HE ipv6 tunnel:

    @toskium said in No traffic gets past HE ipv6 tunnel:

    I have read in the internets that this sometimes happens and it is advisable to just delete the tunnel and request a new prefix.

    Read that where ? ? ?
    Deleting your tunnel will discard your IPv6 /64 and IPv6/48 and you start with new ones.
    I'm using tunnel.he.net nearly 10 years now, and never had to do that.
    The POP in Paris (France) goes down ones in a while : not an issue as IPv4 and IPv6 are complementary for now : one can replace the other.
    If my IPv6 network prefix is not working well, then throwing away a prefix and taking another one on the same POP won't help much : the issue stays.
    If the issue happens often, the I tend to say : it's a local issue.

    I agree. In my case it's the other way round - I had a tunnel with GER/Frankfurt for years running relatively smooth until about a year ago. I guess it's routing issues on Vodafones cable side as before it was flawless. Since then I had around 1-3% pkg drops and latency on that. As I had time a few weeks ago I long-term-tested several end points of HE in Germany and around (France, NL, Suisse) and found to my surprise, that besides being 3 hops further away from me, the Suisse one was much better for my location (less delay, less latency 0% pkg loss all the time) so I created a new tunnel there, changed my settings from FRA to SUI and tested again. Up until now all is running well with that!

    So just dropping your prefix on the same pop for a new one I won't expect much change. Perhaps a change of endpoint is required that suits your connection better.


  • I see. Will tracerouting the ipv4 addresses shown in the registration process be sufficient to tell if a specific tunnel endpoint is a good choice or will it require registration and bringing up the tunnel itself to be sure?