Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HE Tunnel Problem

    Scheduled Pinned Locked Moved IPv6
    27 Posts 6 Posters 6.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimpJ
      jimp Rebel Alliance Developer Netgate
      last edited by

      If you assigned the tunnel interface and set it all properly, and set the gateway as default, it should handle that on its own:

      https://doc.pfsense.org/index.php/Using_IPv6_with_a_Tunnel_Broker

      There is a problem on 2.4 with setting gateways in some cases but that wouldn't affect your 2.3.x box

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      1 Reply Last reply Reply Quote 0
      • B
        bimmerdriver
        last edited by

        @jimp:

        If you assigned the tunnel interface and set it all properly, and set the gateway as default, it should handle that on its own:

        https://doc.pfsense.org/index.php/Using_IPv6_with_a_Tunnel_Broker

        There is a problem on 2.4 with setting gateways in some cases but that wouldn't affect your 2.3.x box

        Thank you for the reply. I looked at that procedure. The tunnel on the 2.3.2_1 system is the one that's working. I'm having problems getting it working on 2.4.

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          In the first post of the thread you said the one that wasn't working was 2.3.2_1, or did I misread that?

          If it fails on 2.4 with a tunnel, then you're probably hitting this:
          https://redmine.pfsense.org/issues/6828

          Any static interface with a gateway will not have a default route due to that, until we fix it. 2.4 isn't meant to be used in production or for more than lab testing at the moment.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • B
            bimmerdriver
            last edited by

            @jimp:

            In the first post of the thread you said the one that wasn't working was 2.3.2_1, or did I misread that?

            If it fails on 2.4 with a tunnel, then you're probably hitting this:
            https://redmine.pfsense.org/issues/6828

            Any static interface with a gateway will not have a default route due to that, until we fix it. 2.4 isn't meant to be used in production or for more than lab testing at the moment.

            Sorry if my original post wasn't clear. My "production" system is running 2.3 (currently 2.3.2_1) and it uses an HE tunnel for ipv6. This system has been working fine for months. I also have a test system which is running 2.4 and it is providing native ipv6. (I have to use 2.4 because the "dhcp6 before RA" fix isn't in the release version yet.) Recently I found that it's not possible to access mail.yahoo.com from the 2.3 system (on any host), but it was working on the test system. I tried tuning MTU to see if it would help, but it didn't make any difference. I'm trying to set up a 2.4 system using a tunnel to confirm that the problem is being caused by the tunnel, not pfsense.

            The bug report sounds like exactly the problem.

            1 Reply Last reply Reply Quote 0
            • JKnottJ
              JKnott
              last edited by

              @bimmerdriver:

              I ran the ICSI / UC Berkeley Netalyzer. The DNS portion of the result is attached below. For no reason in particular, pfsense is configured to use the forwarder. Considering the result, should I try switching to the resolver to see if the results change?

              When I was using a tunnel, it was set to the minimum IPv6 MTU of 1280.  That should work with anything.  Once that's verified to work, you can try larger MTUs.

              PfSense running on Qotom mini PC
              i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
              UniFi AC-Lite access point

              I haven't lost my mind. It's around here...somewhere...

              1 Reply Last reply Reply Quote 0
              • B
                bimmerdriver
                last edited by

                @JKnott:

                @bimmerdriver:

                I ran the ICSI / UC Berkeley Netalyzer. The DNS portion of the result is attached below. For no reason in particular, pfsense is configured to use the forwarder. Considering the result, should I try switching to the resolver to see if the results change?

                When I was using a tunnel, it was set to the minimum IPv6 MTU of 1280.  That should work with anything.  Once that's verified to work, you can try larger MTUs.

                As I said above, I tried lowering MTU all the way down to 1280 and it made no difference. (I was changing the setting of the tunnel and the opt1 interface.) Currently, the tunnel is set to the default MTU, which is 1480, and the opt1 interface is left blank. You can see below that one of the routes has an MTU of 1280. Not sure where that came from.

                IPv6 Routes
                Destination, Gateway, Flags, Use, Mtu, Netif
                default, 2001:xxx:c:211::1, UGS, 45566, 1500, gif0
                ::1, link#4, UH, 0, 16384, lo0
                2001:xxx:c:211::1, link#7, UH, 318447, 1280, gif0
                2001:xxx:c:211::2, link#7, UHS, 0, 16384, lo0
                2001:xxx:d:211::/64, link#5, U, 150524, 1500, hn0
                2001:xxx:d:211::1, link#5, UHS, 1, 16384, lo0

                Also, when I used the iea-software.com mtupath.exe utility, it showed 1500 to google.com and 1480 to mail.yahoo.com using ipv6, so I'm not convinced mtu has anything to do with this issue.

                1 Reply Last reply Reply Quote 0
                • JKnottJ
                  JKnott
                  last edited by

                  ^^^^
                  I read your later comments after I posted that.  Sorry.

                  PfSense running on Qotom mini PC
                  i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
                  UniFi AC-Lite access point

                  I haven't lost my mind. It's around here...somewhere...

                  1 Reply Last reply Reply Quote 0
                  • B
                    bimmerdriver
                    last edited by

                    @JKnott:

                    ^^^^
                    I read your later comments after I posted that.  Sorry.

                    No worries.

                    1 Reply Last reply Reply Quote 0
                    • B
                      bimmerdriver
                      last edited by

                      Does anyone know why the bold route has an MTU of 1280? Where is it defined? The destination of that route is the server end of the tunnel. OPT1 is currently configured with the default MTU.

                      IPv6 Routes
                      Destination, Gateway, Flags, Use, Mtu, Netif
                      default, 2001:xxx:c:211::1, UGS, 45566, 1500, gif0
                      ::1, link#4, UH, 0, 16384, lo0
                      2001:xxx:c:211::1, link#7, UH, 318447, 1280, gif0
                      2001:xxx:c:211::2, link#7, UHS, 0, 16384, lo0
                      2001:xxx:d:211::/64, link#5, U, 150524, 1500, hn0
                      2001:xxx:d:211::1, link#5, UHS, 1, 16384, lo0

                      I did some testing using ping -6. I found that the largest size for mail.yahoo.com is 1432 and the largest size for google.com is 1452. Does that imply the MSS is 1432 and 1452, respectively? I ran mtupath and it reported 1452 MSS for both, suggesting 1500 "estimated MTU". Not sure why there's a discrepancy. What should I add to the MSS to get the MTU?

                      1 Reply Last reply Reply Quote 0
                      • B
                        bimmerdriver
                        last edited by

                        I did some additional testing and I'd appreciate some feedback. I can access mail.yahoo.com on my test pfsense system, running 2.4 with native ipv6. I can also access it using a host through a mullvad vpn (dual-stack) terminating in the USA. I cannot access it using my "production" pfsense system, running 2.3.2_1 and with an HE tunnel.

                        I noticed when I looked at the route table that the route to the server end of the tunnel had mtu of 1280 so I set the mtu of opt1 and the tunnel to 1280. It did not fix the problem. I still can't load mail.yahoo.com. Is there a way to change the MTU? I'm not clear how it got set to 1280 in the first place, because the MTU of opt1 was originally left at the default.

                        I also did a ping test to mail.yahoo.com and google.com using the 1280 mtu setting. I can ping -6 both of them up to 1232 bytes.

                        I've attached the ipv6 routes, opt1 configuration, tracert -6 mail.yahoo.com and the results of the dns and ipv6 sections of the netalyzer test. What's interesting is that now netalyzr is now saying the bottleneck is within the HE network, at 2001:470:0:9d::2. Does that indicate a problem in the HE network? I don't even know if MTU is the cause of this problem or whether it's dns (or something else).

                        Routes.PNG
                        Routes.PNG_thumb
                        OPT1.PNG
                        OPT1.PNG_thumb
                        DNS.PNG
                        DNS.PNG_thumb
                        ipv6.PNG
                        ipv6.PNG_thumb
                        tracert.PNG
                        tracert.PNG_thumb

                        1 Reply Last reply Reply Quote 0
                        • R
                          rdrcrmatt
                          last edited by

                          I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).

                          I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)

                          [2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
                          PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                          ^C
                          --- 5alxq.x.incapdns.net ping6 statistics ---
                          2 packets transmitted, 0 packets received, 100.0% packet loss
                          [2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
                          PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                          1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
                          1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
                          ^C
                          --- 5alxq.x.incapdns.net ping6 statistics ---
                          2 packets transmitted, 2 packets received, 0.0% packet loss
                          round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
                          [2.3.2-RELEASE][root@router]/root:

                          I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.

                          What is causing the requirement for the MTU to be set so incredibly low?  I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low.  I'm wondering if there's anything I can do to fix this.

                          1 Reply Last reply Reply Quote 0
                          • B
                            bimmerdriver
                            last edited by

                            @rdrcrmatt:

                            I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).

                            I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)

                            [2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
                            PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                            ^C
                            --- 5alxq.x.incapdns.net ping6 statistics ---
                            2 packets transmitted, 0 packets received, 100.0% packet loss
                            [2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
                            PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                            1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
                            1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
                            ^C
                            --- 5alxq.x.incapdns.net ping6 statistics ---
                            2 packets transmitted, 2 packets received, 0.0% packet loss
                            round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
                            [2.3.2-RELEASE][root@router]/root:

                            I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.

                            What is causing the requirement for the MTU to be set so incredibly low?  I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low.  I'm wondering if there's anything I can do to fix this.

                            Sorry to hear you are also experiencing this problem, although at least it's confirmation that something is wrong. After testing with ping and mtupath, I'm not convinced this is an MTU issue. Both methods of testing indicate the MTU is 1480. I manually configured the routes to 1480, since pfsense sets them to 1280. It made no difference.

                            However, I read the article "Evaluating IPv4 and IPv6 Packet Fragmentation" from RIPE on path fragmentation: https://labs.ripe.net/Members/gih/evaluating-ipv4-and-ipv6-packet-fragmentation and the behaviour really does seem like a result of path fragmentation. Considering that this problem arose with no changes to pfsense or the tunnelbroker service (according to Hurricane Electric) and it stayed after I upgraded to 2.3.2_1 and replaced the tunnel, maybe something changed in yahoo's network.

                            1 Reply Last reply Reply Quote 0
                            • johnpozJ
                              johnpoz LAYER 8 Global Moderator
                              last edited by

                              what is exactly not working with mail.yahoo.com?  Are you trying to send smtp?  pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue.  Hit that via http/https?

                              [2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
                              PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
                              1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
                              1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
                              1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
                              1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
                              ^C

                              Using my HE tunnel.

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                              1 Reply Last reply Reply Quote 0
                              • B
                                bimmerdriver
                                last edited by

                                @johnpoz:

                                what is exactly not working with mail.yahoo.com?  Are you trying to send smtp?  pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue.  Hit that via http/https?

                                [2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
                                PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
                                1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
                                1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
                                1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
                                1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
                                ^C

                                Using my HE tunnel.

                                I can ping and traceroute -4 -6 no problem, but if I try to open the website, it times out. If I disable ipv6 on pfsense or on the pc, it works fine. It makes no difference whether I'm using IE11, edge or chrome on a pc. Also doesn't work using chrome, gmail app or yahoo mail app on an android phone unless ipv6 is disabled in pfsense. This problem started just over a month ago. I think the problem seems to effect mail.yahoo.com, *.mail.yahoo.com and login.yahoo.com. I've exchanged email with hurricane electric, but they have no idea what could be causing it. If I do a traceroute, only 2/10 hops are on he.net. Hops 4-10 are on yahoo.com. I don't have this problem using my dual-stack vpn or using dual-stack native ipv6 with pfsense 2.4. The problem is only when the tunnel is enabled.

                                1 Reply Last reply Reply Quote 0
                                • B
                                  bimmerdriver
                                  last edited by

                                  I posted about this again on the tunnelbroker forum. The response was it works for me, so it must be software (i.e., pfsense). Waiting for the fix to bug 5993 so I can finally switch everything over to native ipv6…

                                  1 Reply Last reply Reply Quote 0
                                  • johnpozJ
                                    johnpoz LAYER 8 Global Moderator
                                    last edited by

                                    Well clearly is NOT pfsense because I am using pfsense with a HE tunnel.  And not having any issues connecting to yahoo mail.  I do not use it - but I do have an account from years back for something.  So Just logged in via ipv6.

                                    You can see clearly browser showing its connected via ipv6.  Do you want me to disable ipv4 on the machine and check it that way as well?

                                    edit:  So here I disabled ipv4 on my client.  Still accessing it, only connectivity on my client is ipv6 using HE tunnel through pfsense..

                                    yahooipv6.png
                                    yahooipv6.png_thumb
                                    ipv6onlyyahoo.png
                                    ipv6onlyyahoo.png_thumb

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      bimmerdriver
                                      last edited by

                                      Thanks for posting. It confirms what I thought, which is that there is no problem with pfsense. If it's not the clients, pfsense or the tunnel, all that's left is yahoo. It's probably a yahoo issue, but they have no customer support so there little or no chance they will look into this or do anything. I guess the only choice I have is to switch over to 2.4 even though it's not fully baked. Or is it possible to block access to mail.yahoo.com using ipv6 in the firewall?

                                      1 Reply Last reply Reply Quote 0
                                      • DerelictD
                                        Derelict LAYER 8 Netgate
                                        last edited by

                                        FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?

                                        Chattanooga, Tennessee, USA
                                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                        1 Reply Last reply Reply Quote 0
                                        • B
                                          bimmerdriver
                                          last edited by

                                          @Derelict:

                                          FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?

                                          Haha, I hear you. I have several yahoo mail users complaining about this. Unfortunately, this is one of those examples of "old habits die hard."

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.