Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HE Tunnel Problem

    Scheduled Pinned Locked Moved IPv6
    27 Posts 6 Posters 6.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JKnottJ
      JKnott
      last edited by

      ^^^^
      I read your later comments after I posted that.  Sorry.

      PfSense running on Qotom mini PC
      i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
      UniFi AC-Lite access point

      I haven't lost my mind. It's around here...somewhere...

      1 Reply Last reply Reply Quote 0
      • B
        bimmerdriver
        last edited by

        @JKnott:

        ^^^^
        I read your later comments after I posted that.  Sorry.

        No worries.

        1 Reply Last reply Reply Quote 0
        • B
          bimmerdriver
          last edited by

          Does anyone know why the bold route has an MTU of 1280? Where is it defined? The destination of that route is the server end of the tunnel. OPT1 is currently configured with the default MTU.

          IPv6 Routes
          Destination, Gateway, Flags, Use, Mtu, Netif
          default, 2001:xxx:c:211::1, UGS, 45566, 1500, gif0
          ::1, link#4, UH, 0, 16384, lo0
          2001:xxx:c:211::1, link#7, UH, 318447, 1280, gif0
          2001:xxx:c:211::2, link#7, UHS, 0, 16384, lo0
          2001:xxx:d:211::/64, link#5, U, 150524, 1500, hn0
          2001:xxx:d:211::1, link#5, UHS, 1, 16384, lo0

          I did some testing using ping -6. I found that the largest size for mail.yahoo.com is 1432 and the largest size for google.com is 1452. Does that imply the MSS is 1432 and 1452, respectively? I ran mtupath and it reported 1452 MSS for both, suggesting 1500 "estimated MTU". Not sure why there's a discrepancy. What should I add to the MSS to get the MTU?

          1 Reply Last reply Reply Quote 0
          • B
            bimmerdriver
            last edited by

            I did some additional testing and I'd appreciate some feedback. I can access mail.yahoo.com on my test pfsense system, running 2.4 with native ipv6. I can also access it using a host through a mullvad vpn (dual-stack) terminating in the USA. I cannot access it using my "production" pfsense system, running 2.3.2_1 and with an HE tunnel.

            I noticed when I looked at the route table that the route to the server end of the tunnel had mtu of 1280 so I set the mtu of opt1 and the tunnel to 1280. It did not fix the problem. I still can't load mail.yahoo.com. Is there a way to change the MTU? I'm not clear how it got set to 1280 in the first place, because the MTU of opt1 was originally left at the default.

            I also did a ping test to mail.yahoo.com and google.com using the 1280 mtu setting. I can ping -6 both of them up to 1232 bytes.

            I've attached the ipv6 routes, opt1 configuration, tracert -6 mail.yahoo.com and the results of the dns and ipv6 sections of the netalyzer test. What's interesting is that now netalyzr is now saying the bottleneck is within the HE network, at 2001:470:0:9d::2. Does that indicate a problem in the HE network? I don't even know if MTU is the cause of this problem or whether it's dns (or something else).

            Routes.PNG
            Routes.PNG_thumb
            OPT1.PNG
            OPT1.PNG_thumb
            DNS.PNG
            DNS.PNG_thumb
            ipv6.PNG
            ipv6.PNG_thumb
            tracert.PNG
            tracert.PNG_thumb

            1 Reply Last reply Reply Quote 0
            • R
              rdrcrmatt
              last edited by

              I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).

              I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)

              [2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
              PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
              ^C
              --- 5alxq.x.incapdns.net ping6 statistics ---
              2 packets transmitted, 0 packets received, 100.0% packet loss
              [2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
              PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
              1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
              1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
              ^C
              --- 5alxq.x.incapdns.net ping6 statistics ---
              2 packets transmitted, 2 packets received, 0.0% packet loss
              round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
              [2.3.2-RELEASE][root@router]/root:

              I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.

              What is causing the requirement for the MTU to be set so incredibly low?  I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low.  I'm wondering if there's anything I can do to fix this.

              1 Reply Last reply Reply Quote 0
              • B
                bimmerdriver
                last edited by

                @rdrcrmatt:

                I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).

                I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)

                [2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
                PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                ^C
                --- 5alxq.x.incapdns.net ping6 statistics ---
                2 packets transmitted, 0 packets received, 100.0% packet loss
                [2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
                PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
                1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
                1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
                ^C
                --- 5alxq.x.incapdns.net ping6 statistics ---
                2 packets transmitted, 2 packets received, 0.0% packet loss
                round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
                [2.3.2-RELEASE][root@router]/root:

                I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.

                What is causing the requirement for the MTU to be set so incredibly low?  I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low.  I'm wondering if there's anything I can do to fix this.

                Sorry to hear you are also experiencing this problem, although at least it's confirmation that something is wrong. After testing with ping and mtupath, I'm not convinced this is an MTU issue. Both methods of testing indicate the MTU is 1480. I manually configured the routes to 1480, since pfsense sets them to 1280. It made no difference.

                However, I read the article "Evaluating IPv4 and IPv6 Packet Fragmentation" from RIPE on path fragmentation: https://labs.ripe.net/Members/gih/evaluating-ipv4-and-ipv6-packet-fragmentation and the behaviour really does seem like a result of path fragmentation. Considering that this problem arose with no changes to pfsense or the tunnelbroker service (according to Hurricane Electric) and it stayed after I upgraded to 2.3.2_1 and replaced the tunnel, maybe something changed in yahoo's network.

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by

                  what is exactly not working with mail.yahoo.com?  Are you trying to send smtp?  pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue.  Hit that via http/https?

                  [2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
                  PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
                  1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
                  1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
                  1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
                  1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
                  ^C

                  Using my HE tunnel.

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  1 Reply Last reply Reply Quote 0
                  • B
                    bimmerdriver
                    last edited by

                    @johnpoz:

                    what is exactly not working with mail.yahoo.com?  Are you trying to send smtp?  pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue.  Hit that via http/https?

                    [2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
                    PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
                    1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
                    1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
                    1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
                    1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
                    ^C

                    Using my HE tunnel.

                    I can ping and traceroute -4 -6 no problem, but if I try to open the website, it times out. If I disable ipv6 on pfsense or on the pc, it works fine. It makes no difference whether I'm using IE11, edge or chrome on a pc. Also doesn't work using chrome, gmail app or yahoo mail app on an android phone unless ipv6 is disabled in pfsense. This problem started just over a month ago. I think the problem seems to effect mail.yahoo.com, *.mail.yahoo.com and login.yahoo.com. I've exchanged email with hurricane electric, but they have no idea what could be causing it. If I do a traceroute, only 2/10 hops are on he.net. Hops 4-10 are on yahoo.com. I don't have this problem using my dual-stack vpn or using dual-stack native ipv6 with pfsense 2.4. The problem is only when the tunnel is enabled.

                    1 Reply Last reply Reply Quote 0
                    • B
                      bimmerdriver
                      last edited by

                      I posted about this again on the tunnelbroker forum. The response was it works for me, so it must be software (i.e., pfsense). Waiting for the fix to bug 5993 so I can finally switch everything over to native ipv6…

                      1 Reply Last reply Reply Quote 0
                      • johnpozJ
                        johnpoz LAYER 8 Global Moderator
                        last edited by

                        Well clearly is NOT pfsense because I am using pfsense with a HE tunnel.  And not having any issues connecting to yahoo mail.  I do not use it - but I do have an account from years back for something.  So Just logged in via ipv6.

                        You can see clearly browser showing its connected via ipv6.  Do you want me to disable ipv4 on the machine and check it that way as well?

                        edit:  So here I disabled ipv4 on my client.  Still accessing it, only connectivity on my client is ipv6 using HE tunnel through pfsense..

                        yahooipv6.png
                        yahooipv6.png_thumb
                        ipv6onlyyahoo.png
                        ipv6onlyyahoo.png_thumb

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        1 Reply Last reply Reply Quote 0
                        • B
                          bimmerdriver
                          last edited by

                          Thanks for posting. It confirms what I thought, which is that there is no problem with pfsense. If it's not the clients, pfsense or the tunnel, all that's left is yahoo. It's probably a yahoo issue, but they have no customer support so there little or no chance they will look into this or do anything. I guess the only choice I have is to switch over to 2.4 even though it's not fully baked. Or is it possible to block access to mail.yahoo.com using ipv6 in the firewall?

                          1 Reply Last reply Reply Quote 0
                          • DerelictD
                            Derelict LAYER 8 Netgate
                            last edited by

                            FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?

                            Chattanooga, Tennessee, USA
                            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                            1 Reply Last reply Reply Quote 0
                            • B
                              bimmerdriver
                              last edited by

                              @Derelict:

                              FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?

                              Haha, I hear you. I have several yahoo mail users complaining about this. Unfortunately, this is one of those examples of "old habits die hard."

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.