HE Tunnel Problem
-
In the first post of the thread you said the one that wasn't working was 2.3.2_1, or did I misread that?
If it fails on 2.4 with a tunnel, then you're probably hitting this:
https://redmine.pfsense.org/issues/6828Any static interface with a gateway will not have a default route due to that, until we fix it. 2.4 isn't meant to be used in production or for more than lab testing at the moment.
-
In the first post of the thread you said the one that wasn't working was 2.3.2_1, or did I misread that?
If it fails on 2.4 with a tunnel, then you're probably hitting this:
https://redmine.pfsense.org/issues/6828Any static interface with a gateway will not have a default route due to that, until we fix it. 2.4 isn't meant to be used in production or for more than lab testing at the moment.
Sorry if my original post wasn't clear. My "production" system is running 2.3 (currently 2.3.2_1) and it uses an HE tunnel for ipv6. This system has been working fine for months. I also have a test system which is running 2.4 and it is providing native ipv6. (I have to use 2.4 because the "dhcp6 before RA" fix isn't in the release version yet.) Recently I found that it's not possible to access mail.yahoo.com from the 2.3 system (on any host), but it was working on the test system. I tried tuning MTU to see if it would help, but it didn't make any difference. I'm trying to set up a 2.4 system using a tunnel to confirm that the problem is being caused by the tunnel, not pfsense.
The bug report sounds like exactly the problem.
-
I ran the ICSI / UC Berkeley Netalyzer. The DNS portion of the result is attached below. For no reason in particular, pfsense is configured to use the forwarder. Considering the result, should I try switching to the resolver to see if the results change?
When I was using a tunnel, it was set to the minimum IPv6 MTU of 1280. That should work with anything. Once that's verified to work, you can try larger MTUs.
-
I ran the ICSI / UC Berkeley Netalyzer. The DNS portion of the result is attached below. For no reason in particular, pfsense is configured to use the forwarder. Considering the result, should I try switching to the resolver to see if the results change?
When I was using a tunnel, it was set to the minimum IPv6 MTU of 1280. That should work with anything. Once that's verified to work, you can try larger MTUs.
As I said above, I tried lowering MTU all the way down to 1280 and it made no difference. (I was changing the setting of the tunnel and the opt1 interface.) Currently, the tunnel is set to the default MTU, which is 1480, and the opt1 interface is left blank. You can see below that one of the routes has an MTU of 1280. Not sure where that came from.
IPv6 Routes
Destination, Gateway, Flags, Use, Mtu, Netif
default, 2001:xxx:c:211::1, UGS, 45566, 1500, gif0
::1, link#4, UH, 0, 16384, lo0
2001:xxx:c:211::1, link#7, UH, 318447, 1280, gif0
2001:xxx:c:211::2, link#7, UHS, 0, 16384, lo0
2001:xxx:d:211::/64, link#5, U, 150524, 1500, hn0
2001:xxx:d:211::1, link#5, UHS, 1, 16384, lo0Also, when I used the iea-software.com mtupath.exe utility, it showed 1500 to google.com and 1480 to mail.yahoo.com using ipv6, so I'm not convinced mtu has anything to do with this issue.
-
^^^^
I read your later comments after I posted that. Sorry. -
-
Does anyone know why the bold route has an MTU of 1280? Where is it defined? The destination of that route is the server end of the tunnel. OPT1 is currently configured with the default MTU.
IPv6 Routes
Destination, Gateway, Flags, Use, Mtu, Netif
default, 2001:xxx:c:211::1, UGS, 45566, 1500, gif0
::1, link#4, UH, 0, 16384, lo0
2001:xxx:c:211::1, link#7, UH, 318447, 1280, gif0
2001:xxx:c:211::2, link#7, UHS, 0, 16384, lo0
2001:xxx:d:211::/64, link#5, U, 150524, 1500, hn0
2001:xxx:d:211::1, link#5, UHS, 1, 16384, lo0I did some testing using ping -6. I found that the largest size for mail.yahoo.com is 1432 and the largest size for google.com is 1452. Does that imply the MSS is 1432 and 1452, respectively? I ran mtupath and it reported 1452 MSS for both, suggesting 1500 "estimated MTU". Not sure why there's a discrepancy. What should I add to the MSS to get the MTU?
-
I did some additional testing and I'd appreciate some feedback. I can access mail.yahoo.com on my test pfsense system, running 2.4 with native ipv6. I can also access it using a host through a mullvad vpn (dual-stack) terminating in the USA. I cannot access it using my "production" pfsense system, running 2.3.2_1 and with an HE tunnel.
I noticed when I looked at the route table that the route to the server end of the tunnel had mtu of 1280 so I set the mtu of opt1 and the tunnel to 1280. It did not fix the problem. I still can't load mail.yahoo.com. Is there a way to change the MTU? I'm not clear how it got set to 1280 in the first place, because the MTU of opt1 was originally left at the default.
I also did a ping test to mail.yahoo.com and google.com using the 1280 mtu setting. I can ping -6 both of them up to 1232 bytes.
I've attached the ipv6 routes, opt1 configuration, tracert -6 mail.yahoo.com and the results of the dns and ipv6 sections of the netalyzer test. What's interesting is that now netalyzr is now saying the bottleneck is within the HE network, at 2001:470:0:9d::2. Does that indicate a problem in the HE network? I don't even know if MTU is the cause of this problem or whether it's dns (or something else).
-
I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).
I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)
[2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
^C
--- 5alxq.x.incapdns.net ping6 statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
^C
--- 5alxq.x.incapdns.net ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
[2.3.2-RELEASE][root@router]/root:I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.
What is causing the requirement for the MTU to be set so incredibly low? I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low. I'm wondering if there's anything I can do to fix this.
-
I am also having this exact same issue on 2.3.2 (about to upgrade to 2.3.2_1).
I can't get to mail.yahoo.com, kb.vmware.com, my.vmware.com.. and pandora won't stream (though I'm not 100% sure that's related)
[2.3.2-RELEASE][root@router]/root: ping6 -s 1405 -d my.vmware.com
PING6(1453=40+8+1405 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
^C
--- 5alxq.x.incapdns.net ping6 statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[2.3.2-RELEASE][root@router]/root: ping6 -s 1404 -d my.vmware.com
PING6(1452=40+8+1404 bytes) 2001:470:1f10:xxxx::2 –> 2a02:e980:39::13
1412 bytes from 2a02:e980:39::13, icmp_seq=0 hlim=61 time=25.633 ms
1412 bytes from 2a02:e980:39::13, icmp_seq=1 hlim=61 time=26.127 ms
^C
--- 5alxq.x.incapdns.net ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 25.633/25.880/26.127/0.247 ms
[2.3.2-RELEASE][root@router]/root:I just changed my MTU on my IPv6 Tunnel interface to 1280 and I am able to get to the vmware sites now, but still not mail.yahoo.com.
What is causing the requirement for the MTU to be set so incredibly low? I know it's data -> IPv6 -> GRE -> IPv4, so there's a bunch of headers on there, but I've never seen a link that low. I'm wondering if there's anything I can do to fix this.
Sorry to hear you are also experiencing this problem, although at least it's confirmation that something is wrong. After testing with ping and mtupath, I'm not convinced this is an MTU issue. Both methods of testing indicate the MTU is 1480. I manually configured the routes to 1480, since pfsense sets them to 1280. It made no difference.
However, I read the article "Evaluating IPv4 and IPv6 Packet Fragmentation" from RIPE on path fragmentation: https://labs.ripe.net/Members/gih/evaluating-ipv4-and-ipv6-packet-fragmentation and the behaviour really does seem like a result of path fragmentation. Considering that this problem arose with no changes to pfsense or the tunnelbroker service (according to Hurricane Electric) and it stayed after I upgraded to 2.3.2_1 and replaced the tunnel, maybe something changed in yahoo's network.
-
what is exactly not working with mail.yahoo.com? Are you trying to send smtp? pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue. Hit that via http/https?
[2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
^CUsing my HE tunnel.
-
what is exactly not working with mail.yahoo.com? Are you trying to send smtp? pickup mail pop3, imap? I can ping mail.yahoo.com with your 1405 setting without any issue. Hit that via http/https?
[2.3.2-RELEASE][root@pfsense.local.lan]/root: ping6 -s 1405 -d mail.yahoo.com
PING6(1453=40+8+1405 bytes) 2001:470:1f10:9c4::2 –> 2001:4998:44:a10::50
1413 bytes from 2001:4998:44:a10::50, icmp_seq=0 hlim=56 time=22.240 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=1 hlim=56 time=29.972 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=2 hlim=56 time=21.996 ms
1413 bytes from 2001:4998:44:a10::50, icmp_seq=3 hlim=56 time=21.842 ms
^CUsing my HE tunnel.
I can ping and traceroute -4 -6 no problem, but if I try to open the website, it times out. If I disable ipv6 on pfsense or on the pc, it works fine. It makes no difference whether I'm using IE11, edge or chrome on a pc. Also doesn't work using chrome, gmail app or yahoo mail app on an android phone unless ipv6 is disabled in pfsense. This problem started just over a month ago. I think the problem seems to effect mail.yahoo.com, *.mail.yahoo.com and login.yahoo.com. I've exchanged email with hurricane electric, but they have no idea what could be causing it. If I do a traceroute, only 2/10 hops are on he.net. Hops 4-10 are on yahoo.com. I don't have this problem using my dual-stack vpn or using dual-stack native ipv6 with pfsense 2.4. The problem is only when the tunnel is enabled.
-
I posted about this again on the tunnelbroker forum. The response was it works for me, so it must be software (i.e., pfsense). Waiting for the fix to bug 5993 so I can finally switch everything over to native ipv6…
-
Well clearly is NOT pfsense because I am using pfsense with a HE tunnel. And not having any issues connecting to yahoo mail. I do not use it - but I do have an account from years back for something. So Just logged in via ipv6.
You can see clearly browser showing its connected via ipv6. Do you want me to disable ipv4 on the machine and check it that way as well?
edit: So here I disabled ipv4 on my client. Still accessing it, only connectivity on my client is ipv6 using HE tunnel through pfsense..
-
Thanks for posting. It confirms what I thought, which is that there is no problem with pfsense. If it's not the clients, pfsense or the tunnel, all that's left is yahoo. It's probably a yahoo issue, but they have no customer support so there little or no chance they will look into this or do anything. I guess the only choice I have is to switch over to 2.4 even though it's not fully baked. Or is it possible to block access to mail.yahoo.com using ipv6 in the firewall?
-
FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?
-
FWIW mail.yahoo.com hangs for me too on HE tunnel but not on centurylink native. Haven't has time to look at it further and, after all, who needs ANOTHER reason not to use yahoo mail?
Haha, I hear you. I have several yahoo mail users complaining about this. Unfortunately, this is one of those examples of "old habits die hard."
