HE Tunnel will not come back up
-
@a4ehusker
check ipv6 routes eventually post it here
Diagnostics / Routes -
Seems correct:
Internet6: Destination Gateway Flags Netif Expire default 2001:470:CCCC:DDDD::1 UGS gif1 ::1 link#8 UH lo0 2001:470:AAAA:BBBB::1 link#13 UH gif0 2001:470:AAAA:BBBB::2 link#13 UHS lo0 2001:470:CCCC:DDDD::1 link#14 UH gif1 2001:470:CCCC:DDDD::2 link#14 UHS lo0gif0 is the bad tunnel
gif1 is the good tunnel -
@a4ehusker
where is the gateway for gif0? there should be a UGS (UP+Gateway+Static) for gif0 i suppose if it's another tunnel -
@kiokoman I'm not sure - since its down the Gateway Group is set to the one that's up.
But if I force the route for the bad tunnel, it does show up in the Gateway for default... but still doesn't work. I can see all the traffic going out but zero traffic coming in. I'm still not sure who's problem it is, but I do lean HE. But since they said they saw traffic out on their end, I don't know if I have enough to have them check anything else.
-
@a4ehusker
MTU maybe? for my pppoe i have MTU to 1472 and MSS to 1440 for example -
@kiokoman Well this tunnel was working, and the second tunnel is working fine... so I doubt it but I guess it won't hurt to try.
-
@gertjan I've figure out the issue.
I have a modem (Netgear CM1150V) that allows LAGG/LACP connections. It was broken until they released a firmware update, which I noticed shortly after the ping issue with Comcast. Once Comcast fixed my line, I decided to set that up. I had it set up with a prior modem, but never this one due to the firmware bug.
So I undid the LAGG/LACP connection, and just made it failover, and suddenly the HE tunnel came back up! I do not know why it was not working. I'm not sure if the modem has a bug with sending back reply packets - but given that IPv4 works fine otherwise and the tunnel runs over IPv4 I think the issue is in pfSense.
Where do I submit a bug report over this?
(This was driving me crazy because it made no sense, but now it totally does!)
-
@a4ehusker I am having this same issue. HE tunnel works fine on a Netgear CM1200 with LAGG/LACP off. The moment I turn LAGG/LACP on in the Netgear, and then migrate my WAN connection to the LAGG connection on my pfSense box, the HE tunnel drops.
I've ran pfsense continuously since 2011 and HE tunnels since 2009 so pretty familiar with both. I hypothesize the issue is either the modem dropping protocol 41 with LAGG enabled, or a bug in pfSense encapsulating gif/6to4 over LAGG.
I'm running 21.02.2-RELEASE on a Netgate SG-8860. I also swapped the Netgear CM1200 with a Netgear CM1100 - same behavior. Again, Tunnel works fine without LAGG/LACP enabled, then 100% packet loss with LAGG/LACP enabled. I ran packet capture on gif0 and lagg0 and can see the traffic going to Tunnelbroker but zero replies/return traffic from HE.net.
I contacted HE.net support and they don't see any of the traffic coming in.
As soon as I disable LAGG on the modem and on pfsense, with zero other changes and of course no physical layer changes, tunnel pops right back up 0% packet loss no routing issues whatsoever.
Grr……..
-
@akghetto INTERESTING! I'm glad I am not the only one experiencing this.
I use to run a Motorola MB8600 with HE & LAGG without issues, but that was a year and a half ago. So might be the modem (since the commonality is Netgear), or maybe something changed with pfSense.
-
@a4ehusker I opened a bug report with pfsense. Since I had the issue as either pfsense or the Netgear, they closed the bug but gave me a pointer to try and isolate it further. The bug feedback was
Not enough evidence here to conclude that it's a bug in FreeBSD or pfSense. You could test it further by not enabling LAGG on the modem, but setting the pfSense end to use a passive LAGG style such as failover which does not require any special setting on the modem.I followed this advice and zero packet drop on the tunnelbroker gateway. IPv6 tunnel traffic routes correctly, 0% packet loss over the tunnel. So, strongly suspecting the Netgear as the culpret.
Since I bought this modem brand new just two weeks ago it comes with 90 days of support. I've opened a ticket tonight with Netgear along with all my tcpdumps and troubleshooting steps, including isolating it to the modem. I'll let you know what develops.
-
@akghetto Awesome! I wonder if not using a LACP type connection on pfSense would eliminate the issue. Still, as I said I've used a LAGG with a different vender so kinda figured it was something with Netgear.
My modem is a year old (it took them that long to fix the bug where a LAGG connection would freeze after 24 hours). Hope you get some answers!
-
@a4ehusker Well, Netgate support basically stinks. They called me over the phone to confirm the problem, said they'd follow-up, and never did. I'm at my 30-day return window tomorrow so I'll be sending this back to Amazon as defective. Stinkage.
-
@akghetto Ahhhhh yuck. That sucks, but honestly not surprising. Well at least others will have an answer here about what not to do.
-
Netgate or Netgear.. What did you buy from netgate - seems from reading this thread you bought a netgear modem?
I don't see how you would of gotten a 8860 recently?? Via amazon?
Confused..
-
@johnpoz My 8860 I purchased directly from Netgate in 2017. I love it. It's a champ, and my next router will be another Netgate.
The new addition is a Netgear CM1200 cable modem, which I was trying to connect to the Netgate 8860 in a LAGG/LACP configuration due to the cable modem provider provisioning the cable modem with speeds greater than gigabit. LAGG/LACP between the Netgate and Netgear works fine for all things IPv4, however there is some issue when it comes to 6to4/protocol 41 in this configuration that absolutely zero 6to4 traffic makes it past the modem.
I opened a pfsense redmine report on the issue but there was insufficient evidence to definitely tie it to either the Netgate nor the Netgear, though some advice was given to rule out Netgate as the likely culprit which I took and agree it is probably not Netgate and is likely the Netgear CM1200. I have extensive interface tcpdumps and trouble-shooting steps, inclusion of the 6to4 tunnel provider's (HE, which is why I'm in this thread) higher tier support such that I can say its one or the other (Netgear or Netgate). I an also reliably reproduce the issue and its resolution.
I may try another manufacturer's cable modem and the same LAGG/LACP in a month or so and will update this thread if I do, though I am not certain as I may be looking to replace the cable modem provider with a fiber to the home solution later this year so the benefit for LAGG/LACP is becoming less. If I do try another manufacturer's modem though that would definitively determine if the LAGG/LACP issue is Netgate/pfSense or Netgear (as I suspect).
-
@akghetto My question is why exactly are you saying netgate support stinks? Did they not call you back - or was that netgear?
-
@johnpoz Argh! Now I see what you were referring to. I just thought you were generally wanting to confirm if I was referring to Netgear or Netgate.
NETGEAR support stinks. I never reached out to NETGATE because I don't think the problem is the 8860 or pfSense. Ugh....
I tried to edit the post but says too much time has elapsed. Can you edit, John?
Netgate support is great the few times I have reached out!!! :P
