HE ipv6 gateway doesn't come online

krisken

sorted on PKT


pfTop: Up State 1-108/108, View: default, Order: packets
PR    D SRC                   DEST                 STATE   AGE   EXP  PKTS BYTES
ipv6  O 83.101.6.45:0         216.66.80.26:0        2:2   5058    59 20111 1649K
icmp  O 83.101.6.45:57110     83.101.6.1:0          0:0   5058     9 10064  629K
icmp  O 91.182.30.186:57110   91.182.30.1:0         0:0   5058     9 10064  629K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5058     9  5032  314K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5055     9  1686  105K

sorted on bytes


pfTop: Up State 1-95/95, View: default, Order: bytes
PR    D SRC                   DEST                 STATE   AGE   EXP  PKTS BYTES
ipv6  O 83.101.6.45:0         216.66.80.26:0        2:2   5121    60 20363 1670K
icmp  O 83.101.6.45:57110     83.101.6.1:0          0:0   5121    10 10190  636K
icmp  O 91.182.30.186:57110   91.182.30.1:0         0:0   5121    10 10190  636K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5121    10  5095  318K

Because of the change to another ipv6 subnet, these are my new ipv6 details of HE:
Server IPv4 address: 216.66.80.26
Server IPv6 address: 2001:470:1f08:170b::1/64
Client IPv4 address: 83.101.6.45
Client IPv6 address: 2001:470:1f08:170b::2/64

And my new pfsense config with the HE values : http://krisken.dommel.be/ipv6/pfsense-config.txt

Because i don't have a working ipv6 tunnel, i can't ping6 anything (i suppose?).
I've verified everything using the how-to that i've posted in my first post, and after that i've rebooted pfsense completely. Belgacom will get another ipv4 address but dommel don't : this is a fixed ip. So normally i don't have to adjust that with HE.

UnderCover

i have the same issue when i follow the manual…any ideas?

wallabybob

krisken: if I ping6 the server end of your IPv6 tunnel I get a response. When I did a ping6 to the client end of your IPv6 tunnel I didn't see any response within about 10 seconds (at which point I stopped waiting).

pftop output: Because there is no distinction made between Tx and Rx its not clear if the traffic is truly a conversation or just a monologue. But, there is clearly some activity which is a positive.

Suggest you do pfSense shell command: # tcpdump -i <if>host 216.66.80.26</if> (where you replace by the FreeBSD name of the interface out which the tunnel traffic should go, in my case ppoe1). On my system I see a ICMP6 echo request from my system and ICMP6 echo reply from the HE server about every second. I'm guessing apinger generates this traffic.

tcpdump -i pppoe1 -n host 72.52.104.74

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe1, link-type NULL (BSD loopback), capture size 96 bytes
10:32:14.072734 IP my-IPv4 > HE-IPv4: my-IPv6 > HE-IPv6: ICMP6, echo request, seq 43541, length 24
10:32:14.272761 IP HE-IPv4 > my-IPv4: IP6 HE-IPv6 > my-IPv6: ICMP6, echo reply, seq 43541, length 24

If you don't see the requests, check your apinger configuration.

if you don't see the replies, ask your ISPs if they filter out IPv6 in IPv4 tunnel traffic either generally, or on your link?

jimp

I'm seeing a similar issue as well, still trying to track it down, though. In my case the HE.net gateway also never comes online.

A packet capture of the IPv4 traffic on my WAN shows packets going both ways, wireshark shows them to be the encapsulated IPv6 pings from me, and the replies from the far side. However, the ping replies are never seen on the gif interface. A packet capture on gif0 only shows the requests leaving, never coming back.

I see the state, and it shows up as:

ipv6-icmp 	2001:470:xxxx:yyy::2 -> 2001:470:xxxx:yyy::1 	NO_TRAFFIC:NO_TRAFFIC

Even though there is constant outbound traffic. Traffic doesn't appear to be blocked by firewall rules either, as nothing is logged. It's as if the return traffic just never makes it into the gif interface.

I show an ipv6 also:

ipv6 	my.wan.ip.addr -> 209.51.181.2 	MULTIPLE:MULTIPLE

Packets on the gif interface:

: tcpdump -i gif0
tcpdump: WARNING: gif0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on gif0, link-type NULL (BSD loopback), capture size 96 bytes
19:23:01.272308 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 15619, length 24
19:23:02.277803 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 15875, length 24
19:23:03.284616 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 16131, length 24

Packets on the v4 side:

: tcpdump -ni em2 host 209.51.181.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em2, link-type EN10MB (Ethernet), capture size 96 bytes
19:27:52.633747 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24324, length 24
19:27:52.654180 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24324, length 24
19:27:53.639518 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24580, length 24
19:27:53.662861 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24580, length 24
19:27:54.644236 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24836, length 24
19:27:54.675180 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24836, length 24
19:27:55.644757 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 25092, length 24
19:27:55.676139 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 25092, length 24

This may or may not be expected, but I can't ping6 myself on the gif interface either, but I can ping6 my own LAN interface of course.

: ping6 2001:470:xxxx.yyy::2
PING6(56=40+8+8 bytes) 2001:470:xxxx.yyy::2 --> 2001:470:xxxx.yyy::2
--- 2001:470:xxxx.yyy::2 ping6 statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

: ping6 2001:470:zzzz.yyy::1
PING6(56=40+8+8 bytes) 2001:470:zzzz.yyy::1 --> 2001:470:zzzz.yyy::1
16 bytes from 2001:470:zzzz.yyy::1, icmp_seq=0 hlim=64 time=4.259 ms
16 bytes from 2001:470:zzzz.yyy::1, icmp_seq=1 hlim=64 time=2.336 ms
--- 2001:470:zzzz:yyy::1 ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 2.336/3.298/4.259/0.962 ms

Some other info…

: ifconfig gif0
gif0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
        tunnel inet my.wan.ip.addr --> 209.51.181.2
        inet6 2001:470:xxxx.yyy::2 --> 2001:470:xxxx.yyy::1 prefixlen 128 
        inet6 fe80::240:48ff:feb2:8216%gif0 prefixlen 64 scopeid 0x10 
        nd6 options=3 <performnud,accept_rtadv>options=1 <accept_rev_ethip_ver>: netstat -rn -f inet6  (Edited for brevity)
Routing tables

Internet6:
Destination                       Gateway                       Flags      Netif Expire
default                           2001:470:xxxx.yyy::1          UGS        gif0
2001:470:xxxx.yyy::1              2001:470:xxxx.yyy::2          UH         gif0
2001:470:zzzz.yyy::/64            link#1                        U           em0
2001:470:zzzz.yyy::1              link#1                        UHS         lo0</accept_rev_ethip_ver></performnud,accept_rtadv></up,pointopoint,running,multicast>

EDIT: And FYI, a reboot doesn't change anything.

databeestje

Are all the people here affected when the WAN is a PPPoE connection? I've never seen this particular behaviour on static addressing lines.

Did people also add a allow ipv6 icmp for the WAN address of the gif interface? Might be related. I happen to have one, but my internet is cable internet from Ziggo.

Another hunch, if this is DSL and I also see mention of the sagem modems is that they might not pass the protocol 41 over the PPPoE connection. That would mean it's possible to troubleshoot if another dsl modem is available.

I believe a number of the SpeedTouch modems in bridge modem also have this particular issue.

jimp

I have it running over my Cable connection (opt1) which is DHCP.

I see the return traffic coming back on the physical interface so it doesn't appear to be getting blocked by my ISP or equipment, the OS sees it, it just doesn't make it back onto the gif interface for whatever weird reason.

I also have an IPv6 allow for icmp, both on the real WAN and on the gif interface. I'll try some more tests though. I wouldn't think it would be a firewall rule though because it's return traffic, the state should be letting it back in.

jimp

OK, for giggles I reset my default route to my Cable line, opt1, and it came up!

Then I reset it back to my WAN (DSL) and it went back down.

Then I added a static route over my cable to the he.net tunnel broker and it stayed up!

So it appears we might need some code to add a static route to the gif endpoint based off the interface chosen when making the gif. That may be a general 2.0 bug.

databeestje

Don't we all love multiwan when it works.

I'll add code that adds a route for gif tunnel endpoints for the interface they belong on. Perhaps a filter rule could theoretically help.

Adding a filter rule on the correct wan that has reply-to set for that traffic, would that work? Maybe the traffic is stateless and then the state does not match.

jimp

Not sure if I rule with reply-to would be sufficient or not. I can try it, but I'd need to figure out how that rule would look to match just the gif traffic without allowing being too permissive.

vinsomething

I've seen similar weirdness (and different symptoms at one point - I was seeing no return traffic on the physical interface until I added a default IPv6 gateway) but I haven't spent enough time isolating.

See my post here for what I was seeing (I was trying to replace my HE tunnel with a SIXXS one on my Cable/DHCP interface which worked once I set a default GW):
https://www.sixxs.net/forum/?msg=setup-4100929

Yesterday I couldn't get my other HE tunnel up on my PPPoE DSL WAN interface, but I haven't invested too much time into troubleshooting.

Let me know if I can help by providing logs or testing anything.

databeestje

Just do what Jim did. Add a static route for the IPv4 address of the tunnelbroker on the correct WAN interface.

vinsomething

Got it. I also had to upgrade the firmware on the DSL modem to something from the last 5 years :)

I don't think this is quite what I was seeing a couple weeks ago as I was only trying to bring up a single tunnel on my primary WAN at the time (and by adding a v6 default gateway is what kicked it into gear)…... but it's working now the way I want to so I'm happy.

databeestje

I've found that the default route might dissapear when the parent interface goes down.

I have not debugged that.