HE ipv6 gateway doesn't come online

krisken

At the dashboard, i can see the status of the gateways:

DOMMEL 83.101.6.1 17.614ms 0.0% Online
HE_IPV6 2001:470:1f08:170b::1 0.000ms 100.0% Offline
BELGACOM 91.182.30.1 21.449ms 0.0% Online

Dommel and Belgacom are both VDSL connections using a sagem modem/router. I've disabled the router options so that both modems are just used as modem. The PPPOE session for both connections is done by pfsense itself. Belgacom has a variable ipv4 ip (change every 36 hours), the dommel line has a fixed ip (i always get the same ip when i set up a PPPOE connection).

IPV4 icmp packs can be send to the ipv4 ip address of my dommel connection. You can verify this by pinging office.it2go.eu or the IP itself. This is what i've put in the firewall

Firewall >> Rules >> Dommel
Proto:IPv4 ICMP echoreq
Source: *
Port: *
Destination: DOMMEL address
Port: *
Gateway: *
Queue: none
Description: ICMP rule

So i suppose that there have to be something wrong with the config settings that i've changed…

wallabybob

I have not checked to see what HE send me. My firewall rules are more open than your firewall rule in that I allow ICMP any while you allow only ICMP echo req.

Suggest you reset our IP address in the HE tunnel parameters, then a minute or two later check the firewall log (Status -> System Logs, Firewall tab) to see if something from HE has been blocked. If so, adjust your rules accordingly.

Oh and you know that after major firewall rules changes it is necessary to reset firewall states if you want the rules activated. (See Diagnostics -> States, click on Reset States tab.)

For comparison, my HE tunnel shows up in Dashboard -> Gateways as:

HE_NET 2001:470:pqrs:wxyz::1 223.387ms 0.0% Warning, Latency

krisken

I've changed the firewall rule from ICMP echo to ICMP any.
After reset the states, i can't mention the ip 216.66.80.26 (tunnelbroker IP) as blocked in the firewall.
When going to diagnostics - states, i can see the line

ipv6 83.101.6.45 -> 216.66.80.26 MULTIPLE:MULTIPLE

So i suppose that the firewall rules are correct?

johnpoz

Something is not right here

you post
Server IPv6 address: 2001:470:1f12:753::1/64

But in your gateway widgit your showing this?

HE_IPV6 2001:470:1f08:170b::1

Well thats not your gateway from you posted your tunnel info was.

the
Server IPv6 address: 2001:470:1f12:753::1/64

Should be your gateway as listed on pfsense.. Where are you geting that 1f08:170b from??

krisken

Everything is right, but a few hours ago i got recommended to ask HE for a new ipv6 ip space. Didn't work as well…

wallabybob

@krisken:

Everything is right, but a few hours ago i got recommended to ask HE for a new ipv6 ip space. Didn't work as well…

Guess you should ask for another one then :)

@krisken:

IWhen going to diagnostics - states, i can see the line

ipv6 83.101.6.45 -> 216.66.80.26 MULTIPLE:MULTIPLE

I have a similar line in the output of the pftop command. And the PKTS and BYTES columns change if I watch for a while (currently 459078 38564226 respectively).

If you have non-zero values do you get a response when you ping6 the IPv6 server address?

At this stage it could be helpful if you stuck with one set of tunnel parameters, posted them here and verified you have them correctly set in pfSense. It might also be a good idea to reboot your pfSense to attempt to make sure everything gets correctly set to whatever set of parameters you have decided to use. Of course the reboot might cause you to get assigned a new IP v4 address so you will have to adjust that with HE.

If your ping6 succeeds you might need to edit the IPv6 default gateway as I described in an earlier reply.

krisken

sorted on PKT


pfTop: Up State 1-108/108, View: default, Order: packets
PR    D SRC                   DEST                 STATE   AGE   EXP  PKTS BYTES
ipv6  O 83.101.6.45:0         216.66.80.26:0        2:2   5058    59 20111 1649K
icmp  O 83.101.6.45:57110     83.101.6.1:0          0:0   5058     9 10064  629K
icmp  O 91.182.30.186:57110   91.182.30.1:0         0:0   5058     9 10064  629K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5058     9  5032  314K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5055     9  1686  105K

sorted on bytes


pfTop: Up State 1-95/95, View: default, Order: bytes
PR    D SRC                   DEST                 STATE   AGE   EXP  PKTS BYTES
ipv6  O 83.101.6.45:0         216.66.80.26:0        2:2   5121    60 20363 1670K
icmp  O 83.101.6.45:57110     83.101.6.1:0          0:0   5121    10 10190  636K
icmp  O 91.182.30.186:57110   91.182.30.1:0         0:0   5121    10 10190  636K
ipv6- O 2001:470:1f08:170b::2 2001:470:1f08:170b::  0:0   5121    10  5095  318K

Because of the change to another ipv6 subnet, these are my new ipv6 details of HE:
Server IPv4 address: 216.66.80.26
Server IPv6 address: 2001:470:1f08:170b::1/64
Client IPv4 address: 83.101.6.45
Client IPv6 address: 2001:470:1f08:170b::2/64

And my new pfsense config with the HE values : http://krisken.dommel.be/ipv6/pfsense-config.txt

Because i don't have a working ipv6 tunnel, i can't ping6 anything (i suppose?).
I've verified everything using the how-to that i've posted in my first post, and after that i've rebooted pfsense completely. Belgacom will get another ipv4 address but dommel don't : this is a fixed ip. So normally i don't have to adjust that with HE.

UnderCover

i have the same issue when i follow the manual…any ideas?

wallabybob

krisken: if I ping6 the server end of your IPv6 tunnel I get a response. When I did a ping6 to the client end of your IPv6 tunnel I didn't see any response within about 10 seconds (at which point I stopped waiting).

pftop output: Because there is no distinction made between Tx and Rx its not clear if the traffic is truly a conversation or just a monologue. But, there is clearly some activity which is a positive.

Suggest you do pfSense shell command: # tcpdump -i <if>host 216.66.80.26</if> (where you replace by the FreeBSD name of the interface out which the tunnel traffic should go, in my case ppoe1). On my system I see a ICMP6 echo request from my system and ICMP6 echo reply from the HE server about every second. I'm guessing apinger generates this traffic.

tcpdump -i pppoe1 -n host 72.52.104.74

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe1, link-type NULL (BSD loopback), capture size 96 bytes
10:32:14.072734 IP my-IPv4 > HE-IPv4: my-IPv6 > HE-IPv6: ICMP6, echo request, seq 43541, length 24
10:32:14.272761 IP HE-IPv4 > my-IPv4: IP6 HE-IPv6 > my-IPv6: ICMP6, echo reply, seq 43541, length 24

If you don't see the requests, check your apinger configuration.

if you don't see the replies, ask your ISPs if they filter out IPv6 in IPv4 tunnel traffic either generally, or on your link?

jimp

I'm seeing a similar issue as well, still trying to track it down, though. In my case the HE.net gateway also never comes online.

A packet capture of the IPv4 traffic on my WAN shows packets going both ways, wireshark shows them to be the encapsulated IPv6 pings from me, and the replies from the far side. However, the ping replies are never seen on the gif interface. A packet capture on gif0 only shows the requests leaving, never coming back.

I see the state, and it shows up as:

ipv6-icmp 	2001:470:xxxx:yyy::2 -> 2001:470:xxxx:yyy::1 	NO_TRAFFIC:NO_TRAFFIC

Even though there is constant outbound traffic. Traffic doesn't appear to be blocked by firewall rules either, as nothing is logged. It's as if the return traffic just never makes it into the gif interface.

I show an ipv6 also:

ipv6 	my.wan.ip.addr -> 209.51.181.2 	MULTIPLE:MULTIPLE

Packets on the gif interface:

: tcpdump -i gif0
tcpdump: WARNING: gif0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on gif0, link-type NULL (BSD loopback), capture size 96 bytes
19:23:01.272308 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 15619, length 24
19:23:02.277803 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 15875, length 24
19:23:03.284616 IP6 2001:470:xxxx:yyy::2 > 2001:470:xxx:yyy::1: ICMP6, echo request, seq 16131, length 24

Packets on the v4 side:

: tcpdump -ni em2 host 209.51.181.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em2, link-type EN10MB (Ethernet), capture size 96 bytes
19:27:52.633747 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24324, length 24
19:27:52.654180 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24324, length 24
19:27:53.639518 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24580, length 24
19:27:53.662861 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24580, length 24
19:27:54.644236 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 24836, length 24
19:27:54.675180 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 24836, length 24
19:27:55.644757 IP my.wan.ip.addr > 209.51.181.2: IP6 2001:470:xxxx.yyy::2 > 2001:470:xxxx.yyy::1: ICMP6, echo request, seq 25092, length 24
19:27:55.676139 IP 209.51.181.2 > my.wan.ip.addr: IP6 2001:470:xxxx.yyy::1 > 2001:470:xxxx.yyy::2: ICMP6, echo reply, seq 25092, length 24

This may or may not be expected, but I can't ping6 myself on the gif interface either, but I can ping6 my own LAN interface of course.

: ping6 2001:470:xxxx.yyy::2
PING6(56=40+8+8 bytes) 2001:470:xxxx.yyy::2 --> 2001:470:xxxx.yyy::2
--- 2001:470:xxxx.yyy::2 ping6 statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

: ping6 2001:470:zzzz.yyy::1
PING6(56=40+8+8 bytes) 2001:470:zzzz.yyy::1 --> 2001:470:zzzz.yyy::1
16 bytes from 2001:470:zzzz.yyy::1, icmp_seq=0 hlim=64 time=4.259 ms
16 bytes from 2001:470:zzzz.yyy::1, icmp_seq=1 hlim=64 time=2.336 ms
--- 2001:470:zzzz:yyy::1 ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 2.336/3.298/4.259/0.962 ms

Some other info…

: ifconfig gif0
gif0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
        tunnel inet my.wan.ip.addr --> 209.51.181.2
        inet6 2001:470:xxxx.yyy::2 --> 2001:470:xxxx.yyy::1 prefixlen 128 
        inet6 fe80::240:48ff:feb2:8216%gif0 prefixlen 64 scopeid 0x10 
        nd6 options=3 <performnud,accept_rtadv>options=1 <accept_rev_ethip_ver>: netstat -rn -f inet6  (Edited for brevity)
Routing tables

Internet6:
Destination                       Gateway                       Flags      Netif Expire
default                           2001:470:xxxx.yyy::1          UGS        gif0
2001:470:xxxx.yyy::1              2001:470:xxxx.yyy::2          UH         gif0
2001:470:zzzz.yyy::/64            link#1                        U           em0
2001:470:zzzz.yyy::1              link#1                        UHS         lo0</accept_rev_ethip_ver></performnud,accept_rtadv></up,pointopoint,running,multicast>

EDIT: And FYI, a reboot doesn't change anything.

databeestje

Are all the people here affected when the WAN is a PPPoE connection? I've never seen this particular behaviour on static addressing lines.

Did people also add a allow ipv6 icmp for the WAN address of the gif interface? Might be related. I happen to have one, but my internet is cable internet from Ziggo.

Another hunch, if this is DSL and I also see mention of the sagem modems is that they might not pass the protocol 41 over the PPPoE connection. That would mean it's possible to troubleshoot if another dsl modem is available.

I believe a number of the SpeedTouch modems in bridge modem also have this particular issue.

jimp

I have it running over my Cable connection (opt1) which is DHCP.

I see the return traffic coming back on the physical interface so it doesn't appear to be getting blocked by my ISP or equipment, the OS sees it, it just doesn't make it back onto the gif interface for whatever weird reason.

I also have an IPv6 allow for icmp, both on the real WAN and on the gif interface. I'll try some more tests though. I wouldn't think it would be a firewall rule though because it's return traffic, the state should be letting it back in.

jimp

OK, for giggles I reset my default route to my Cable line, opt1, and it came up!

Then I reset it back to my WAN (DSL) and it went back down.

Then I added a static route over my cable to the he.net tunnel broker and it stayed up!

So it appears we might need some code to add a static route to the gif endpoint based off the interface chosen when making the gif. That may be a general 2.0 bug.

databeestje

Don't we all love multiwan when it works.

I'll add code that adds a route for gif tunnel endpoints for the interface they belong on. Perhaps a filter rule could theoretically help.

Adding a filter rule on the correct wan that has reply-to set for that traffic, would that work? Maybe the traffic is stateless and then the state does not match.

jimp

Not sure if I rule with reply-to would be sufficient or not. I can try it, but I'd need to figure out how that rule would look to match just the gif traffic without allowing being too permissive.

vinsomething

I've seen similar weirdness (and different symptoms at one point - I was seeing no return traffic on the physical interface until I added a default IPv6 gateway) but I haven't spent enough time isolating.

See my post here for what I was seeing (I was trying to replace my HE tunnel with a SIXXS one on my Cable/DHCP interface which worked once I set a default GW):
https://www.sixxs.net/forum/?msg=setup-4100929

Yesterday I couldn't get my other HE tunnel up on my PPPoE DSL WAN interface, but I haven't invested too much time into troubleshooting.

Let me know if I can help by providing logs or testing anything.

databeestje

Just do what Jim did. Add a static route for the IPv4 address of the tunnelbroker on the correct WAN interface.

vinsomething

Got it. I also had to upgrade the firmware on the DSL modem to something from the last 5 years :)

I don't think this is quite what I was seeing a couple weeks ago as I was only trying to bring up a single tunnel on my primary WAN at the time (and by adding a v6 default gateway is what kicked it into gear)…... but it's working now the way I want to so I'm happy.

databeestje

I've found that the default route might dissapear when the parent interface goes down.

I have not debugged that.