4. IPv6 ping craziness

IPv6 ping craziness

  • T

    Ok, this issue has me (he.net ipv6 sage), my ISP, and those in ##pfsense ive spoken to utterly stumped.

    I will attempt to describe this as best I can without screaming!

    And before I go any further, understand that there are firewall rules in place currently to allow ALL ipv6 traffic from any source, to any destination, via any protocol, so this is not a rules issue.

    Network setup:

    Pfsense:
    WAN: IPv6 SLAAC, which is assigned 2a02:13a0:ad02:21:21c:25ff:fe4d:f77d  pppoe1 on sge0
    LAN: IPv6 Static: 2a02:13a0:a006:1::/64  em0

    Pfsense v6 routing table:

    IPv6
Destination	Gateway	Flags	Refs	Use	Mtu	Netif	Expire
default	fe80::f2f7:55ff:fe0c:5700%pppoe1	UGS	0	0	1492	pppoe1	 
::1	::1	UH	0	0	16384	lo0	 
2001:4860:4860::8844	fe80::f2f7:55ff:fe0c:5700	UGHS	0	692	1492	pppoe1	 
2001:4860:4860::8888	fe80::f2f7:55ff:fe0c:5700	UGHS	0	2061	1492	pppoe1	 
2a02:13a0:a006:1::	link#5	UHS	0	0	16384	lo0	=>
2a02:13a0:a006:1::/64	link#5	U	0	1681	1500	em0	 
2a02:13a0:a006:2::	link#12	UHS	0	0	16384	lo0	=>
2a02:13a0:a006:2::/64	link#12	U	0	842	1500	re0_vlan90	 
2a02:13a0:ad02:21::/0	link#18	U	0	35	1492	pppoe1	 
2a02:13a0:ad02:21:21c:25ff:fe4d:f77d	link#18	UHS	0	3	16384	lo0	 
fe80::%sge0/64	link#4	U	0	0	1500	sge0	 
fe80::21c:25ff:fe4d:f77d%sge0	link#4	UHS	0	0	16384	lo0	 
fe80::%em0/64	link#5	U	0	125	1500	em0	 
fe80::202:a5ff:fe4f:2a81%em0	link#5	UHS	0	0	16384	lo0	 
fe80::%em1/64	link#6	U	0	0	1500	em1	 
fe80::202:a5ff:fe4f:2a80%em1	link#6	UHS	0	0	16384	lo0	 
fe80::%re0/64	link#7	U	0	0	1500	re0	 
fe80::4a02:2aff:fe07:35d5%re0	link#7	UHS	0	0	16384	lo0	 
fe80::%lo0/64	link#10	U	0	0	16384	lo0	 
fe80::1%lo0	link#10	UHS	0	0	16384	lo0	 
fe80::%re0_vlan90/64	link#12	U	0	962	1500	re0_vlan90	 
fe80::21c:25ff:fe4d:f77d%re0_vlan90	link#12	UHS	0	0	16384	lo0	 
fe80::%re0_vlan1/64	link#13	U	0	0	1500	re0_vlan1	 
fe80::21c:25ff:fe4d:f77d%re0_vlan1	link#13	UHS	0	0	16384	lo0	 
fe80::%re0_vlan99/64	link#14	U	0	0	1500	re0_vlan99	 
fe80::21c:25ff:fe4d:f77d%re0_vlan99	link#14	UHS	0	0	16384	lo0	 
fe80::%re0_vlan64/64	link#15	U	0	0	1500	re0_vlan64	 
fe80::21c:25ff:fe4d:f77d%re0_vlan64	link#15	UHS	0	0	16384	lo0	 
fe80::%re0_vlan20/64	link#16	U	0	0	1500	re0_vlan20	 
fe80::21c:25ff:fe4d:f77d%re0_vlan20	link#16	UHS	0	0	16384	lo0	 
fe80::%re0_vlan30/64	link#17	U	0	0	1500	re0_vlan30	 
fe80::21c:25ff:fe4d:f77d%re0_vlan30	link#17	UHS	0	0	16384	lo0	 
fe80::%pppoe1/64	link#18	U	0	0	1492	pppoe1	 
fe80::21c:25ff:fe4d:f77d%pppoe1	link#18	UHS	0	0	16384	lo0	 
fe80::548c:995a:9b35:e530%pppoe1	link#18	UHS	0	0	16384	lo0	 
ff01::%sge0/32	fe80::21c:25ff:fe4d:f77d%sge0	U	0	0	1500	sge0	 
ff01::%em0/32	fe80::202:a5ff:fe4f:2a81%em0	U	0	0	1500	em0	 
ff01::%em1/32	fe80::202:a5ff:fe4f:2a80%em1	U	0	0	1500	em1	 
ff01::%re0/32	fe80::4a02:2aff:fe07:35d5%re0	U	0	0	1500	re0	 
ff01::%lo0/32	::1	U	0	0	16384	lo0	 
ff01::%re0_vlan90/32	fe80::21c:25ff:fe4d:f77d%re0_vlan90	U	0	0	1500	re0_vlan90	 
ff01::%re0_vlan1/32	fe80::21c:25ff:fe4d:f77d%re0_vlan1	U	0	0	1500	re0_vlan1	 
ff01::%re0_vlan99/32	fe80::21c:25ff:fe4d:f77d%re0_vlan99	U	0	0	1500	re0_vlan99	 
ff01::%re0_vlan64/32	fe80::21c:25ff:fe4d:f77d%re0_vlan64	U	0	0	1500	re0_vlan64	 
ff01::%re0_vlan20/32	fe80::21c:25ff:fe4d:f77d%re0_vlan20	U	0	0	1500	re0_vlan20	 
ff01::%re0_vlan30/32	fe80::21c:25ff:fe4d:f77d%re0_vlan30	U	0	0	1500	re0_vlan30	 
ff01::%pppoe1/32	fe80::21c:25ff:fe4d:f77d%pppoe1	U	0	0	1492	pppoe1	 
ff02::%sge0/32	fe80::21c:25ff:fe4d:f77d%sge0	U	0	0	1500	sge0	 
ff02::%em0/32	fe80::202:a5ff:fe4f:2a81%em0	U	0	0	1500	em0	 
ff02::%em1/32	fe80::202:a5ff:fe4f:2a80%em1	U	0	0	1500	em1	 
ff02::%re0/32	fe80::4a02:2aff:fe07:35d5%re0	U	0	0	1500	re0	 
ff02::%lo0/32	::1	U	0	0	16384	lo0	 
ff02::%re0_vlan90/32	fe80::21c:25ff:fe4d:f77d%re0_vlan90	U	0	0	1500	re0_vlan90	 
ff02::%re0_vlan1/32	fe80::21c:25ff:fe4d:f77d%re0_vlan1	U	0	0	1500	re0_vlan1	 
ff02::%re0_vlan99/32	fe80::21c:25ff:fe4d:f77d%re0_vlan99	U	0	0	1500	re0_vlan99	 
ff02::%re0_vlan64/32	fe80::21c:25ff:fe4d:f77d%re0_vlan64	U	0	0	1500	re0_vlan64	 
ff02::%re0_vlan20/32	fe80::21c:25ff:fe4d:f77d%re0_vlan20	U	0	0	1500	re0_vlan20	 
ff02::%re0_vlan30/32	fe80::21c:25ff:fe4d:f77d%re0_vlan30	U	0	0	1500	re0_vlan30	 
ff02::%pppoe1/32	fe80::21c:25ff:fe4d:f77d%pppoe1	U	0	0	1492	pppoe1

    So first of all outbound ipv6, this works fine.

    Now the problems come when tryign to connect to this network from outside, but NOT on the initial connection.
    Selected lines from the 60+ tcpdumps i have done today….

    First an outgoing ping,

    
C:\Users\James>ping 2001:1b40:5000:22::123

Pinging 2001:1b40:5000:22::123 with 32 bytes of data:
Reply from 2001:1b40:5000:22::123: time=10ms
Reply from 2001:1b40:5000:22::123: time=9ms
Reply from 2001:1b40:5000:22::123: time=9ms
Reply from 2001:1b40:5000:22::123: time=9ms

Ping statistics for 2001:1b40:5000:22::123:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 9ms, Maximum = 10ms, Average = 9ms

    And the attempt packet captured from the pfsense LAN:

    17:46:10.349597 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 314, length 40
17:46:10.358630 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 314, length 40
17:46:11.352816 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 315, length 40
17:46:11.361746 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 315, length 40
17:46:12.355661 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 316, length 40
17:46:12.365088 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 316, length 40
17:46:13.361255 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 317, length 40
17:46:13.370471 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 317, length 40

    And the same thing tcpdump'ed from the WAN interface:

    17:47:06.735520 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 318, length 40
17:47:06.744655 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 318, length 40
17:47:07.738672 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 319, length 40
17:47:07.747773 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 319, length 40
17:47:08.741265 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 320, length 40
17:47:08.750426 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 320, length 40
17:47:09.744745 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo request, seq 321, length 40
17:47:09.753764 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo reply, seq 321, length 40

    Ok, so far so good, but now the random issue begins when i try to ping my internal ip from the outside.

    
james@observium:~$ ping6 2a02:13a0:a006:1::dead:beef:cafe
PING 2a02:13a0:a006:1::dead:beef:cafe(2a02:13a0:a006:1:0:dead:beef:cafe) 56 data bytes
^C
--- 2a02:13a0:a006:1::dead:beef:cafe ping statistics ---
21 packets transmitted, 0 received, 100% packet loss, time 20159ms

    This is the capture from the pfsense WAN:

    17:48:39.990853 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 1, length 64
17:48:40.998513 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 2, length 64
17:48:42.006188 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 3, length 64
17:48:43.014294 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 4, length 64
17:48:44.022198 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 5, length 64
17:48:45.030307 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 6, length 64
17:48:46.038432 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 7, length 64

    So request is coming in just fine, but now for the wierd part
    packet capture from the pfsense LAN:

    17:49:36.790236 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 1, length 64
17:49:36.790403 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 1, length 64
17:49:37.798329 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 2, length 64
17:49:37.798494 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 2, length 64
17:49:38.806216 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 3, length 64
17:49:38.807339 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 3, length 64
17:49:39.814559 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 4, length 64
17:49:39.814677 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 4, length 64
17:49:39.858532 IP6 2a02:13a0:a006:1:: > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, destination unreachable, unreachable address 2001:1b40:5000:22::123, length 112
17:49:40.822219 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 5, length 64
17:49:40.823268 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 5, length 64
17:49:41.829894 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 6, length 64
17:49:41.830610 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 6, length 64
17:49:42.838462 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 7, length 64
17:49:42.839326 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 7, length 64
17:49:43.057770 IP6 2a02:13a0:a006:1:: > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, destination unreachable, unreachable address 2001:1b40:5000:22::123, length 112
17:49:43.846133 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 8, length 64
17:49:43.846295 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 8, length 64
17:49:44.854030 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 9, length 64
17:49:44.854540 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 9, length 64
17:49:45.862159 IP6 2001:1b40:5000:22::123 > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, echo request, seq 10, length 64
17:49:45.862353 IP6 2a02:13a0:a006:1:0:dead:beef:cafe > 2001:1b40:5000:22::123: ICMP6, echo reply, seq 10, length 64
17:49:46.912023 IP6 2a02:13a0:a006:1:: > 2a02:13a0:a006:1:0:dead:beef:cafe: ICMP6, destination unreachable, unreachable address 2001:1b40:5000:22::123, length 112

    So what this shows me is that the ping request came in, was sent out of the LAN, the local PC responded to it, and only then every few times does the pfsense say destination unreachable.
    Given that I can simultaneously ping the host outbound, while inbound ping responses are told 'destination unreachable' makes absolutely no sense to me.

    Can anyone shed any light on this?

    Update:
    ifconfig and pfctl -sr as requested by someone on IRC: https://gist.github.com/tandyuk/b8d97d127f2e20f9624f

    0
  • T

    Just to note that all the above was done on pfsense 2.1.5, and the router has now been upgraded to 2.2.4 with exactly the same issues present.

    0
  • T

    Ok turns out this is an actual bug!

    https://redmine.pfsense.org/issues/5258

    If you suffer from this, System -> Advanced > Firewall/NAT and Disable reply-to rules (tick the box).
    Not sure what multi WAN ipv6 users can do to fix it.

    0
Log in to reply
 

Products

Applications

Support

Services

News

Resources

Company

Our Mission

As host of the pfSense open source firewall project, Netgate believes in enhancing network connectivity that maintains both security and privacy. We also believe everyone should be able to afford it.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy