Trouble configuring IPv6
-
Hi,
I have a Netgate SG-3100 running 21.02.2-RELEASE and I'm having trouble getting IPv6 to work.
I'm using DHCP6 on the WAN and TRACK on the LAN using Prefix ID 0. Both interfaces receive valid global addresses. Both IPv4 and IPv6 gateways are configured automagically by DHCP6.
The LAN runs both DHCP6 & RA (unmanaged) and all devices on the LAN receive valid global addresses.
I have added rules to the firewall to pass ALL protocols of ALL ipv6 from anywhere to one of my LAN servers. There is no independent firewall running on this server. I can reach my server from any point on my LAN, but I cannot reach the server from outside the LAN.
I can ping the LAN interface from any device on the LAN.
I can ping the WAN's global ip from any device on the LAN.I can ping google from the WAN port.
I CANNOT ping google from the LAN port.
I CANNOT ping google from any device on the LAN.Any thoughts on what I might be doing wrong?
-
@wineguy said in Trouble configuring IPv6:
I can ping the ISPs default ipv6 gateway (using its global ip) from any device on the WAN.
Perhaps you meant LAN?
Can you ping google from pfsense?
When you try from a device on the LAN, what error message do you get?
-
@jknott Yes, I meant LAN. Using pfSense, I can ping google from the WAN port, but not from the LAN port.
or from the server...
$ ping -6 -c 3 google.com
PING google.com(ord37s33-in-x0e.1e100.net (2607:f8b0:4009:809::200e)) 56 data bytes--- google.com ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2027ms -
@wineguy
and this is the result of a packet capture on the WAN for host 2607:f8b0:4009:809::200e (google.com) during the 'ping from LAN' test.
It appears that the ping results are presented to the WAN, but never make it to the LAN.
-
@wineguy What is interesting about the above...
google is ::200e, fe11:354 is my WAN, not my LAN, which should have originated the packets. (though this is common behavior on some *nix.)
When I do a packet capture for IPv6 on the WAN for host google while pinging from the server, the resulting packet capture is empty. No outgoing packets made it to the WAN interface.
-
Please post the actual capture file. It contains a lot more info that what's listed above.
-
@jknott Here's the raw footage...
-
I see both requests and replies in that capture. BTW, you don't have to zip the file. You can paste the capture file as is.
-
I tried to paste the capture, but the interface complained that it wasn't a recognized file type. It took the zip.
The replies are there, yes, but they are addressed to the WAN, not the LAN, implying that the OS substituted the WAN interface on the ping.
The capture on the LAN side is empty.
-
That shouldn't be happening. A while ago, someone (might have been me) complained that the file captures from pfsense weren't accepted, but that was corrected. I have uploaded capture files many times.
-
You're saying you pinged from a computer on the LAN side and the WAN addresses were used? That doesn't make sense, unless you're using NAT.
What do you see on the LAN side? -
@jknott -- Thank you very much for your time!
The LAN side shows 100% packet loss.
I have a single ipv4 port forward active. No ipv6 NAT.
Digression...
The ping behavior is typical of some *nix flavors. If I have a server with multiple interfaces and cripple one, so that it is UP, but has no route to a destination, then"ping -I CRIPPLED_IFACE UNREACHABLE_DEST"
I'll get a warning from ping that it may use a different interface and then it does exactly that - and the ping results reflect the ip of the interface it used, rather than the ip of the interface I requested. Quite like what's happening here.
I'm not familiar with how pfSense is written (which is why I'm here), so I have no idea what's really happening, but it looks like the OS doesn't see a route to google from LAN.
-
Where are you pinging from? Yes, I know you can force an interface in some routers.
Ping from a computer on the LAN. Use packet capture to see what's on the LAN interface or run Wireshark on the computer you're pinging from. Also, do a packet capture on the WAN side. Are you saying the packets on the LAN side have different addresses than they have on the WAN side?
BTW, pfsense runs on FreeBSD, which is a *nix flavour.
-
@jknott
Sorry I didn't make this clear. The results I just showed were from the following test:Start Packet Capture
pfSense:Diagnostics-Packetcapture (WAN,IPv6,any, 2607:f8b0:4009:80b::200e)Start Ping
pfSense:Diagnostics-Ping (2607:f8b0:4009:80b::200e,IPv6,LAN,10)Stop Packet Capture
I shall now run the following test:
Ping google from server somewhere on LAN
+Packet capture on LAN
+Packet capture on WAN
BTW, this is the error I see when attaching packet captures...
-
-
OK, both those show no response. So, when you got a response, it was from pfsense and not a computer behind it. When you get the successful pings, it goes out from the WAN address? Is that correct? The prefix on the one that worked is different from the one that failed, though from the same ISP. This tells me that the WAN address works, but not a LAN address. I ran into this same situation a couple of years ago, when there was a problem with my ISP. I did some more testing with a 2nd connection to my ISP (they provide 2 IPv4 addresses and support 2 devices on the cable modem) and was able to show that traffic for the WAN address arrived, but that for a LAN address didn't. This indicated a problem elsewhere than my network. In my testing, I determined there was a problem with my ISP and even identified the failing system by host name. The way I did that was I used Wireshark and a managed switch configured as a data tap to capture the full DHCPv6-PD sequence. By examining that, I could see that the CMTS at my ISP's head end was failing. If you don't have a spare managed switch and computer to run Wireshark on, you can still use Packet Capture. To use Packet Capture, shut down pfsense and disconnect from your modem. Then power up pfsense and start Packet Capture on the WAN port, filtering on DHCPv6 (port 546 or 547). Then reconnect the modem. This should capture the full DHCPv6-PD sequence. Post the capture here.
PfSense running on Qotom mini PC
i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
UniFi AC-Lite access pointI haven't lost my mind. It's around here...somewhere...
-
BTW, I just tried pinging your WAN and LAN addresses.
Here's what I get when I ping the LAN address:
ping 2603:300a:164f:10e0::167
PING 2603:300a:164f:10e0::167(2603:300a:164f:10e0::167) 56 data bytes
From 2001:558:4040:189:3a17:e1ff:fef8:ef5c: icmp_seq=1 Destination unreachable: Address unreachable
From 2001:558:4040:189:3a17:e1ff:fef8:ef5c: icmp_seq=2 Destination unreachable: Address unreachable
From 2001:558:4040:189:3a17:e1ff:fef8:ef5c: icmp_seq=3 Destination unreachable: Address unreachableBut I get nothing when I try your WAN address. Is it blocked by your firewall?
-
I also tried traceroute:
WAN
traceroute 2603:300a:164f:1000:208:a2ff:fe11:a354
traceroute to 2603:300a:164f:1000:208:a2ff:fe11:a354 (2603:300a:164f:1000:208:a2ff:fe11:a354), 30 hops max, 80 byte packets
1 firewall.jknott.net (2607:fea8:4c82:5900:4262:31ff:fe12:b66c) 0.258 ms 0.192 ms 0.174 ms
2 * * *
3 2607:f798:10:10d3:0:241:5615:221 (2607:f798:10:10d3:0:241:5615:221) 20.620 ms 20.790 ms 20.772 ms
4 2607:f798:10:10e1:0:690:6324:9086 (2607:f798:10:10e1:0:690:6324:9086) 21.165 ms 2607:f798:10:31f:0:2091:4823:3185 (2607:f798:10:31f:0:2091:4823:3185) 20.947 ms 2607:f798:10:2f0:0:2091:4823:2041 (2607:f798:10:2f0:0:2091:4823:2041) 20.934 ms
5 2607:f798:10:8b6:0:2091:4823:5230 (2607:f798:10:8b6:0:2091:4823:5230) 23.449 ms 23.010 ms 23.181 ms
6 2607:f798:10:378:0:2091:4823:7021 (2607:f798:10:378:0:2091:4823:7021) 59.283 ms 44.252 ms 63.357 ms
7 * * *
8 * * *
9 be-1213-cr13.350ecermak.il.ibone.comcast.net (2001:558:3:c9::2) 41.470 ms 26.243 ms be-1113-cr13.350ecermak.il.ibone.comcast.net (2001:558:3:c8::2) 25.578 ms
10 * * *
11 be-1312-cs03.chicago.il.ibone.comcast.net (2001:558:3:10a::1) 34.865 ms be-1212-cs02.chicago.il.ibone.comcast.net (2001:558:3:109::1) 38.610 ms be-1112-cs01.chicago.il.ibone.comcast.net (2001:558:3:108::1) 54.106 ms
12 2001:558:3:209::2 (2001:558:3:209::2) 31.555 ms * *
13 2001:558:300:202f::2 (2001:558:300:202f::2) 51.430 ms 37.341 ms 52.052 ms
14 2001:558:320:bd::2 (2001:558:320:bd::2) 38.598 ms 36.015 ms 40.934 ms
15 2001:558:302:840d::2 (2001:558:302:840d::2) 39.504 ms 37.290 ms 35.159 ms
16 2001:558:4040:189:3a17:e1ff:fef8:ef5c (2001:558:4040:189:3a17:e1ff:fef8:ef5c) 63.104 ms 46.823 ms 48.533 ms
17 * * *
18 * * *
19 * * *
20 * * *This one fails after 16 hops.
LAN
traceroute 2603:300a:164f:10e0::167
traceroute to 2603:300a:164f:10e0::167 (2603:300a:164f:10e0::167), 30 hops max, 80 byte packets
1 firewall.jknott.net (2607:fea8:4c82:5900:4262:31ff:fe12:b66c) 0.255 ms 0.208 ms 0.192 ms
2 * * *
3 2607:f798:10:10d3:0:241:5615:221 (2607:f798:10:10d3:0:241:5615:221) 21.805 ms 21.968 ms 23.550 ms
4 2607:f798:10:31d:0:2091:4823:3177 (2607:f798:10:31d:0:2091:4823:3177) 23.742 ms 2607:f798:10:34c:0:2091:4823:5121 (2607:f798:10:34c:0:2091:4823:5121) 23.321 ms 2607:f798:10:31d:0:2091:4823:3177 (2607:f798:10:31d:0:2091:4823:3177) 23.918 ms
5 2607:f798:10:8b6:0:2091:4823:5230 (2607:f798:10:8b6:0:2091:4823:5230) 23.496 ms 28.617 ms 28.385 ms
6 2607:f798:10:378:0:2091:4823:7021 (2607:f798:10:378:0:2091:4823:7021) 50.861 ms 51.807 ms 51.402 ms
7 * * *
8 * * be-2101-cs01.350ecermak.il.ibone.comcast.net (2001:558:3:140::1) 55.113 ms
9 be-1313-cr13.350ecermak.il.ibone.comcast.net (2001:558:3:ca::2) 40.466 ms be-1213-cr13.350ecermak.il.ibone.comcast.net (2001:558:3:c9::2) 31.590 ms *
10 be-301-cr12.chicago.il.ibone.comcast.net (2001:558:3:1b4::2) 26.862 ms * be-302-cr12.chicago.il.ibone.comcast.net (2001:558:3:1b5::2) 38.452 ms
11 * be-1412-cs04.chicago.il.ibone.comcast.net (2001:558:3:10b::1) 39.337 ms be-1112-cs01.chicago.il.ibone.comcast.net (2001:558:3:108::1) 39.259 ms
12 2001:558:3:209::2 (2001:558:3:209::2) 23.795 ms 40.070 ms 2001:558:3:20a::2 (2001:558:3:20a::2) 41.728 ms
13 2001:558:300:202f::2 (2001:558:300:202f::2) 52.476 ms 39.521 ms 52.740 ms
14 2001:558:320:bd::2 (2001:558:320:bd::2) 35.428 ms 33.893 ms 35.697 ms
15 2001:558:302:840d::2 (2001:558:302:840d::2) 49.037 ms 49.158 ms 49.005 ms
16 2001:558:4040:189:3a17:e1ff:fef8:ef5c (2001:558:4040:189:3a17:e1ff:fef8:ef5c) 50.663 ms 49.072 ms 53.657 ms
17 2001:558:4040:189:3a17:e1ff:fef8:ef5c (2001:558:4040:189:3a17:e1ff:fef8:ef5c) 3041.129 ms !H 3033.028 ms !H 3027.778 ms !HThis one seems to have a successful completion at hop 17.
-
@jknott I've been off-site. I will do this in a few hours. Thank you!
-
@jknott
@jknott Re: your traceroute... I saw this in my logs and thought I was going nuts. Who the *** has been pinging me? And was successful?!? LOL.When I do some voodoo (disconnect WAN from pfSense, reboot ISP modem, reboot pfSense, reconnect WAN to pfSense after both reboots complete), I can get some flaky, intermittent IPv6 connectivity, sometimes lasting for days.
This is starting to sound like an ISP issue, even though they claim the issue is in my firewall config. (If I connect the server directly to the ISP modem, the IPv6 connection is rock solid. In fact I can multi-home the server and get perfect IPv6 from its direct connection to the ISP modem, while getting no IPv6 through the firewall. But I really want the server behind the firewall...)
I'll run your reboot capture as soon as I get in. Thank you!
