IPv6 default route is lost
-
Hello,
I'm using 2.4.3-RELEASE-p1 on comcast. The problem I'm having is that after a few minutes/hours after release/renew or reboot IPv6 stops working. On investigation, I see that there is no IPv6 default route, which is consistent with IPv6 no longer working. Both WAN and LAN interfaces still have their IPv6 addresses, only the default route is missing. What I don't understand is why the default route is lost after initially working.
As a workaround I have disabled IPv6. This is unfortunate however unreliability over such a short timescale is unacceptable and renders the connection near unusable.
This appeared to start happening after I upgraded to 2.4.3-RELEASE-p1. I believe I was previously on a 2.3.x version so it's a pretty big jump in versions so unfortunately this doesn't really narrow down when the bug was introduced.
Has anyone else hit this problem?
-
@tungsten said in IPv6 default route is lost:
Has anyone else hit this problem?
No, it's solid here.
-
Hi tungsten,
Please can you describe if it is the WAN interface that looses its upstream default route or do you experience that on one of your devices looses their default route on your pfSense router LAN?I would suggest you try to capture some packets - e.g. 1 - 2 minutes of packet captures on the network interface that is 'connected' to the problem.
Say you experience the problem on your WAN then go to your router - Diagnostics - Packet Capture.
Select your WAN interface (you might have renamed it to something else than WAN.)
You might want to enable promiscuous mode although I do not believe it is necessary in this case.Now, when you have captured the packets and downloaded them install Wireshark: https://www.wireshark.org/
What you are interested to look for are ICMPv6 packets. In this case you can use the capture filter: icmp6
, (without a v meaning not icmpv6, but icmp6.)
besides some Neighbor Solicitation/Advertisement packets what is important to look out for are Router Advertisement packets from your Internet service provider. If they lack you will loose the default route on your WAN interface after some time. These Router Advertisement packets tell devices whether they are sub-routers or e.g. normal computeres that a route should be made to the router that sent those packets. https://en.wikipedia.org/wiki/Neighbor_Discovery_ProtocolI do not know how comcast provides adresses and the subnet your router should use.
Are they provided by DHCPv6 using prefix delegation (PD) or do you manually enter static addresses and a subnet for your LAN? -
I would suggest you try to capture some packets - e.g. 1 - 2 minutes of packet captures on the network interface that is ‘connected’ to the problem.
hm..... what's interesting is that I can't seem to repro when doing a packet capture.
That made me think, what's different when I'm doing a packet capture, and the first thing that occurred to me is, the interface is in promiscuous mode. So I set the interface to promiscuous mode without doing a packet capture and left it that way. Now it's been stable for about a day.
I couldn't find a setting in the web interface to set promiscuous mode. In order to have this happen without me having to log in on every reboot I have installed shellcmd and set it up to set promiscuous mode at boot.
This is a tolerable workaround for now, but it does suggest a bug on pfsense's end. This shouldn't be necessary. The hardware is a Netgate SG-1000 and it looks like the network interfaces are some sort of internal switch with VLAN tagging, maybe something to do with that.
-
@tungsten said in IPv6 default route is lost:
I would suggest you try to capture some packets - e.g. 1 - 2 minutes of packet captures on the network interface that is ‘connected’ to the problem.
hm..... what's interesting is that I can't seem to repro when doing a packet capture.
That made me think, what's different when I'm doing a packet capture, and the first thing that occurred to me is, the interface is in promiscuous mode. So I set the interface to promiscuous mode without doing a packet capture and left it that way. Now it's been stable for about a day.
I couldn't find a setting in the web interface to set promiscuous mode. In order to have this happen without me having to log in on every reboot I have installed shellcmd and set it up to set promiscuous mode at boot.
This is a tolerable workaround for now, but it does suggest a bug on pfsense's end. This shouldn't be necessary. The hardware is a Netgate SG-1000 and it looks like the network interfaces are some sort of internal switch with VLAN tagging, maybe something to do with that.
Sounds strange. I do not know what kind of basic configuration that the Netgate SG-1000 comes with or your specific configuration on that device nor the setup of your ISP. Maybe the friendly Netgate people is able to understand the situation already from what you have described and help, but I guess more info about your pfSense setup on the device and your ISP might be needed, but I am happy you have a workaround. :)
-
Sounds strange. I do not know what kind of basic configuration that the Netgate SG-1000 comes with or your specific configuration on that device nor the setup of your ISP. Maybe the friendly Netgate people is able to understand the situation already from what you have described and help, but I guess more info about your pfSense setup on the device and your ISP might be needed, but I am happy you have a workaround. :)
SG-1000 with ISP on cpsw0 and LAN on cpsw1. Reproed the problem after software update and resetting to defaults. ISP is comcast. Pretty simple setup.
-
Did a packet capture using tcpdump:
21:04:22.040097 00:01:5c:7a:d0:46 > 33:33:00:00:00:01, ethertype IPv6 (0x86dd), length 198: fe80::201:5cff:fe7a:d046 > ff02::1: ICMP6, router advertisement, length 144
What I think might be wrong is the interface doesn't see these RA's. Comcast is sending the RA to the correct IPv6 multicast address, and using the correct MAC address for that IPv6 multicast address, but for whatever reason the interface doesn't notice unless it's set to promiscuous mode. This suggests the interface doesn't correctly recognize multicast packets in the IPv6 multicast MAC address range 33:33:xx:xx:xx:xx.
-
Yup, when I do the tcpdump without first setting promisc and using the -p flag to prevent tcpdump setting promisc, it doesn't see the RA's.
-
Let us say you do not get any replies here then you might want to file a bug report here: https://redmine.pfsense.org/projects/pfsense/roadmap
You might want to set release to 2.4.4 in the bug report so that the report is not forgotten. It might be addressed or moved to a later release or someone might be able to explain what is wrong/what needs to be done.
But cool that you have been able to narrow down what the problem seems to be about. Remember to link to this forum thread from the bug report in that way you might help other people as well. :)
-
Great defect report you have made: https://redmine.pfsense.org/issues/8611
