Random kernel panic and restart on 2.7.2
-
@stephenw10 except for PIDs, they are near identical from one crash to another.
I'll take a look at your link.
Thank you! -
I would guess it's triggered by an IPSec tunnel carrying IPv6 if you have that?
Of course that should not panic...
-
Well, it seems that the issue can be related to the bug...!!
I provide some context for better understanding.
This is my home-office firewall.
I do a lot of remote work as IT manager for some customers, and I connect to the internet via a dual 4G setup (using two different providers and pointing different BTS), since I live in a remote location (not a great choice for an IT guy, but hey...
).So I've setted up many IPSec VPNs with dual tunnel for every customer using router VTI and BGP dynamic routing via FRR.
With this setup, I can continue to work even when one of my WANs goes down or loss packets (which is quite common).
All this on IPv4 world. (no IPv6 connectivity from mobile providers here)But I've also the same connection schema to a location with IPv6 from which I take a /64 subnet to my house for experimental purposes.
So one of the two tunnel on this location is routing also this IPv6 subnet to my house on a second Phase 2 IPSec.The whole system worked very fine, but after learning about this bug, it could be that during a connection drop, even a brief one, there is some IPv6 traffic trying to pass through the offline ipsec interface...
This setup has actually been working for more than 1 year, but I previously tunneled via OpenVPN and only recently switched this IPv6 routing to IPSec.
I'll try shutting down the IPv6 stack entirely and see if that fixes it.
Thank you!
-
Yeah, that seems likely from that backtrace. If it always happens ai less than 12hrs that should be easy enough to test.
I've never been able to replicate that panic locally which means it's far more difficult to pin down.
-
Pretty frequent, so I think we'll know by tomorrow
Feb 8 17:36:18 root 26063 Bootup complete Feb 8 16:49:44 root 428 Bootup complete Feb 8 14:47:03 root 21766 Bootup complete Feb 8 13:45:17 root 93044 Bootup complete Feb 8 07:10:15 root 60172 Bootup complete Feb 8 00:14:39 root 77642 Bootup complete Feb 7 15:36:30 root 45258 Bootup complete Feb 7 14:53:23 root 98846 Bootup complete Feb 7 13:40:11 root 15021 Bootup complete Feb 7 12:38:21 root 49059 Bootup complete Feb 7 10:34:49 root 34494 Bootup complete Feb 7 10:16:51 root 75312 Bootup complete Feb 7 08:35:36 root 62958 Bootup complete Feb 7 07:53:43 root 54954 Bootup complete Feb 6 23:09:37 root 60224 Bootup complete Feb 6 22:28:15 root 96734 Bootup complete Feb 6 21:22:09 root 802 Bootup complete -
I am curious as to what I can search through / look for in my crash dumps as I have been crashing pretty regularly on 2.7.2 to see if this is similar. Or what files would someone like to view ?
-
The most telling line in the backtrace is probably:
ip6_output()
Though that doesn't always appear as you can see in the bug report where is happens on ppp links. -
Uptime: 15h 44m
I'll keep an eye on it today too, but it seems that disabling IPv6 subnet tunneling solved the problem, so it's probably the same anomaly.
Do you know if there is a planned fix for this problem also on pfSense CE?
As a workaround on my specific problem, I could try to restore that routing on OpenVPN tunnels as before (which had never given this problem), but it would be nice if it were solved.
Thanks,
Edoardo -
@EdoFede said in Random kernel panic and restart on 2.7.2:
I could try to restore that routing on OpenVPN tunnels as before
That would be a good test. I would expect both VPN types to go down at the same time so IPv6 sessions over both should behave similarly. So if it doesn't panic over OpenVPN then it's handling that differently which could be a clue.
-
I'm trying to replicate the setup of IPv6 routing on OpenVPN, but something doesn't work as expected.
I'm not doing the exact same way as before (that worked...both IPv4 and IPv6 tunneling) because I've already setted up IPv4 over IPSec + BGP for this site and don't want to brake the whole setup.I'm trying to route only IPv6 traffic over the OpenVPN tunnel (that has IPv4 endpoints as before), but something is wrong, I think on routing.
I'm able to ping6 google from the "remote" firewall via the tunnel, but not on the internal "remote" IPv6 network.I'll investigate and let you know if I can reproduce the issue even on OpenVPN.
I don't think that will happen anyway, because nothing like this has ever happened to me before with IPv6 and OpenVPN.Bye!
Edo -
Sounds like a missing iroute at the server end.
-
yeah, missing iroute!
Now fixed, thanks!Now I'll monitor the system and let you know if the issue happens also on OpenVPN.
Bye,
Edo -
it seems to be stable on OpenVPN... no reboots/crash at the moment.
The only setup difference with the IPSec configuration is that on IPSec I had to manually enter the default route (route -6 add default <tunnel endpoint>) because for some strange reason it was not set automatically (even if I selected the gateway as default in the routing menu).
I'll write if it happens again, but I would say that the problem only seems to be present on IPSec.
