Reproducible kernel panic with pfSense 2.2 and IPSEC
-
@w0w:
And you don't have "IP Random id generation" enabled?
"IP Random id generation" is off. This setting I have never changed. Otherwise, I have only MSS clamping enabled and the Unity plugin disabled. "IPsec Mobile Client Support" is also turned off. All other settings are defaults.
Apparently the kernel timesource is not the cause of the error. Could someone have another look at the dump? What information is still needed?
-
We need to use bugtracker to report the bug. Please report this issue here https://redmine.pfsense.org/
Don't link the forum page, post your dumps and I'll add mine too.
FYI I don't have changed MSS and unity settings. -
https://redmine.pfsense.org/issues/4454
-
Similar issue here. If I connect via VPN, PFSense stops and restarts - the IPSEC connection lasts between 30 seconds and 5 minutes. This issue is 100% reproducible - I've crashed my box about six times today.
Fatal double fault:
eip = 0xc12c62a8
esp = 0xecf4cff8
ebp = 0xecf4d000
cpuid = 0; apic id = 00
panic: double fault
cpuid = 0
KDB: enter: panic
panic.txt0600001412471723700 7136 ustarrootwheeldouble faultversion.txt06000025112471723700 7614 ustarrootwheelFreeBSD 10.1-RELEASE-p4 #0 36d7dec(releng/10.1)-dirty: Thu Jan 22 15:12:38 CST 2015
root@pfsense-22-i386-builder:/usr/obj.i386/usr/pfSensesrc/src/sys/pfSense_SMP.10PFSense 2.2 - upgraded from 2.1.5
Hardware:
CPU: Intel(R) Atom(TM) CPU N270 @ 1.60GHz
Mobo: KINO-945GSE
Storage: 2 GB CF Card
Dual LAN: Realtek PCIe 8111CP GbE controllerIPSEC Details:
Mobile Client
No IP Compression
Unity plugin disabled
IKE v1
Virtual IP Address Assigned to Clients
IP Random ID Generation at default value (default is 0: sequential IP IDs)Interfaces:
RE0: WAN: PPOE
RE1: LAN/OPT1/OPT2 using VLAN taggingMay try a clean install of v2.2 if you think there's any mileage in it.
-
@afasoas: are you running 32 or 64 bit version? So far I think all the reports are coming from x86 versions. Is this reproducible under x64??
-
32 bit.
Well spotted.
I will add my crash dump to the bug tracker shortly. -
Upgraded my hardware so I could run the 64-bit version. No issues to report thus far. IPSEC seems solid and stable.
-
Great solution, but this is like cutting the head and sewing back a new one, more "brainful".
It could be also driver Ethernet issue with physical low memory installed. I can only wait when somebody really smart will check our crash dumps to find out the reason of double triple crash and panic. -
@w0w:
Great solution, but this is like cutting the head and sewing back a new one, more "brainful".
It could be also driver Ethernet issue with physical low memory installed. I can only wait when somebody really smart will check our crash dumps to find out the reason of double triple crash and panic.I appreciate that this isn't the most helpful solution. I just wanted to confirm that the problem went away using the same configuration with a 64-bit version.
On Edit: I realise that you are using an Atom D2500 - you can run the 64-bit version of pfSense on it, if that helps?
-
I can, really, but what if I don't need it? My typical memory usage is less then 6% and CPU is mostly 90% in peak (300Mbit internet, three clients). I can buy some XEON based proliant G8 but why? :)
Maybe I'll move to x64 platform if we can't trust x86 anymore. But I need an answer for the question. DO we really need to move to x64 just because x86 is not supported or what?
I think i'll wait for answer before buying some needless hardware. :) -
And what about the ALIX boards? As far as I know they are all 32bit.
-
And what about the ALIX boards? As far as I know they are all 32bit.
Not getting any IPsec panics on Alix. (Also, make sure you did not enable some stupid features, like the infamous "Insert a stronger id into IP header of packets passing through the filter.")
-
But your hardware is already 64-bit capable, at least as far as pfSense is concerned!
Yes Intel don't provide 64-bit video drivers but seems to be a non-issue here.@w0w:
My system is D2500CC mini-ITX motherboard from Intel, all embedded into it.
-
And what about the ALIX boards? As far as I know they are all 32bit.
Not getting any IPsec panics on Alix. (Also, make sure you did not enable some stupid features, like the infamous "Insert a stronger id into IP header of packets passing through the filter.")
I don't think that hiding your client OS unique ID behind firewall is so stupid as you think about it.
-
But your hardware is already 64-bit capable, at least as far as pfSense is concerned!
Yes Intel don't provide 64-bit video drivers but seems to be a non-issue here.@w0w:
My system is D2500CC mini-ITX motherboard from Intel, all embedded into it.
Yep. But what is the point to use 64-bit OS with 2GB of RAM? It does not fix the problem in 32-bit version also :) There is some bug, that must be fixed and this is good, maybe, that it is pointed now to 32-bit version only, but next time it could be related to 64-bit only, so migrating between platforms is useless for me, until I read something like "64-bit freebsd is more secure and stable, don't use 32-bit anymore".
-
Migrating between platforms resolves your problem, for the time being.
If your memory usage is at 6% then I figure there should not be a problem switching over to 64-bit. -
@w0w:
…so migrating between platforms is useless for me, until I read something like "64-bit freebsd is more secure and stable, don't use 32-bit anymore".
Ahh, I think you mean this:
"[_…64 bit is more widely used, what we test the most with, and what most of our development is done using.32 bit is a dying breed. FreeNAS and DragonflyBSD both just put out their last releases with 32 bit support. While we'll still continue to support 32 bit in 2.2.x releases and possibly beyond that, ending 32 bit support is certainly on the road map and will happen sooner than later.
There is no reason to use 32 bit over 64 today, if your hardware is 64 bit capable, you should only be running 64 bit._](https://forum.pfsense.org/index.php?topic=84679.msg464432#msg464432)"
Chris Buechler, November 27th, 2014
-
See also:
https://doc.pfsense.org/index.php/Does_pfSense_support_64_bit_systemshttps://doc.pfsense.org/index.php/Is_32-bit_or_64-bit_pfSense_Preferred
-
Ok… At least I'll give it a try. Later, next week maybe :)
-
Still, do we have any clues on the issue itself? It affects Alix boards and similar hardware to at least some degree (besides some other reports, I had to downgrade some production boxes myself due to random reboots which I am sure are related to all this). Unfortunately I couldn't find a specific trigger but there seem to be several crash dumps and reproducible configs available