2.2.4 daily page faults - fresh install with reloaded config.
-
Well, that did not take long…
Different crash this time...
<118>Aug 28 18:44:05 ipsec_starter[47323]: shunt policy 'bypasslan' uninstalled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xa40c050150 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80cf0d26 stack pointer = 0x28:0xfffffe001abfa710 frame pointer = 0x28:0xfffffe001abfa7a0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: netisr 0) version.txt06000024712570030643 7616 ustarrootwheelFreeBSD 10.1-RELEASE-p15 #0 c5ab052(releng/10.1)-dirty: Sat Jul 25 20:20:58 CDT 2015 root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10 -
Something strange going on as I just posted a suggestion but it had dissappeared.
Anyway disable OpenVPN and see if that resolves the problem, as I suspect OpenVPN is being used to crash your system.
-
There is not enough information in the small portion of the crash dump posted to determine anything. Please post the entire crash dump, or submit it and let us know what IP address it was submitted from along with the approximate time.
Typically a crash that changes (different areas each time) tends to be more likely a hardware issue than a software issue, but without seeing the backtraces and other info it's impossible to determine or even make a proper educated guess.
-
Thanks Guys. Was not sure what was needed re. info.
Latest full dump is attached.
firewalluser, the crash happens even when OpenVPN is disabled (both gateway and service) - so I do not think it's openvpn causing it.
Thanks again folks!
ps. wrt hardware, it's running under esxi along with a number of other vm's. It had been running fine with no issues for months. None of the other vm's have issues. Incase it was a disk issue, I have tried cloning and also a fresh install (with reloaded config).
I am considering doing a fresh install and slowly adding in config. to see where it breaks but that is a REAL pain in the but! So hopefully the dump can narrow it down.
-
All of the backtraces are different but they all end in the same place, IPsec. Might be the same as one of the other IPsec crashes we've been tracking. Can you elaborate on your IPsec config (number of tunnels, ciphers used, etc)
-
Thanks for the quick response.
It is a very simple IPsec setup to enable remote VPN from an iPhone.
Let me know if you need more info. than the below screen caps.
I'll disable the IPsec VPN and see if the crashes stop.
Current Uptime 03 Hours 37 Minutes 37 Seconds
-
Uptime 1 Day 01 Hour 09 Minutes 00 Seconds
IPsec is looking like the culprit.
-
Uptime 2 Days 14 Hours 13 Minutes 21 Seconds
i think we have a winner!!
IPSec.
-
It's a bit strange, nothing on there would seem to be out of the ordinary… was the mobile IPsec device connected at all times? Or was it connected at all?
Curious if maybe the device was on at all times if it might have been timed such that the phone roamed from tower to tower or went to sleep/woke up, etc.
-
Hi Jimp.
The mobile was an occasional connection and had no correlation with the crashes.
I have since rebooted due to some isp issues. Once I get a chance, I'll turnthe ipsec back on and see how it behaves.
Thanks.
-
In android, you can down load free apps which will force your phone to use a particular cell tower, this will remove one variable namely you phone switching between cell towers as phone companies have software running on these towers to bunk users around to load balance the connection, but it can be overridden with a simple free app which also happens to make it harden to triangulate your position.
Dont know if similar apps exist on iphone or others, androids a bit of a free for all.
-
ok, so it's been running sold with no crash for a few days now with IPSec turned off…
Now, I'll turn it on and see what happens
-
Turned it on and it lasted 12 hours before crashing :(
-
So it's definitely IPsec then. We've seen some other IPsec crashes but I'm not sure we've seen anything that regular, especially for a mobile only tunnel.
We are bringing back a bunch of IPsec updates from FreeBSD as soon as we can, might be in a 2.2.5 release, though I don't think it's there yet.
-
You mentioned an iPhone, is that just a VPN for a single iPhone? If not, knowing which other devices and how many might help.
-
Thanks guys.
Yes, it's just a single iphone that accesses the IPSec VPN (mine). Note is will crash when the vpn is enabled but there has been no access via the iphone/vpn. So just being enabled will cause a crash…not accessing it.
Happy to provide whatever config., logs and do whatever tests you guys want to help narrow it down.
I guess I can also look at moving over to openvpn client on the iphone.
-
If you could get me a backup of your config, that would definitely help. Can PM it to me here, or email to cmb at pfsense dot org, or email me to arrange other means of transfer. I don't see a means of replicating from that, so that should help.
