Crashing on new hardware with Fatal trap 12: page fault while in kernel mode

taenzerme

Thanks, I downgraded using the integrated firmware update function with the full_version upgrade .tgz from that build. Let's see how that works.

doktornotor

Confirmed… The latest batch of FTP-related patches seems really bad, managed to crash my Alix testbox repeatedly… As simple as browsing to ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/Latest/ e.g., not even required to upload/download anything.


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x70
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc04c96c1
stack pointer           = 0x28:0xe31e6c20
frame pointer           = 0x28:0xe31e6c2c
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 7 (pfpurge)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 1d1h13m36s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort

Please revert those ASAP. :(

taenzerme

July 1st build - no crashes so far. Crashed almost every 15 minutes before.

doktornotor

@taenzerme:

July 1st build - no crashes so far. Crashed almost every 15 minutes before.

https://redmine.pfsense.org/issues/2650#note-19

kejianshi

I don't know. That could be coincidental.

Example. I have a machine here that I built 8 years ago and I over clocked it 8 years ago and it worked flawlessly every single day with no issues 24/7 without power offs. Later, I tried installing Linux and it would reliably crash after a few minutes, so I figured "Must be a compatibility issue with linux mint". Well, I removed the over clock settings and as it turns out, Mint is now rock solid on that machine.

So its was stable under XP overclocked but not with Mint - But it wasn't Mint's fault.

In nearly every case where an OS was unstable and I was certain it must be the OS it turned out to be hardware in in every single instance prime95 would expose that in short order. Most recently this was with a new mobo with new ram and processor and turned out the capacitors on a ram bank were weak.

doktornotor

@kejianshi:

I don't know. That could be coincidental.

I reproduced this on 3 different boxes (2 amd64, 1 x86) with post-July1 snapshots. Does not happen on any of them with previous snapshots that don't include the faulty FTP patches.

kejianshi

Then I guess you are right. Thats pretty solid evidence that its a problem with the patch. Makes sense.

eri--

I made fixes to prevent the panics.
Check later coming snapshots better from tomorrow ones.

taenzerme

Sounds good, thanks!

doktornotor

Thanks, will test tomorrow…

DrPit

Hi I have the same problem and reverted to 1st July version. I don't get crashes anymore but I also have problem on ftp traffic. The connections are too slow or I get a lot of disconnection from ftp server.

taenzerme

Here, too. FTP connections are very unrealibly with July 1st version.

taenzerme

Has anyone tested the latest releases? Is it safe to upgrade again?

athurdent

Seems to be stable again, but you can always downgrade with a backup.tgz on full installs or switch slices on nano.
FTP still does not work very good with the sites I have tested, tough.

Reiner030

Hi,
we have this crashes too… dependent on hardware it gaves a TRAP 12 (attached image with firmware from July, 4th).
Other hardware get stucked without any hint what happened.
Yesterday evening updated to
2.1-RC0 (amd64)
built on Mon Jul 8 21:53:11 EDT 2013
FreeBSD 8.3-RELEASE-p8
because last possible version to download was from July, 2nd. But still same problem…

Has someone a link to a stable version from July, 1st please (older version is from April, 22th so rolling back is not so nice)?
Thanks.

gw1-jws1.jpg_thumb

doktornotor

No such problem with Tue Jul 9 11:08:17 EDT 2013 FreeBSD 8.3-RELEASE-p8. However, the patches do nothing useful to improve FTP, in fact they break things badly here (not as in kernel panic, but breaking FTP functionality which was working perfectly fine before).

https://redmine.pfsense.org/issues/3077#note-3

I don't see how these regressions are good at this stage. There is a crapload more users with normal FTP usage, as opposed to esoteric multi-WAN scenarios these were supposed to improve.

Reiner030

Here the actual crashdump output of yesterday firmware:

Fatal trap 12: page fault while in kernel mode
cpuid = 4; apic id = 04
fault virtual address   = 0x90
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff802122cf
stack pointer           = 0x28:0xffffff8055b9cac0
frame pointer           = 0x28:0xffffff8055b9cad8
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 8 (pfpurge)

I found in it also Interesting snippet from yesterday install/update part which could cause problems, too:
<118>Configuring crash dumps…
<118>Using /dev/ad6s1b for dump device.
<118>Mounting filesystems...
ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior.
Consider tuning vm.kmem_size and vm.kmem_size_max
in /boot/loader.conf.
ZFS filesystem version 5
ZFS storage pool version 28
<118>Disabling APM on /dev/ad6

Edit these are the default parameters setup by installer (June 4th):
[2.1-RC0][root@gw1.zws8.local]/root(1): cat /boot/loader.conf
autoboot_delay="3"
vm.kmem_size="435544320"
vm.kmem_size_max="535544320"
kern.ipc.nmbclusters="0"
hw.usb.no_pf="1"

athurdent

@doktornotor:

I don't see how these regressions are good at this stage. There is a crapload more users with normal FTP usage, as opposed to esoteric multi-WAN scenarios these were supposed to improve.

Well, FTP is broken on my second WAN link, too. I could swear it used to work about 2 month ago…

Reiner030

@Reiner030:

I found in it also Interesting snippet from yesterday install/update part which could cause problems, too:
Edit these are the default parameters setup by installer (June 4th):
[2.1-RC0][root@gw1.zws8.local]/root(1): cat /boot/loader.conf
autoboot_delay="3"
vm.kmem_size="435544320"
vm.kmem_size_max="535544320"
kern.ipc.nmbclusters="0"
hw.usb.no_pf="1"

mmh, I changed this to
vm.kmem_size="536870912"
vm.kmem_size_max="1073741824"
didnt helped. It crashed 3 time again since then…

taenzerme

Has anyone updated to the latest snapshots? Is it safe to upgrade again? Any feedback from you guys here?