Kernel panic 4-5 Nov (i386)
-
Well, doesn't this look kinda the same problem? And since you are the one fixing this I thought you might be interesting. If you do not want the info on bugs I'll happily install 1.2.3 again and leave the testing up to you…
-
Unfortunately, I am still fighting with the kernel panic :'( :'( :'( :'(
I had hopefully upgraded to "Sat Nov 27 03:13:22 EST 2010 "-Build via USB-Stick, which worked pretty fine.
Today, I tried to use the built-in update feature and… BANG! Kernel-Panic (Fatal Trap 12), see attached screenshot...
(the screenshot is from an older build, but the error is still the same)ermal, are you still fighting this problem? :-)
Hopefully there will be a solution soon :)
Best regards,
Christian
-
without a backtrace i cannot say that this is related to this.
-
I tested both the 27th and 29th snapshots, doing an upgrade to the 27th and then a clean install for the 29th. Kernel panic still pops up for me under the same conditions as before. Oddly, the panic doesn't happen until after the update / install has been in place for a while (1-4 hours?) If I install an update, both GUI and fetch and will work for a bit, then a couple hours later, I run fetch from the console or log into the GUI again and it bonks almost immediately.
I'll get a current backtrace for you in the next hour or two.
-
Ok - here's the panic and back trace from the 11/29 build.
FreeBSD/i386 (xxxxxx.xxx) (ttyu0) login: root Password: # fetch http://snapshots.pfsense.org/FreeBSD_RELENG_8_1/i386/pfSense_HEAD/update s/pfSense-Full-Update-2.0-BETA4-20101129-2231.tgz Kernel page fault with the following non-sleepable locks held: exclusive sleep mutex fxp0 (network driver) r = 0 (0xc36de018) locked @ /usr/pfS ensesrc/src/sys/kern/kern_mutex.c:147 KDB: stack backtrace: X_db_sym_numargs(c0ea7a7e,c3304a4c,c0a33fb5,93,0,...) at X_db_sym_numargs+0x146 kdb_backtrace(93,0,ffffffff,c144e8a4,c3304a84,...) at kdb_backtrace+0x29 witness_display_spinlock(c0ea9f96,c3304a98,4,1,0,...) at witness_display_spinloc k+0x75 witness_warn(5,0,c0ee832e,1,c35ab7f8,...) at witness_warn+0x20d trap(c3304b24) at trap+0x19e alltraps(c36e6100,dedeadc0,c36e6100,c36e6100,c3304bac,...) at alltraps+0x1b m_tag_delete_chain(c36e6100,0,df,0,c36de000,...) at m_tag_delete_chain+0x3f reallocf(c36e6100,100,0,9e3,c0ea29ae,...) at reallocf+0x8a5 uma_zfree_arg(c1d7e380,c36e6100,0,c36df1f0,c3304c20,...) at uma_zfree_arg+0x29 m_freem(c36e6100,c36e4d40,8,c36de000,c36cc800,...) at m_freem+0x43 fwohci_init(c36de018,4,c0e63eff,82a,c36de018,...) at fwohci_init+0x545c fwohci_init(c36de000,1,c0ea5a5d,189,c130de98,...) at fwohci_init+0x7a25 softclock(c130de60,c3304cc8,c09dfff4,c1311c00,c35d45b8,...) at softclock+0x24a intr_event_execute_handlers(c35ab7f8,c35d4580,c0e9f7b8,533,c35d45f0,...) at intr _event_execute_handlers+0x125 intr_event_add_handler(c35aa110,c3304d38,c0e9f51d,344,c35ab7f8,...) at intr_even t_add_handler+0x42f fork_exit(c09c8da0,c35aa110,c3304d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc3304d70, ebp = 0 --- Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xdedeadc0 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0a53258 stack pointer = 0x28:0xc3304b64 frame pointer = 0x28:0xc3304b74 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock) [thread] Stopped at m_tag_delete+0x48: movl 0(%ecx),%eax db> lock order reversal: (Giant after non-sleepable) 1st 0xc36de018 fxp0 (network driver) @ /usr/pfSensesrc/src/sys/kern/kern_mutex. c:147 2nd 0xc130d110 Giant (Giant) @ /usr/pfSensesrc/src/sys/dev/usb/input/ukbd.c:170 4 KDB: stack backtrace: X_db_sym_numargs(c0ea7a7e,c330489c,c0a33fb5,c0a24b9b,c0eaa9ea,...) at X_db_sym_n umargs+0x146 kdb_backtrace(c0a24b9b,c0eaa9ea,c355f040,c355e1a0,c33048f8,...) at kdb_backtrace +0x29 witness_display_spinlock(c0eaa9ea,c130d110,c0eceae9,c355e1a0,c0e93752,...) at wi tness_display_spinlock+0x75 witness_checkorder(c130d110,9,c0e93752,6a8,0,...) at witness_checkorder+0x839 _mtx_lock_flags(c130d110,0,c0e93752,6a8,c392ea80,...) at _mtx_lock_flags+0xc4 ucom_attach(c3c4b000,1,c130bd68,c1309f20,c3304978,...) at ucom_attach+0x1c08 ixgbe_init_fdir_perfect_82599(c35b2c00,1,c1309fa4,c130bd68,1,...) at ixgbe_init_ fdir_perfect_82599+0x6456 sc_attach_unit(c122daa0,78,c3304990,c09b0166,c33049b0,...) at sc_attach_unit+0x5 23 cncheckc(c33049b0,c0526fe5,c0e3bd0d,c0528290,c33049ac,...) at cncheckc+0x48 cngetc(c0e3bd0d,c0528290,c33049ac,c33049e8,1,...) at cngetc+0x16 db_readline(c12dad60,78,c33049cc,c0525c26,c0e3bd0d,...) at db_readline+0x75 db_read_line(c0e3bd0d,c3304a20,c0527add,c0ee41e5,0,...) at db_read_line+0x1a db_command_loop(c0ee41e5,0,c33049f4,c0e0cf9d,0,...) at db_command_loo3304b24,... ) at 0xed kdb_trap(c,0,c3304 db> db> bt Tracing pid 12 tid 64007 td 0xc35ad280 m_tag_delete(c36e6100,dedeadc0,c36e6100,c36e6100,c3304bac,...) at m_tag_delete+0 x48 m_tag_delete_chain(c36e6100,0,df,0,c36de000,...) at m_tag_delete_chain+0x3f reallocf(c36e6100,100,0,9e3,c0ea29ae,...) at reallocf+0x8a5 uma_zfree_arg(c1d7e380,c36e6100,0,c36df1f0,c3304c20,...) at uma_zfree_arg+0x29 m_freem(c36e6100,c36e4d40,8,c36de000,c36cc800,...) at m_freem+0x43 fwohci_init(c36de018,4,c0e63eff,82a,c36de018,...) at fwohci_init+0x545c fwohci_init(c36de000,1,c0ea5a5d,189,c130de98,...) at fwohci_init+0x7a25 softclock(c130de60,c3304cc8,c09dfff4,c1311c00,c35d45b8,...) at softclock+0x24a intr_event_execute_handlers(c35ab7f8,c35d4580,c0e9f7b8,533,c35d45f0,...) at intr _event_execute_handlers+0x125 intr_event_add_handler(c35aa110,c3304d38,c0e9f51d,344,c35ab7f8,...) at intr_even t_add_handler+0x42f fork_exit(c09c8da0,c35aa110,c3304d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc3304d70, ebp = 0 --- db> I noticed some KDB: stack backtrace messages in the boot up sequence also, so I've included those as well. [code] Configuring firewall.....lock order reversal: 1st 0xc12d8a0c pf task mtx (pf task mtx) @ /usr/pfSensesrc/src/sys/contrib/pf/n et/pf_ioctl.c:1397 2nd 0xc147a864 ifnet_rw (ifnet_rw) @ /usr/pfSensesrc/src/sys/net/if.c:2055 KDB: stack backtrace: X_db_sym_numargs(c0ea7a7e,d43de9c0,c0a33fb5,c0a24b9b,c0eaa9ea,...) at X_db_sym_n umargs+0x146 kdb_backtrace(c0a24b9b,c0eaa9ea,c3565638,c35627e8,d43dea1c,...) at kdb_backtrace +0x29 witness_display_spinlock(c0eaa9ea,c147a864,c0eb3e25,c35627e8,c0eb3e04,...) at wi tness_display_spinlock+0x75 witness_checkorder(c147a864,1,c0eb3e04,807,0,...) at witness_checkorder+0x839 _rw_rlock(c147a864,c0eb3e04,807,c392e930,3,...) at _rw_rlock+0x9c ifunit(c392e930,0,c0e3ab59,575,c0eb2f6d,...) at ifunit+0x27 pfioctl(c3923500,c0104414,c392e930,3,c3c37780,...) at pfioctl+0x2724 dev2udev(c3c62888,c0104414,c392e930,c35a8400,c3c37780,...) at dev2udev+0x898 kern_ioctl(c3c37780,8,c0104414,c392e930,1a2d860,...) at kern_ioctl+0x1fd ioctl(c3c37780,d43decf8,c0ee8261,c0ec0326,c3c3ad48,...) at ioctl+0x134 syscall(d43ded38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x281e0533, esp = 0xbfbfdd9c, ebp = 0xbfbfddd8 --- .done. Starting PFLOG...done. Setting up gateway monitors...No matching processes were found done. Synchronizing user settings...lock order reversal: 1st 0xcc04c260 bufwait (bufwait) @ /usr/pfSensesrc/src/sys/kern/vfs_bio.c:2559 2nd 0xc3c27000 dirhash (dirhash) @ /usr/pfSensesrc/src/sys/ufs/ufs/ufs_dirhash. c:285 KDB: stack backtrace: X_db_sym_numargs(c0ea7a7e,d44187c0,c0a33fb5,c0a24b9b,c0eaa9ea,...) at X_db_sym_n umargs+0x146 kdb_backtrace(c0a24b9b,c0eaa9ea,c3561fc8,c3565ec0,d441881c,...) at kdb_backtrace +0x29 witness_display_spinlock(c0eaa9ea,c3c27000,c0ed4887,c3565ec0,c0ed4515,...) at wi tness_display_spinlock+0x75 witness_checkorder(c3c27000,9,c0ed4515,11d,0,...) at witness_checkorder+0x839 _sx_xlock(c3c27000,0,c0ed4515,11d,c3c59d24,...) at _sx_xlock+0x85 ufsdirhash_enduseful(cc04c200,d4418944,34,cc1833e8,d44188ec,...) at ufsdirhash_e nduseful+0x2f5 ufsdirhash_add(c3c59d24,d4418944,3e8,d44188d8,d44188dc,...) at ufsdirhash_add+0x 13 ufs_direnter(c3957648,c3e55c90,d4418944,d4418bd4,0,...) at ufs_direnter+0x729 ufs_itimes(d4418bd4,0,d4418b30,d4418a8c,c0dba8d5,...) at ufs_itimes+0x14bd ufs_itimes(d4418b30,d4418b48,0,0,d4418ba8,...) at ufs_itimes+0x17b0 VOP_CREATE_APV(c12bd740,d4418b30,d4418bd4,d4418ac8,0,...) at VOP_CREATE_APV+0xa5 vn_open_cred(d4418ba8,d4418c5c,1a4,0,c35a8400,...) at vn_open_cred+0x215 vn_open(d4418ba8,d4418c5c,1a4,c3c63818,281df000,...) at vn_open+0x3b kern_openat(c3d31280,ffffff9c,bfbfd2f8,0,603,...) at kern_openat+0x125 kern_open(c3d31280,bfbfd2f8,0,602,1a4,...) at kern_open+0x35 open(c3d31280,d4418cf8,c,c3d31280,c3c3cd48,...) at open+0x30 syscall(d4418d38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip = 0x281a7a83, esp = 0xbfbfd2bc, ebp = 0xbfbfd708 --- done. Starting webConfigurator...done. Configuring CRON...done. Starting OpenNTP time client...done. lock order reversal: 1st 0xc3e5edf4 ufs (ufs) @ /usr/pfSensesrc/src/sys/kern/vfs_mount.c:1058 2nd 0xc3e848b8 devfs (devfs) @ /usr/pfSensesrc/src/sys/kern/vfs_subr.c:2090 KDB: stack backtrace: X_db_sym_numargs(c0ea7a7e,d4455800,c0a33fb5,c0a24b9b,c0eaa9ea,...) at X_db_sym_n umargs+0x146 kdb_backtrace(c0a24b9b,c0eaa9ea,c3565e58,c3565d88,d445585c,...) at kdb_backtrace +0x29 witness_display_spinlock(c0eaa9ea,c3e848b8,c0e96afc,c3565d88,c0eb1da5,...) at wi tness_display_spinlock+0x75 witness_checkorder(c3e848b8,9,c0eb1da5,82a,0,...) at witness_checkorder+0x839 __lockmgr_args(c3e848b8,80100,c3e848d4,0,0,...) at __lockmgr_args+0x7f5 vop_stdlock(d4455978,c0a33d5b,c0e96d43,80100,c3e84860,...) at vop_stdlock+0x62 VOP_LOCK1_APV(c1292060,d4455978,c3dea5a4,c12d68e0,c3e84860,...) at VOP_LOCK1_APV +0xb5 _vn_lock(c3e84860,80100,c0eb1da5,82a,8,...) at _vn_lock+0x5e vget(c3e84860,80100,c3dea500,15e,c0e96c5a,...) at vget+0xb9 devfs_allocv(c3ddf880,c3c4f284,d4455a10,9d,c147a63c,...) at devfs_allocv+0x102 devfs_rules_apply(c3c4f284,80000,d4455c30,430,0,...) at devfs_rules_apply+0x14a vfs_donmount(c3dea500,0,c3ddf980,c3ddf980,bfbfde09,...) at vfs_donmount+0x14c2 nmount(c3dea500,d4455cf8,c,c3dea500,c3e817f8,...) at nmount+0x75 syscall(d4455d38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280ee5db, esp = 0xbfbfdddc, eb p = 0xbfbfe338 --- Starting DHCP service...done. Starting DNS forwarder...done. Starting captive portal... load_dn_sched dn_sched FIFO loaded load_dn_sched dn_sched QFQ loaded load_dn_sched dn_sched RR loaded load_dn_sched dn_sched WF2Q+ loaded load_dn_sched dn_sched PRIO loaded done Configuring firewall......done. Generating RRD graphs...done. Starting CRON... done. Executing rc.d items... Starting /usr/local/etc/rc.d/*.sh...done. Bootup complete [/code] [/thread] -
Can you please test the new snapshots and see if this is fixed?
-
Will do. Thanks ermal!
Do you believe the update made it into the current builder run (looks like it started 10:40am EST), or do I need to wait for the build after this?
-
pfSense-Full-Update-2.0-BETA4-20101203-2137.tgz – 5 hours - flight normal. I hope further problems it will not be.
Thanks!
-
Ok, I've done several tests, need to do a couple more, but here's what I've discovered so far.
It appears that the GUI triggered panic may be cleared up. I'm still able to trigger a panic by calling fetch from the console under specific configuration. It appears to become a problem when CaptivePortal is enabled even though I'm not working from or through the CP interface. I'm not sure which feature in CP is the trigger, so I need to do a little more testing.
Since this thread was more related to the GUI caused kernel panic I suggest we mark this one as solved and I'll start a new thread for the CP related kernel panic once I've narrowed it down.
Ermal - Thanks again for tracking this down!
-
Unfortunately, it still doesn't work for me :-(
I just tried the December 5 Snapshot but I still get a panic when trying to update from the GUI…
I will try the most current snapshot tomorrow, just wanted to let you know ;DBest regards,
Christian
-
Trying anything but the most current snapshot would undoubtedly be problematic.
-
Update:
Tried the newest snapshot (20.12.2010) but I still get the kernel panic when I try the online update via GUI :-(Could I do s.th. to help you to figure this problem out?
Thanks and best regards,
Christian
-
Are you running captive portal?
-
PJ2,
thanks for your reply.
Actually yes, I have a CP set up… Do I have to deactivate it?Best regards,
Christian
-
turning off CP does the trick for me.
What network drivers are you using? fxp?
-
Actually yes, I have a CP set up… Do I have to deactivate it?
That's something to try, it may narrow down the issue considerably.
-
@PJ2:
I have these interfaces:
vr0 <- on-Board
ste <- 4-port 100Mbit Card (D-Link)@cmb:
I have now switched off CP and just clicked on "Auto Update". This time pfSense was able to check if there is a new version without a restart / kernel panic.
Tomorrow will be interesting, as then there'll be an update available.I will report back, if the update went OK tomorrow, or if I still got a kernel panic :-)
Thanks for your help anyway so far!!
Best regards,
Chris
-
Thanks Chris -
I've been trying to sort out what it is besides CP that makes the difference. I was hoping that switching NICs would solve the problem for me, but that doesn't seem to be the case. :-/
-
Hm… Well if CP really makes the difference, then I am sure that we can sort this, together with the dev, out somehow ;-)
They would be having the same issues if they turn on CP as well, wouldn't they? Or is it indeed coupled with the NIC-type?
I am eager to see what happens tomorrow ;-)
Best regards,
Chris
-
Update:
Tried the update today and… it worked ;DSo the problem really seems to be that I get a kernel panic with the update function if there is the CP enabled...
Strange isn't it?
Best regards
Christian
