pfsense has detected a crash report 2.5.0 upgraded today

KpuCko

Hello,
How all has started? Today there was power outage in my home. Even I have UPS and pfSense has configured to read the UPS and shutdown on power outage I ended up with broken pfSense after reboot, how I knew that - I noticed LLDP daemon is flagged as red. I tried to start it via WebUI but it refused to start, then I went to the console and ran lldpd and an error has occurred - something mentioned about missing so libraries.

Then I tried to re-install the package but pfSense hanged on the page of package installation. I tried to remove it and install it again, without success.
Then I realized there is a new version 2.5.0 which is marked as stable, so I assumed when I upgrade the whole system to latest, all the packages will be reinstalled and the issue will be fixed.

And I think I did it, the upgrade went fine, the system rebooted and booted up again fine, all good. After few hours I heard the sound which is played when the system boot up. Then I logged to the console and noticed that crash message report.

I'm attaching the files here, not sure if this can help or I have to completely reinstall my system.

info.0 textdump.tar.0

jimp

Do you have traffic shaping enabled?

The backtrace in the crash dump you posted looks like one someone else posted that they claimed was related to CBQ traffic shaping.

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x28
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80ec014e
stack pointer	        = 0x28:0xfffffe00748ee850
frame pointer	        = 0x28:0xfffffe00748ee880
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi4: clock (0))
trap number		= 12
panic: page fault
cpuid = 0
time = 1613671545
KDB: enter: panic

db:0:kdb.enter.default>  bt
Tracing pid 12 tid 100035 td 0xfffff800055ed000
kdb_enter() at kdb_enter+0x37/frame 0xfffffe00748ee510
vpanic() at vpanic+0x197/frame 0xfffffe00748ee560
panic() at panic+0x43/frame 0xfffffe00748ee5c0
trap_fatal() at trap_fatal+0x391/frame 0xfffffe00748ee620
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00748ee670
trap() at trap+0x286/frame 0xfffffe00748ee780
calltrap() at calltrap+0x8/frame 0xfffffe00748ee780
--- trap 0xc, rip = 0xffffffff80ec014e, rsp = 0xfffffe00748ee850, rbp = 0xfffffe00748ee880 ---
ether_8021q_frame() at ether_8021q_frame+0x2e/frame 0xfffffe00748ee880
vlan_transmit() at vlan_transmit+0xc8/frame 0xfffffe00748ee8f0
vlan_altq_start() at vlan_altq_start+0xb4/frame 0xfffffe00748ee920
cbqrestart() at cbqrestart+0x64/frame 0xfffffe00748ee950
rmc_restart() at rmc_restart+0x6f/frame 0xfffffe00748ee980
softclock_call_cc() at softclock_call_cc+0x141/frame 0xfffffe00748eea30
softclock() at softclock+0x79/frame 0xfffffe00748eea50
ithread_loop() at ithread_loop+0x23c/frame 0xfffffe00748eeab0
fork_exit() at fork_exit+0x7e/frame 0xfffffe00748eeaf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00748eeaf0

KpuCko

@jimp I have removed all traffic shapers and rebooted the router.

Now we have to wait.

KpuCko

By now the router has more then 4 days uptime, so we can safely consider that the issue is in traffic shaping

jimp

OK, keep an eye on https://redmine.pfsense.org/issues/11470 which is where we are tracking this specific issue.

KpuCko

@jimp Thanks, will do!