[crash report] system went down and took 3h30 to self recover

[Le_zOU]

Hello,

I'm a new home pfsense user
it runs on a minisforum GK41 system

Version	2.7.2-RELEASE (amd64)
CPU Type	Intel(R) Celeron(R) J4125 CPU @ 2.00GHz
Current: 1700 MHz, Max: 2001 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No

I installed it last sunday and it worked fine since.
Today, while I was away, it went down.
about 3h30 later it came back up
(I got email alerts for another device being monitored on my home network.

Despite having a relatively long experience reading fault logs (essentially on storage array), I couldn't find the culprit in the crash logs.

This seems to be the relevant part of the logs.

Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address	= 0x18
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80d7cec0
stack pointer	        = 0x28:0xfffffe00845ffbe0
frame pointer	        = 0x28:0xfffffe00845ffc40
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi1: netisr 2)
rdi: fffff80112cd3300 rsi: 00000000000005d8 rdx: 0000000000000228
rcx: 0000000000000000  r8: 0000000000000201  r9: 0000000000000001
rax: 0000000000000595 rbx: 000000000000023c rbp: fffffe00845ffc40
r10: 0000000000000010 r11: 0000000000000000 r12: fffff80004f4bd00
r13: 0000000000000228 r14: 0000000000000595 r15: 0000000000000000
trap number		= 12
panic: page fault
cpuid = 2
time = 1706180648
KDB: enter: panic

here are the crash logs
info.0 textdump.tar.0

Thank you

[stephenw10]

Backtrace:

db:0:kdb.enter.default>  bt
Tracing pid 12 tid 100039 td 0xfffffe0011fb2720
kdb_enter() at kdb_enter+0x32/frame 0xfffffe00845ff8c0
vpanic() at vpanic+0x163/frame 0xfffffe00845ff9f0
panic() at panic+0x43/frame 0xfffffe00845ffa50
trap_fatal() at trap_fatal+0x40c/frame 0xfffffe00845ffab0
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00845ffb10
calltrap() at calltrap+0x8/frame 0xfffffe00845ffb10
--- trap 0xc, rip = 0xffffffff80d7cec0, rsp = 0xfffffe00845ffbe0, rbp = 0xfffffe00845ffc40 ---
m_copym() at m_copym+0x30/frame 0xfffffe00845ffc40
ip_fragment() at ip_fragment+0x255/frame 0xfffffe00845ffce0
ip_tryforward() at ip_tryforward+0x41a/frame 0xfffffe00845ffd90
ip_input() at ip_input+0x350/frame 0xfffffe00845ffdf0
swi_net() at swi_net+0x128/frame 0xfffffe00845ffe60
ithread_loop() at ithread_loop+0x257/frame 0xfffffe00845ffef0
fork_exit() at fork_exit+0x7f/frame 0xfffffe00845fff30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00845fff30
--- trap 0x7677f2e7, rip = 0xe8bf66d9600bad34, rsp = 0x8f44203226e4eacf, rbp = 0x3ee457881ec68306 ---

Your log is filled with:

<6>re0: watchdog timeout
<6>re0: link state changed to DOWN
<6>re0: link state changed to UP
<6>re0: watchdog timeout
<6>re0: link state changed to DOWN
<6>re0: link state changed to UP
<6>re1: watchdog timeout
<6>re1: link state changed to DOWN
<6>re1: link state changed to UP

If you can't change the NICs for something better than Realtek you can only try using the alternative driver from the realtek-re-kmod pkg.

You might also consider disabling logging ARP movements since that is also spamming your logs:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/logs-arp-moved.html

Steve

[Le_zOU]

@stephenw10 Thank you very much !!