My setup with pfSense 2.7.2 crashes daily

toni8

Hi everyone!

My setup is a mini PC with i5-1165G7, 4G Ram, Intel i226 2.5G
( https://www.aliexpress.com/item/1005004508816806.html?spm=a2g0o.order_list.order_list_main.158.4db81802VBkEDu )

I bought it 2 years ago and had some hiccups, the power adapter was faulty and the pc rebooted after ~ 1 week of continually running. After the swap, it was fine, it rebooted rarely but without any issues after.

This year has gotten worse, after the upgrade cu 2.7.2, it started crashing more often , currently is crashes almost every 24h.

The crashes are in 3 different ways,

it crashes, store a dump and reboot, that's the happy scenario.
most of the time, it crashes, it remains like that and it is or not responsive on keyboard press.
sometimes it's a wall of text scrolling fast and at some point it gets stuck.

Each time i need to force shutdown from the button. To made things worse if filesystem is under ZFS , it will just corrupt itself and can't even boot, i need a full pfsense reinstall almost on every crash. (i attempted repairs on pool and zfs scrub but still doesn't boot)
I reinstalled it under UFS now , it more resistant on crashes but now i have inode hash check fails

I love pfSense for its features, but man when there are problems, it fails so spectaculous, it almost made me go explore other solutions.

Here are the dump that the system manage to collect, most of them aren't there.

What kernel panic corrupts the filesystem in such a manner to not be recoverable ?

Unfortunately, I do not have enough knowledge to understand the reason for the failures from the dump. Please help me understand the reason for the failures. I would like some kind of indication what can be wrong. Maybe a bios setting is not ok, ethernet nic/driver issues, sleep / power state issues, etc.

here are some video proof and pictures here: https://drive.google.com/drive/folders/1sHjPu40powFgP_rfhJ6pgBRMUuirHAyd?usp=sharing

I checked the power supply, did a memtest on RAM and swapped the nvme ssd with a sata ssd.

These months i have migrated my pfsense config to a virtual machine on other server which ran with no issues.

PS. For those who throws faulty hardware first (not excluding the possibility), I loaded windows 11 on it and used it for 1 week without any issues or crashes (The only thing i noticed is the nvme overheated and stalled the system but it recovered slowly, that's why i swapped the drive)

I have read this post:
https://forum.netgate.com/topic/189717/pfsense-crashing-randomly-pfsnese-plus-24-03/6 and applied the patch.

stephenw10

All of those crashes are different. Randomisation like that alm ost always indicates some sort of hardware issue. I would also have checked the RAM first. Did you run memtest through a few cycles?

toni8

@stephenw10 I did 2 cycles. How many do you recommend?

stephenw10

At least 3. You might also try a different memory test to be sure. Because that really looks like a hardware/ram issue.

toni8

@stephenw10
You were right, it was the ram !

I did a memetest and ended up with 4 passes but 0 errors. That was strange.

I ended buying a random stick of ram with same specs and replace it and also putting it in other ram slot. It's solid for 4 days now.