pfsense crash - firewall has encountered an error

grocerylist

Hi,
I've never submitted a troubleshooting request, I'm not really sure if this is the appropriate section or what all is required for assistance.

I started getting the 'Netgate pfSense Plus has detected a crash report or programming bug. Click here for more information.' banner messages after upgrading to v22.05.

As far as I can tell, I'm not experiencing any outages. The message below says that it 'rebooted after panic' but I never noticed any downtime and never had to intervene to restore a connection.

I've tried looking through the crash report but it's cryptic to me. The only thing that stuck out to me was this:

savecore 101 - - reboot after panic: invalid packet in handshake queue

But I don't know what it means, nor how to resolve it.

I'm attaching the crash dump file.crash.txt

Please let me know if anything else is required. I also have multiple info.# and textdump.tar.# files.

Thanks!

stephenw10

The important parts are:
The backtrace

db0kdb.enter.default  bt
Tracing pid 0 tid 100262 td 0xfffff80151a7a740
kdb_enter() at kdb_enter+0x37frame 0xfffffe004709e8d0
vpanic() at vpanic+0x194frame 0xfffffe004709e920
panic() at panic+0x43frame 0xfffffe004709e980
wg_softc_handshake_receive() at wg_softc_handshake_receive+0x9c7frame 0xfffffe004709eaa0
gtaskqueue_run_locked() at gtaskqueue_run_locked+0x121frame 0xfffffe004709eb00
gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xd2frame 0xfffffe004709eb30
fork_exit() at fork_exit+0x7eframe 0xfffffe004709eb70
fork_trampoline() at fork_trampoline+0xeframe 0xfffffe004709eb70
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

And the panic string:

panic invalid packet in handshake queue
cpuid = 3
time = 1656887367
KDB enter panic

So it's something in Wireguard.

I can't see that referenced anywhere else. How are you using Wireguard?

Steve

stephenw10

At the command line try running: pkg info -x wireguard

You may have an incomplete upgrade that still has the old package version.

grocerylist

@stephenw10
Thanks for the reply. I'm using wireguard as a server so I can access my network remotely for various services that I host.

In regards to a possible incomplete upgrade with the old package version, would I uninstall/reinstall Wireguard? I looked and as far as I can tell, there's no option to backup/restore Wireguard settings if I deleted it.

It's been a while but I think I remember after completing the upgrade I seem to remember I wasn't able to connect with Wireguard from any devices. Then I saw that Wireguard needed to be upgraded, which I did and then I was able to connect again. So I'm able to connect via Wireguard but something could still be slightly off with the Wireguard upgrade?

pkg info -x wireguard returned:

pfSense-pkg-WireGuard-0.1.6_2
wireguard-kmod-0.0.20211105_1
wireguard-tools-lite-1.0.20210914_1

stephenw10

Hmm, those look correct. The only reference to that panic I could find was during 22.05 upgrade testing but that isn't what you're hitting.

The Wireguard config is retained as long as you have 'Keep Configuration' set in the main Wireguard settings. It's set by default.
It's worth reinstalling the package to be sure.

Steve

grocerylist

@stephenw10
Well, I deleted Winguard, reinstalled Winguard, started the Winguard service and confirmed I can still connect.

I've deleted the crash report. Other than waiting around to see if another crash report is generated, I'm not sure how to know if this has been resolved.

Thanks for digging through the crash log. If I get another crash report, I'll update again.

stephenw10

It's possible that happened during the upgrade before the package was updated. The crash shows the dumptime as: Sun Jul 3 152927 2022.
However is also shows 5 crashes and all the logged panics look to be the same. That seems unlikely to be from the upgrade.

If the crash report returns that will confirm it's current, yes.

Steve