Diagnosing a PFSense Crash Report

stephenw10

Yeah, that. If the crashes are frequent and identical then does swapping the NIC cause the fault to move to the new LAN NIC or remain on bce0 on the new interface?

Steve

solis

@gertjan i have only 4 lan ports bce3 is also a stand by port for backup internet i have no slot available to switch it with

Gertjan

@solis

Well, it's solved then : make your bce3 do what you bce0 now does.
Make bce0 your stand by / spare port.
That called swapping.

Remember :
bce0 doesn't need to be LAN WAN or OPTx
bce1 doesn't need to be LAN WAN or OPTx
bce2 doesn't need to be LAN WAN or OPTx
bce3 doesn't need to be LAN WAN or OPTx
LAN can be port 3 - WAN 2, spare 1 and 0 doing nothing.

solis

@gertjan brother i have 2 wans ready, 1 wan standby and 1 lan
all ports are full are you telling me crash are happening due to port bce0 ?

Gertjan

@solis said in Diagnosing a PFSense Crash Report:

are happening due to port bce0 ?

Exact.
So, - again - swap 'bce0' with 'bcex'. If the problem stays on 'bce0' you know that bce0 interface is bad.
If the issue shown before on bce0 is now on bcex, you know - pretty sure - that the NIC is ok.

solis

@gertjan [textdump.tar.1] i am still getting crashes

Gertjan

@solis said in Diagnosing a PFSense Crash Report:

[textdump.tar.1] i am still getting crashes

Where ? What interface ? Did you swap ?

What is :
[textdump.tar.1]

viktor_g

It could be a bce(4) bug:
https://bugs.freebsd.org/bugzilla/buglist.cgi?quicksearch=%5Bbce%5D

stephenw10

Mmm, looks like the attachment failed.

Does the crash follow the hardware or is it still on the same pfSense interface?

Steve

solis

@stephenw10 no it stay on pfsense

solis

@solis textdump.tar.0 check it i change the interface with bce2 but i dnt know maybe this crash report is old

stephenw10

Shows:

db:0:kdb.enter.default>  show pcpu
cpuid        = 2
dynamic pcpu = 0xfffffe0080e3d380
curthread    = 0xfffff808337bd740: pid 12 tid 100185 "irq257: bce1"
curpcb       = 0xfffff808337bdce0
fpcurthread  = none
idlethread   = 0xfffff80004084000: tid 100005 "idle: cpu2"
curpmap      = 0xffffffff8368d5a8
tssp         = 0xffffffff837176f0
commontssp   = 0xffffffff837176f0
rsp0         = 0xfffffe00ba05fe00
kcr3         = 0x8000000003d03002
ucr3         = 0xffffffffffffffff
scr3         = 0x8d1f6f8ae
gs32p        = 0xffffffff8371df08
ldt          = 0xffffffff8371df48
tss          = 0xffffffff8371df38
tlb gen      = 10148
curvnet      = 0
db:0:kdb.enter.default>  bt
Tracing pid 12 tid 100185 td 0xfffff808337bd740
kdb_enter() at kdb_enter+0x37/frame 0xfffffe00ba05f8d0
vpanic() at vpanic+0x197/frame 0xfffffe00ba05f920
panic() at panic+0x43/frame 0xfffffe00ba05f980
trap_fatal() at trap_fatal+0x391/frame 0xfffffe00ba05f9e0
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00ba05fa30
trap() at trap+0x286/frame 0xfffffe00ba05fb40
calltrap() at calltrap+0x8/frame 0xfffffe00ba05fb40
--- trap 0xc, rip = 0xffffffff805d97cb, rsp = 0xfffffe00ba05fc10, rbp = 0xfffffe00ba05fc90 ---
bce_intr() at bce_intr+0x48b/frame 0xfffffe00ba05fc90
ithread_loop() at ithread_loop+0x23c/frame 0xfffffe00ba05fcf0
fork_exit() at fork_exit+0x7e/frame 0xfffffe00ba05fd30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ba05fd30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

current process		= 12 (irq257: bce1)
trap number		= 12
panic: page fault
cpuid = 2
time = 1643621859
KDB: enter: panic

1643621859 is Monday, 31 January 2022 09:37:39

So showing in bce1. But you swapped bce0 with bce2?

<118>WAN  -> bce1
<118>LAN  -> bce2
<118>OPT1 -> bce3
<118>OPT2 -> bce0

It looks like you might have re-assgined all of them....

solis

@stephenw10 i did swap lan on bce2 but today my device crash again when i check server display was on boot when boot complete configuration was deleted please help me recent crash report textdump.tar.0 info.0 please help me if possible should i change server or should a create vm server and install on it

solis

@stephenw10 textdump.tar.0 info.0 please help latest crash report

Gertjan

@solis

Are the bce0-3 network interfaces on a PCI card ? if so, you could change the entire card form some other quad NIC Intel based card.

@solis said in Diagnosing a PFSense Crash Report:

recent crash report textdump.tar.0 info.0

Well, you can see it yourself now : line 11 :

curthread    = 0xfffff808337bf740: pid 12 tid 100185 "irq257: bce1"

IRQ handling for bce1 crashed.

solis

@gertjan no its its not pci card its motherboard nic should i change server will it work?

Gertjan

@solis

I had some NIC's dying on me, during the last 10 years or so. But the were all PCI card NIC's - so, when dead, I swap them.

Your solution : get a good 4 port NIC, put in a 'big' type type device, fire up a VM (Hyper-V on Windows 10 will do, as I know it works as I'm doing that @home) and you install pfSense in the VM : works fine.

Can you tell us what you're using right now ?

solis

@gertjan i have installed pfsense directly to raid bootable linux should i change server or just create it on vm it will better with it? with these port because getting nic pci card is not an option for me as i have spare server avaliable

Gertjan

@solis said in Diagnosing a PFSense Crash Report:

i have installed pfsense directly to raid bootable linux

Don't know what that is - looks complicated to me.
What about a dedicated device ? Or a VM ?

@solis said in Diagnosing a PFSense Crash Report:

should i change server

pfSense is not a server - pfSense doesn't "serve" anything.
It's a router (firewall).

@solis said in Diagnosing a PFSense Crash Report:

just create it on vm it will better with it?

If it's ok for you it's ok for me.
Best are these.
Or build your own "motherboard + quand NIC PCI card + power.
Or create a VM, assign 4 NIC ports.
Just my opinion of course.

@solis said in Diagnosing a PFSense Crash Report:

is not an option for me as i have spare server avaliable

Wait .... Language issue here ?
You have a spare server device - but that's not an option. Then why even mentioning it ?

You issue seesm easy to my :
You have a bad NIC - or multiple bad NIC's.
The NIC are build onto the motherboard, so you have to use another solution.
Pick the one that pleases you most.
No one can say what is "better" - that's up to you.

stephenw10

I'm unclear what you swapped here. The crashes were initially in bce0 and now they appear to be in bce1, so that looks like the crash following the assignment.

But you said you swapped bce2?

It could be failing hardware. It could be a bce(4) bug. Either way if you cannot swap out the bce NIC I would suggest installing on something different if you can.

Steve