25.03-BETA: pftop core dump

patient0

Addition: just tested on my 24.11 "production hardware" (OPNsense DEC740), same thing happens.

pftop -f icmp core dumps right away and running pftop with filter set to icmp crashes it eventually. The WAN dpinger are enough, don't have to run a ping myself manually.

stephenw10

Mmm, OK that's pretty easy to replicate! And, yeah, seems to affect a number of versions.

stephenw10

Looks like it's covered by this: https://redmine.pfsense.org/issues/15595

patient0

@stephenw10 does indeed sound like it. The linked FreeBSD bug is interesting, don't think pftop go patched with what Kristof suggested.

Addition: I was able to reproduce it on a 15-CURRENT from 20250213 with a minimal pf.conf. It does take longer when started without arguments and setting the filter to icmp but it crashes after several minutes while running two ping-s. Calling pftop -f icmp crashes right away.

For reference the lldb backtrace

pfTop: Up State no entries (1), View: default, Order: none, Cache: 1000010:34:44
Process 5397 stopped
* thread #1, name = 'pftop', stop reason = signal SIGBUS: object-specific hardware error
    frame #0: 0x00002ef884a7dfae pftop`print_state(s=0x00003e14818312b0, ent=0xa5a5a5a5a5a5a5a5) at pftop.c:990:31
   987          }
   988          print_fld_uint(FLD_RULE, s->rule);
   989          if (cachestates && ent != NULL) {
-> 990                  print_fld_rate(FLD_SI, ent->rate);
   991                  print_fld_rate(FLD_SP, ent->peak);
   992          }
   993
(lldb) bt
* thread #1, name = 'pftop', stop reason = signal SIGBUS: object-specific hardware error
  * frame #0: 0x00002ef884a7dfae pftop`print_state(s=0x00003e14818312b0, ent=0xa5a5a5a5a5a5a5a5) at pftop.c:990:31
    frame #1: 0x00002ef884a7cbc6 pftop`print_states at pftop.c:1004:12
    frame #2: 0x00002ef884a84050 pftop`disp_update at engine.c:883:4
    frame #3: 0x00002ef884a84a68 pftop`engine_loop(countmax=0) at engine.c:1213:4
    frame #4: 0x00002ef884a81139 pftop`main(argc=0, argv=0x00002f00a510afc0) at pftop.c:2322:2
    frame #5: 0x00002f00a8a5a404 libc.so.7`__libc_start1(argc=1, argv=0x00002f00a510afb8, env=0x00002f00a510afc8, cleanup=<unavailable>, mainX=(pftop`main at pftop.c:2178)) at libc_start1.c:172:7
    frame #6: 0x00002ef884a7be2d pftop`_start at crt1_s.S:83

stephenw10

Fix is incoming.

pst

@stephenw10 I'm starting to see frequent core dumps in the May1 beta. The reason is different though, here's a snippet from running /status.php:

Firewall-pftop Rules

<jemalloc>: jemalloc_extent.c:1195: Failed assertion: "p[i] == 0"

stephenw10

Any more to go on? Like it does that every time you try to generate the status_output?

What about if you check the Rules view in Diag > pfTop?

Or directly run the command?: /usr/local/sbin/pftop -w 150 -a -b -v rules

pst

@stephenw10 said in 25.03-BETA: pftop core dump:

/usr/local/sbin/pftop -w 150 -a -b -v rules

I have used /status.php a lot recently so the entries in the system log most likely relates to those runs.

When I run the command directly I get

 605  90       Any     Q !igb0  sprite-rp          0        0        *       pass inet6 from <pfB_CF_IP_Whitelist_v6> to any port = http user group  f
Bus error (core dumped)

but I'm not sure if its related to a specific rule, if I pipe the command through tail I get a different result:

595  12       Any       igb0   tcp                0        0        0      Bus error (core dumped)
[25.03-BETA][root@pfsense.local.lan]/root:

stephenw10

Mmm, OK I replicated that. I don't think it's rule specific. Seems like a bug in pftop when called with those options.

stephenw10

Hmm, nope more specific than that. I can only replicate it on two test boxes. Most 25.03 instances don't hit this.

pst

@stephenw10 pftop seems a bit flakey. I actually managed to view the rules a couple of times today by just starting pftop and then navigate to rules. But now that doesn't work any more. Running the full command gives randomly either bus error, segmentation fault or the assert