25.03-BETA: pftop core dump

patient0

While trying to see the connections when testing PREF64 pftop did crash quite often. But not always. And if it did crash it was when I set the filter to icmp or started it with that filter pftop -f icmp.

pfSense+ 25.03-BETA is running as a VM on Proxmox, interfaces are vtnetX.

Attached two core dumps:

pftop.core_20250216-1010CET.tgz
pftop.core_20250216-0935CET.tgz

Additional infos: it failes with the message "Bus error (cored dumped)". And I run pftop from the shell.

stephenw10

So always when running manually at the CLI? Not in the GUI? or from the console menu?

patient0

@stephenw10 after @fireodo comments on the "25.03-BETA NAT64 / PREF64 Success and feedback" thread I tested it on 2.7.2 CE too. And it does fail there too while having a ping 1.1.1.1 running on a client.

Same behavior in 25.03-BETA and 2.7.2 CE:
No crash from the GUI. But pretty reliable crashing when calling pftop -f icmp from the shell (right now everytime). Calling only pftop and then setting the filter to icmp crashes it sometimes (right now all the time) but at times not very often (earlier this day).

Testing the console menu the behavior is the same as calling it from the shell, it crashes quite often when setting the filter to icmp while running a ping on a client. The ssh is then not usable anymore for me.

Below the screenshot of the moment when the pftop started from the console crashes (two screenshots merged from the VNC of the VM, not everything fitted on the screen).

Screenshot 2025-02-17 at 19.23.14.jpeg

patient0

Addition: just tested on my 24.11 "production hardware" (OPNsense DEC740), same thing happens.

pftop -f icmp core dumps right away and running pftop with filter set to icmp crashes it eventually. The WAN dpinger are enough, don't have to run a ping myself manually.

stephenw10

Mmm, OK that's pretty easy to replicate! And, yeah, seems to affect a number of versions.

stephenw10

Looks like it's covered by this: https://redmine.pfsense.org/issues/15595

patient0

@stephenw10 does indeed sound like it. The linked FreeBSD bug is interesting, don't think pftop go patched with what Kristof suggested.

Addition: I was able to reproduce it on a 15-CURRENT from 20250213 with a minimal pf.conf. It does take longer when started without arguments and setting the filter to icmp but it crashes after several minutes while running two ping-s. Calling pftop -f icmp crashes right away.

For reference the lldb backtrace

pfTop: Up State no entries (1), View: default, Order: none, Cache: 1000010:34:44
Process 5397 stopped
* thread #1, name = 'pftop', stop reason = signal SIGBUS: object-specific hardware error
    frame #0: 0x00002ef884a7dfae pftop`print_state(s=0x00003e14818312b0, ent=0xa5a5a5a5a5a5a5a5) at pftop.c:990:31
   987          }
   988          print_fld_uint(FLD_RULE, s->rule);
   989          if (cachestates && ent != NULL) {
-> 990                  print_fld_rate(FLD_SI, ent->rate);
   991                  print_fld_rate(FLD_SP, ent->peak);
   992          }
   993
(lldb) bt
* thread #1, name = 'pftop', stop reason = signal SIGBUS: object-specific hardware error
  * frame #0: 0x00002ef884a7dfae pftop`print_state(s=0x00003e14818312b0, ent=0xa5a5a5a5a5a5a5a5) at pftop.c:990:31
    frame #1: 0x00002ef884a7cbc6 pftop`print_states at pftop.c:1004:12
    frame #2: 0x00002ef884a84050 pftop`disp_update at engine.c:883:4
    frame #3: 0x00002ef884a84a68 pftop`engine_loop(countmax=0) at engine.c:1213:4
    frame #4: 0x00002ef884a81139 pftop`main(argc=0, argv=0x00002f00a510afc0) at pftop.c:2322:2
    frame #5: 0x00002f00a8a5a404 libc.so.7`__libc_start1(argc=1, argv=0x00002f00a510afb8, env=0x00002f00a510afc8, cleanup=<unavailable>, mainX=(pftop`main at pftop.c:2178)) at libc_start1.c:172:7
    frame #6: 0x00002ef884a7be2d pftop`_start at crt1_s.S:83

stephenw10

Fix is incoming.