NDP Table Data Missing (GUI)

[RobbieTT]

23.09.a.20230905.0551

No NDP data:

Sep 6 17:07:38	kernel		pid 79239 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)

The error persists through reboots.

CLI works as normal with ndp -a so the issue is confined to the GUI alone.

️

[jimp]

Curious I was able to make this happen just once on one system here but then I refreshed and it worked, and now I can't make it happen again.

The GUI page just runs NDP same as you would from the CLI, but it uses ndp -na

[jimp]

I opened https://redmine.pfsense.org/issues/14755 to track this, though I tried on quite a few different systems and couldn't make it happen again.

[RobbieTT]

@jimp
Just checked and both ndp -a and ndp -na work just fine via the CLI.

I also noticed this at the end of the CLI table this time around:

Segmentation fault (core dumped)

I don't recall seeing that fault message before.

<118>Bootup complete
<6>pid 54956 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>pid 98799 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>pid 36036 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>pid 52178 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>pid 79239 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>pid 95436 (ndp), jid 0, uid 0: exited on signal 11 (core dumped)
<6>ng0: changing name to 'pppoe0'
pf_test6: kif == NULL, if_xname pppoe0
<6>ng0: changing name to 'pppoe0'

️

[RobbieTT]

@jimp

For reasons unexplained the NDP table became populated again. Nothing has changed since yesterday or this morning but the data is back:

Weird but yes, it is intermittent, if you give it long enough.

[edit: Post reboot the NDP Table is empty again.]

️

[RobbieTT]

@jimp

Ok, after a few days of testing it seems the NDP Table only becomes populated after around 24 hours post-boot. This may be linked to previous observations that the idle/offline flags on the Status/DHCPv6 Leases page are invariably incorrect and that MAC and Hostnames are typically missing.

It is like the whole IPV6 stack has a gap between its normal functioning and the GUI monitoring and display pages, as the raw data via the CLI is correct.

[Issue remains present with v23.09.a.20230907.0600 too.]

️

[cmcdonald]

@RobbieTT

I have built ndp with debug symbols for testing. It would be great to get a core dump from this version

https://redmine.pfsense.org/issues/14755#note-3

[RobbieTT]

@cmcdonald

I guess that is on me but I've no real idea how to install this or recover back to normal again.

I've downloaded the file I can see that a couple of move files / mv command's are needed but don't presume I know what I am doing - both pfSense and BSD are new to me. Oh and I only have my in-production Netgate devise to test with (gulp!).

That said, my NDP table has vanished again this evening...

...so perhaps that is a sign I should crack-on with this.

️

[cmcdonald]

@RobbieTT

1. Download the ndp.full file from Redmine (this is the ndp binary built without compiler optimizations and with debugging symbols).
2. Navigate to Diagnostics > Command Prompt and upload ndp.full to pfSense, it should report Uploaded file to /tmp/ndp.full.
3. Still on Diagnostics > Command Prompt, enter mv /usr/sbin/ndp /usr/sbin/ndp.orig , This just renames your existing ndp binary so that the next step doesn't clobber it.
4. Now run mv /tmp/ndp.full /usr/sbin/ndp and then chmod +x /usr/sbin/ndp. The first command moves the new binary into the well-known location and the second command ticks the execute bit so that it can be, well, executed.

I would go ahead too and purge any existing core files with rm -f /root/*.core. Once you get an empty NDP Table again, there is a great chance that you've hit the issue and a new ndp.core file will be at /root/ndp.core, download this file and share it with me cmcdonald<at>netgate<dot>com

Thanks

[RobbieTT]

@cmcdonald
Perfect.

️

[cmcdonald]

@RobbieTT Any updates? We are eager to obtain a core dump from anyone who can replicate using the debugging build of ndp binary

[RobbieTT]

@cmcdonald

Not yet as this is my only pfSense device so I have to wait until the dark hours of the weekend to attempt it (remote working and all that).

I've kinda got used to having 2 routers when I was running EdgeRouters; it's more difficult to test things with only 1. I'm sure I will get another device at some point.

️

[RobbieTT]

@cmcdonald said in NDP Table Data Missing (GUI):

@RobbieTT Any updates? We are eager to obtain a core dump from anyone who can replicate using the debugging build of ndp binary

You should now have 2 emails - first with the core dump as per instructions and the second taken after a system reboot.

Good luck.

️

[cmcdonald]

@RobbieTT Thanks for the core. We were able to track this one down!

[RobbieTT]

@cmcdonald said in NDP Table Data Missing (GUI):

@RobbieTT Thanks for the core. We were able to track this one down!

Glad to help you and the team.

️

[marcosm]

@RobbieTT Has this happened since the fix?

[RobbieTT]

@marcosm
No, the NDP table has been fully populated in the GUI since the fix.

️