DHCP lease screen not loading

DrQuinn24

@gertjan Here is what is seen when using the instruction for Firefox when choosing Status/DHCP leases:

I have had the same issue since upgrading to 2.5 ( currently on 2.5.1). The only major change prior to 2.5 that I can recall on my system was installing the NextDNS cli.

chance

@drquinn24 It's timing out just like it was for everyone above. the solution is to delete your leases database and restart the dhcp service. That "solves" the problem for an undetermined amount of time, but at the cost of you now don't know what leases are out.

Nothing in 2.5.1 is related to this issue, and I'm not even sure it's going to be a priority for netgate anytime soon.

karlfife

@chance
Has anyone reported this as a bug?

This problem appears to be a function of how many entries are in the dhcp.leases file which DO NOT contain a client-hostname field. MANY offending entries may cause a delay long enough to create a timeout condition.

Reproducable like this:

With zero entries in the dhcp.leases file, on my hardware it takes LESS THAN one second to load the dhcp leases page. Note that my page also renders 36 additional static mappings, even with an empty leases file.

The following entry appears in my dhcp.leases file after some period of time.

lease 10.179.11.33 {
  starts 5 2021/04/16 02:21:58;
  ends 5 2021/04/16 04:21:58;
  tstp 5 2021/04/16 04:21:58;
  cltt 5 2021/04/16 02:21:58;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet e8:4e:06:7a:b1:e2;
  set vendor-class-identifier = "android-dhcp-10";
  client-hostname "Atari5200";
}

Page load time is still under 1 sec. After some time, the next entry appears in dhcp.leases. Note that the new one has no name!

lease 10.179.11.70 {
  starts 5 2021/04/16 02:44:01;
  ends 5 2021/04/16 04:44:01;
  tstp 5 2021/04/16 04:44:01;
  cltt 5 2021/04/16 02:44:01;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet 34:af:b3:c5:f2:d2;
  set vendor-class-identifier = "android-dhcp-9";
}

In this state, on my specific hardware, it takes about 6 seconds to load the dhcp leases page. I'm suggesting that the second entry is causing the problem. To fix the issue, open the dhcp.leases file in vi, and add the following line to the second lease.

  client-hostname "Commodore64-10GBase-LR";

If you figure out how to save/exit from vi, the file looks like this:

lease 10.179.11.33 {
  starts 5 2021/04/16 02:21:58;
  ends 5 2021/04/16 04:21:58;
  tstp 5 2021/04/16 04:21:58;
  cltt 5 2021/04/16 02:21:58;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet e8:4e:06:7a:b1:e2;
  set vendor-class-identifier = "android-dhcp-10";
  client-hostname "Atari5200";
}
lease 10.179.11.70 {
  starts 5 2021/04/16 02:44:01;
  ends 5 2021/04/16 04:44:01;
  tstp 5 2021/04/16 04:44:01;
  cltt 5 2021/04/16 02:44:01;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet 34:af:b3:c5:f2:d2;
  set vendor-class-identifier = "android-dhcp-9";
  client-hostname "Commodore64-10GBase-LR";
}

Without doing anything else (no confounding dhcp service restarts etc.) the dhcp leases page again loads in under 1 second.

I have tested other variations of this test involving an increased number of rogue unnamed entries. More rogue entries appears to cause longer page load times. In every case, opening the dhcp.leases file in vi, and assigning the rogue entries a name appears to eliminate the page load delays all of the time.

If others can reproduce this, I wills file a bug report.

Gertjan

Great write-up.
This is what is needed to find the issue.

I just looked am my /var/dhcpd/var/db/dhcpd.leases file.
About 72 Kbytes - 196 leases, and (only) 19 leases have a "client-hostname".
177 lease entries have no client-hostname at all.
Still : I never had any issues looking at Status > DHCP leases.

So, is it ok to think that the presence of absence of a "client-hostname" line in a lease is not the issue ?

This file : /usr/local/www/status_dhcp_leases.php

When you place a // in front line 91, like this :

// $leases = system_get_dhcpleases();

( do the same thing for the next line 95 - just to be sure )

Status >> DHCP should show an empty page like this :

If still nothing shows up, the issue is not "DHCP" related at all.

chance

I've not reported this, I've actually abandoned ship after 2.5.0 gave me fits with IPSec and GRE tunnels.

I'm glad you guys are making progress though.

I would recommend going to the spot where system_get_dhcpleases() is and find the DNS lookup lines. Add an IF the hostname is null (in php obviously) then skip it.

That should solve the issue.

I checked in the source on github and this is the file.

https://github.com/pfsense/pfsense/blob/0f03681f9b16583ac0f6a6b98272ab9b2d11d79e/src/etc/inc/system.inc#L675

It's already checking for empty, so I assume it's getting a false positive. (or maybe it's not the current version.)

I think it should be something like this:

if (preg_match('/}$/', $line)) {
			if ($lease) {
				if (empty($item['hostname'])) {
					$hostname = gethostbyaddr($item['ip']);
					if (!empty($hostname) &$ empty($hostname) != "") {
						$item['hostname'] = $hostname;
					}
				}
				$leases['lease'][] = $item;
				$lease = false;
				$dedup_lease = true;
			} else if ($failover) {
				$leases['failover'][] = $item;
				$failover = false;
				$dedup_failover = true;
			}
			continue;
		}

Relevant line I added (&& $hostname != "")
if (!empty($hostname) && $hostname != "") {

~~No idea if it will work or not.~~

Nevermind, just comment out the gethostbyaddr line and live without it I guess...

karlfife

@chance
I believe this is covered in this bug report:
https://redmine.pfsense.org/issues/11512

Parallel discussion can be found here:
https://forum.netgate.com/topic/161121/after-upgrade-to-2-5-status_dhcp_leases-php-nor-diag_arp-php-will-load/13

zarteg

Turns out I had the same issue.
Kill your DNS Resolver and try again.

Gertjan

@chance said in DHCP lease screen not loading:

Nevermind, just comment out the gethostbyaddr line and live without it I guess...

The definition of gethostbyaddr is https://www.php.net/manual/fr/function.gethostbyaddr.php

Look somewhat down n on that page, you find :

The problem of broken DNS servers.....

So is this a 'broken DNS issue' after all ?

drumnbisco

Can confirm, had my primary dns set to 9.9.9.9 for dnssec, switched it to googles and clicked save, dhcp leases status page loaded fine. Switched it back and pages loads fine a 2nd time.

Gertjan

@drumnbisco said in DHCP lease screen not loading:

had my primary dns set to 9.9.9.9

That means you're forwarding.
When forwarding, DNSSEC is meaningless.
Stop forwarding, and "dhcp leases status page" loads fine right out of the box.

drumnbisco

@gertjan
I never had an issue loading the DHCP leases page on the previous version. This only happened after I upgraded.

I followed this guide https://linuxincluded.com/configuring-quad9-on-pfsense/ which recommended having it checked.

Gertjan

@drumnbisco said in DHCP lease screen not loading:

This only happened after I upgraded.

Most probably, the upgrade just exposed an issue : pfSense itself has no DNS any more.
This situation is often experiences by users that have no LAN issues (DNS issues for LAN clients) but wonder why pfSense doesn't propose updates any more.

No working DNS for pfSense itself also means that the DHCP status generation page doesn't work any more, as every DNS request fails after 'many seconds'. If you have many DHCP leases, the page construction will takes minutes, or far more : the page takes forever to get generated, the browser or web server will finally bail out.

Yet another good reason NOT to forward or NOT to change any DNS settings.
Because the ones with default DNS settings do never experience " DHCP status page' " issues.
And if you forward : ok, but be ready to do some testing every time there is an issue. Even if the issue doesn't seem to be DNS related.
Like : can I load and update the list with proposed pfSense packages ?
Is this

actual ?

check system logs.
check DNS logs.

I've just checked this myself :
I 'killed' unbound by pressing :

Then I visited the Status DHCP Leases page.
It wouldn't load any more - that is : it loads partially. The "wheel" kept on spinning turning for minutes .....
When I started unbound, the page loaded instantly.

plague311

@gertjan said in DHCP lease screen not loading:

@drumnbisco said in DHCP lease screen not loading:

This only happened after I upgraded.

Most probably, the upgrade just exposed an issue : pfSense itself has no DNS any more.
This situation is often experiences by users that have no LAN issues (DNS issues for LAN clients) but wonder why pfSense doesn't propose updates any more.

No working DNS for pfSense itself also means that the DHCP status generation page doesn't work any more, as every DNS request fails after 'many seconds'. If you have many DHCP leases, the page construction will takes minutes, or far more : the page takes forever to get generated, the browser or web server will finally bail out.

Yet another good reason NOT to forward or NOT to change any DNS settings.
Because the ones with default DNS settings do never experience " DHCP status page' " issues.
And if you forward : ok, but be ready to do some testing every time there is an issue. Even if the issue doesn't seem to be DNS related.
Like : can I load and update the list with proposed pfSense packages ?
Is this

actual ?

check system logs.
check DNS logs.

I've just checked this myself :
I 'killed' unbound by pressing :

Then I visited the Status DHCP Leases page.
It wouldn't load any more - that is : it loads partially. The "wheel" kept on spinning turning for minutes .....
When I started unbound, the page loaded instantly.

I'm not really sure what most of this means, mind you I'm a network\systems administrator that has a degree in this, but this doesn't make a lot of sense.

In my situation, which I believe is much like drumnbisco's issue, there was never an issue with DNS. I was able to get the proposed upgrade button, that's how I upgraded to 2.5 in the first place. The button continued to work as that's how I got upgraded to 2.5.1 while having issues with the DHCP screen. The two weren't connected at all. My pfsense handles DHCP (still) but hands out a DNS to the PiHole, which then handles all of the requests. That configuration is what I've used at my house for over a year or 2, since I installed the PiHole.

I'm not sure this is related to the issue anyone here is seeing.

Gertjan

@plague311
Ah, didn't know about the PiHole ...

I rephrase my last (above) post :
Stop the DNS - unbound on pfSense, or your PiHole, and you see that the the DHCP lease page doesn't show up any more : it won't come to completion.
Because it can't 'resolve' host names any more. And that would explain the issue.

You can check easily if 'DNS' is the explication of your issue : "DHCP Lease page not loading " :
Disable line https://github.com/pfsense/pfsense/blob/f528b6a9cba18d7f299fdeed8c84f22abb16fcf1/src/etc/inc/system.inc#L742 by putting a
//
in front of the line.

Or make it look like

/* $hostname = gethostbyaddr($item['ip']); */

if the issue is gone, you know that the issue is that pfSense has no access to a 'DNS'.

plague311

@gertjan My issue is already resolved with whatever netgate did with 2.5.1. My DHCP leases screen works without issue now. I didn't have to change anything. Just updated and it started working again.

karlfife

@plague311
Incorrect. v2.5.1 did not resolve the problem.
The upgrade temporarily masks the problem.

plague311

@karlfife Oh that might be. It's been slow, but worked the few times I've needed to use it. Admittedly I use pfsense on a VM at my home. It doesn't get a lot of attention since I built it. One of those set it and forget it type of things. Every few weeks I check for updates, that's about it.

maraun

I found the error :-)
The timeout comes by /etc/inc/system.inc
line 735 $hostname = gethostbyaddr($item['ip']);

bobby_hill1983

I updated to 5.1 the other day and bam my DHCP Leases page stopped working.

I have a pihole and wanted to try some new config options to make it the logging better so I turned on Conditional Forwarding in the pihole for hostname resolution.

I also took the pihole's IP out of "System > General Setup > DNS Servers field."

Once I put the IP back in to the DNS Servers field under General Setup the DHCP page started working again instantly. Up until this point which coincides with my upgrade of pfsense I never had this issue and I always had the IP of my DNS server (pihole) in that field. EDIT: I also turned off Conditional Forwarding on the pihole before this so thought it was this but it was actually CF. Removing the DNS from General Setup did not cause it since it was set under DHCP Server settings as well.

I also had the IP of the pihole in "Services > DHCP Server > LAN" under Servers > DNS Servers. So in conclusion for my setup that existed this way for many years, removing the DNS IP from General Setup borked my DHCP page. Don't know why and I did that in an attempt to make the logging more accurate on the pihole.

EDIT: Use Conditional Forwarding is the issue actually on the pihole settings. I was tinkering and ultimately it was only this causing it. Some sort of loopback thing?
Leaving this message for future self-researchers. I am stupid to not just use pfBlockerNG but I love my little pi with LCD screen and PADD.

easpem

Same problem here after upgrading to 2.5.0 and keeps happening after upgrading to 2.5.1
Deleting dchp.leases works for a short time, then fails to load again.

Same goes when opening /diag_arp.php.
Our setup is quite standard, just openvpn-client-export package.