DHCP - No Free Leases (pf_2.4.3-release-p1)

iHugoF

So it's strange...Even with just 1500+ leases it started complaining about no free leases...
I also tried to change range, stop/start and nothing, always the same problem.

Really out of ideas on how to solve this...besides relying on an external dhcp server...:-/

Derelict

I guess you should learn the format of the dhcp leases file and take a look there to see what you can see.

https://www.freebsd.org/cgi/man.cgi?query=dhcpd.leases&sektion=5&manpath=freebsd+ports

That file is located at /var/dhcpd/var/db/dhcpd.leases in pfSense.

iHugoF

@derelict

I've looked into the files.
It's pretty standard stuff and I've not spotted any problems.

Here's a example of a lease:

lease 10.0.210.31 {
  starts 3 2018/08/01 12:28:38;
  ends 3 2018/08/01 12:43:38;
  tstp 3 2018/08/01 12:43:38;
  cltt 3 2018/08/01 12:28:38;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet 88:d7:f6:xx:xx:xx;
  set vendor-class-identifier = "android-dhcp-7.0";
  client-hostname "android-xxxxxxxxxxxxxxxx";

The pool permits 8000+ IPs, but after 1500+ it starts giving "No Free Leases" error messages.

Any ideas?

Thank you!

iHugoF

Update:

I've deleted all the leases from dhcpd.leases and deleted the dhcpd.leases~ file and activated the dhcp server on pfsense again and until now, no problems...:-)

2039+ leases and going, without any issues...

Will let this run for a couple of days to see if the problem is resolved...
The 1st time, the system was already running for at least 1 week, before ppl started complaining.

iHugoF

Same problem...:(

Just 1248 leases in use this morning, but already the message "network 10.0.192.0/19: no free leases"...

It must be some pfsense dhcpd issue...since I never had this problem with other dhcpds...

Anyone any ideas?

Thank you!

Derelict

It's ISC dhcpd just like everyone else runs.

You have enough RAM?

iHugoF

@derelict

That's the thing...it's pretty standard stuff...or it should be!
Never had this problem with any other dhcpd in the past...

It's something on pfsense...what...I don't know...
If I switch to another dhcpd, on another *nix box...no problems, cisco router...no problems, windows server...no problems...
Just pfsense having issues...

RAM is OK...
Memory usage
62% of 16325 MiB

iHugoF

I found this on another forum...Makes any sense?

This and similar errors RECENTLY started happening to me too. It appears that dhcpd made a significant change in how it handles ranges assigned to a subnet. It appears that if you have any hosts with a fixed-address that exists in your range, the entire range is ignored for anything other than your host entries! This IMHO is a step backwards, but here is my workaround (declare every address in your range as a range itself)

From what I understand...if someone has a fixed address, the dhcpd goes crazy and ignores the range?! Is this correct?!

Thank you!

kpa

That's how it has been as long as I can remember, the dynamic range is for truly dynamic addresses and static mappings have to outside any dynamic range.

johnpoz

pfsense gui will not let you put in a reservation for an IP that is inside your pool.

Other than windows dhcp, which allows for reservations inside a pool.. Yeah your reservations are outside your pool range.. Is that what causing you problems?

Gertjan

What I make of it :

@ihugof said in DHCP - No Free Leases (pf_2.4.3-release-p1):

It appears that if you have any hosts with a fixed-address that exists in your range,

So : "If there is some device that has a fixed IP - and this IP is within a pool of the serving DHCP, dhcpd goes on strike".
It can be any client with a user (who doesn't like DHCP) and he assigned to his device a static IP, which brings down the whole cirque ?

If this IP can be arpped down, that you should try to firewall it out, and restart dhcpd ...

iHugoF

@kpa

The thing is...anyone can decide to try to fix their IP...and that seems to "crash" the dhcpd...

iHugoF

@johnpoz

It seems that anyone that try to fix their own IP...can cause problems...Which is kind of dumb...dhcpd side anyway...

johnpoz

I have never seen that... I can try and duplicate here.

Just so everyone is clear your pool is say .100 to .200

Then some client sets itself static at .150 and your saying dhcpd stops handing out any IP/Leases in that pool.. Or fails to hand out to a client asking for .150?

That would be stupid and for sure an issue, because your saying any single client on a network could cause what amounts to a dos attack just by setting a static IP to some other IP in the range it got from the dhcpd at first.

iHugoF

@gertjan

Yes...it seems to be that...which is dumb, but it seems that if anyone fixes one IP, the dhcpd considers that subnet as "full"...
That would explain why some ppl still were able to get an IP and others didn't...

Will need to test this someway...

iHugoF

@johnpoz

I never had this issue on other *nix boxes or any other dhcp servers...

I still don't know if this is the issue, but it kind of makes sense...since it's the only explanation until now...:)

What I understand from some other post was:

I have for example a /19 network that goes from 10.0.192.0 to 10.0.223.255
If someones fixes the IP 10.0.194.10, for example...then all leases from that IP forward, will remain as "in use";
Only IPs from 10.0.192.0 to 10.0.194.9 will be available for lease.

If it's not this...then I've no clue why a scope with 8000+ IPs, gets full after 1200+ leased IPs...

Thx!

johnpoz

And you tracked down this 10.0.194.10 IP and it was set static?

In all my years have not seen this.. What I have seen is duplicate IP issues where someone set IP in pool to static, then for whatever reason the IP detection mech fails.. client with dupe off for example during the leasing ie client/server send out out arp/ping looking to see if IP is in use before requested by client or handed out by dhcpd.

This should be easy enough to try and duplicate.. And I agree this could be a issue for sure since it would mean that any client could even without even trying on purpose create a dos.

https://www.isc.org/wp-content/uploads/2018/02/dhcp44.html#IP%20ADDRESS%20CONFLICT%20PREVENTION
IP ADDRESS CONFLICT PREVENTION

The DHCP server checks IP addresses to see if they are in use before allocating them to clients. It does this by sending an ICMP Echo request message to the IP address being allocated. If no ICMP Echo reply is received within a second, the address is assumed to be free. This is only done for leases that have been specified in range statements, and only when the lease is thought by the DHCP server to be free - i.e., the DHCP server or its failover peer has not listed the lease as in use.

If a response is received to an ICMP Echo request, the DHCP server assumes that there is a configuration error - the IP address is in use by some host on the network that is not a DHCP client. It marks the address as abandoned, and will not assign it to clients. The lease will remain abandoned for a minimum of abandon-lease-time seconds.

If a DHCP client tries to get an IP address, but none are available, but there are abandoned IP addresses, then the DHCP server will attempt to reclaim an abandoned IP address. It marks one IP address as free, and then does the same ICMP Echo request check described previously. If there is no answer to the ICMP Echo request, the address is assigned to the client.

The DHCP server does not cycle through abandoned IP addresses if the first IP address it tries to reclaim is free. Rather, when the next DHCPDISCOVER comes in from the client, it will attempt a new allocation using the same method described here, and will typically try a new IP address.

Gertjan

I had some time this night, and did some tests.

My dhcpd pool range is 192.168.2.10 -> 192.168.2.254. Gateway and DNS == pfSense = 192.168.2.1 / 24
It concerns an interface running the captive portal.

I used 2 windows PC's, and gave them static IP's right into the pool (and DNS / Gateway = 192.168.2.1 ). I checked if the IP that was assigning was used ones but expired some time ago.
I didn't notice any suspected behavior, and could use the portal as any other device.
Further DHCP clients could obtain a lease, the portal kept working for them.
No unusual lines in the dhcp log.

Btw : Status => DHCP Leases and click on "Show all configured leases" at the bottom of the screen.
For me, this will show all leases, expired, or not. There are actually 254-10=244 IP listed, they all have been used ones in the past, but are recycled by dhcpd when needed.

Leases are expired after 12 hours - captive portal hard time out is 6 hours.

0_1533878297656_05ade006-8c87-4508-b012-0c6e5c48950a-image.png

Works for .... nearly ten years now.

iHugoF

@johnpoz

No, it was just an example from what I got on some other post...
That was what someone said it tried and what he found to be the truth...:)

I'll try to do some tests, but last time it took more than 24h for the problem to resurface...

Thx!

iHugoF

I've enabled the dhcp server on pfsense again, at 07:23am (UTC+1)...let's see how long it takes to start getting the "no free leases" message...

On Status/DHCP Leases I've 1718 leases in use.
If I press "Show all configures leases" button it shows 3489 in use.

Most of these leases shown are "offline" and "expired"...since yesterday...
Will dhcpd use these "expired" ones when it needs a free lease?

It looks to me that it's keeping them and not releasing them, even when they expired the day before...

It's 09:42am (UTC+1) and still no problems...:-)