DHCP - No Free Leases (pf_2.4.3-release-p1)
-
That's what I thought I knew about dhcpd...:o)
Don't know why it's being different on pfsense...As you can see from my last post, all leases are used up and new clients can't get a lease...even when there are just a 300+ users at this time, from the 8000+ leases.
Aug 13 07:17:46 dhcpd DHCPDISCOVER from 6c:27:79:xx:xx:xx via em11: network 10.0.192.0/19: no free leases Aug 13 07:17:46 dhcpd DHCPDISCOVER from 94:0e:6b:xx:xx:xx via em11: network 10.0.192.0/19: no free leases Aug 13 07:17:45 dhcpd DHCPDISCOVER from 58:e2:8f:xx:xx:xx via em11: network 10.0.192.0/19: no free leases Aug 13 07:17:43 dhcpd DHCPDISCOVER from 00:22:57:xx:xx:xx via em11: network 10.0.192.0/19: no free leases Aug 13 07:17:43 dhcpd DHCPACK on 10.0.219.170 to 3c:fa:43:xx:xx:xx (android-2800483908312c30) via em11
I can't find these clients on the leases files, so they are new ones...
So why don't they get an expired lease?
There are like 7000+ expired leases...About the dhcpd process...that's a weird one...seems to be running from just today...
dhcpd 35140 0.0 0.1 20748 9724 - Ss 07:23 0:00.38 /usr/local/sbin/ 35140 dhcpd 07:23 10:37
-rw-r--r-- 1 dhcpd _dhcp 2.2M Aug 13 07:56 dhcpd.leases
But all pfsense services seem to be doing the same...
Last reboot was on 23 Jul'18.What kind of file system problems?
I could I check those?Thx!
-
This image shows the number of leases that are active and thus not in the expired state :
Next image : this shows merely all possible leases - the size of the pool : 8157 - expired, active; whatever - all of it. It's just a dump of the dhcpd leases file (I guess).
When showing all leases...
So the idea that dhcpd on pfsense it's not releasing the IPs as it should...it's correct!
There is not such thing as "dhpcd releasing the leases". They expire - or are renewed before the end of the leasey by the device that uses a lease.
When no renew happens, after the lease time, they just expire and are re distributable to any device asking (new) for one.
When the lease isn't expired, a lease can only be renew by the device "owning" the IP at that moment.For me it clear that, when you took the screen shots, 340 leases are open - out of the 8157 available.
Example :
My pool is 243 IP's : (250-8).
Still, my dhcpd is serving IP's - because most of these IP's (leases) are in the expired state, so given to any one who needs one.
Could you describe your dhcp server settings - what did you take out the the default state ?
Some smart switches or other devices down stream that could manipule the DHCP requets ?
Just an idea : check out the DHCP log : a device is hammering with changing MAC address thus depleting the pool rapidly ?
Whatever ...Remember : your DHCP server and mine are exactly the same. The only thing that is different are the settings.
edit Read : https://serverfault.com/questions/151224/dhcpd-wont-let-go-of-old-leases - an example how a roque client can eat all leases rapidly - and a method how to find that client.
Worse : "Client ignore packet or never receive it" : another DHCPDISCOVER will follow, with another DHCPOFFER with another IP .... -
Here are the settings:
option domain-name "gestao"; option ldap-server code 95 = text; option domain-search-list code 119 = text; option arch code 93 = unsigned integer 16; # RFC4578 default-lease-time 7200; max-lease-time 86400; log-facility local7; one-lease-per-client true; deny duplicates; ping-check true; update-conflict-detection false; authoritative; subnet 10.0.192.0 netmask 255.255.224.0 { pool { option domain-name-servers 10.0.223.250; ignore bootp; range 10.0.192.20 10.0.223.240; } option routers 10.0.223.250; option domain-name-servers 10.0.223.250; default-lease-time 3600; max-lease-time 3601; option ntp-servers pool.ntp.org; }
That's the thing...I don't think they are being marked as expired...
Maybe because of this on the .conf file...?default-lease-time 7200; max-lease-time 86400;
I say this because I noticed that most leases have 24h expiration time and that's is not what I configured...
I tried to manually alter this but it always reverts to the same...Help?! :)
Thx!
-
I tried to change file by shell and kill/start process, after deleting all leases and no luck...
To be "Expired" leases are marked with "24h"...even if the max-lease-time and default-lease-time is set to 3601 and 3600s.
If I restart process via GUI, file is always overwritten...Where is it getting the configs from?!
Even if I remove file permissions, it chowns and chmods the file again with the right ones...
-
If you want to edit something in a conf you have to edit the services.inc file since yes this is what creates conf on service start, etc.
https://github.com/pfsense/pfsense/blob/master/src/etc/inc/services.inc
$dhcpdconf = <<<EOD option domain-name "{$syscfg['domain']}"; option ldap-server code 95 = text; option domain-search-list code 119 = text; option arch code 93 = unsigned integer 16; # RFC4578 {$custoptions} default-lease-time 7200; max-lease-time 86400; log-facility local7; one-lease-per-client true; deny duplicates; ping-check true; update-conflict-detection false; EOD;
-
@ihugof said in DHCP - No Free Leases (pf_2.4.3-release-p1):
default-lease-time 7200;
max-lease-time 86400;default-lease-time 7200; max-lease-time 86400;
are global time out values.
Every pool has its own, pool-specific settings, like in your case :
default-lease-time 3600; max-lease-time 3601;
-
You're the best! :)
It's seems to be that!
I've changed the services.inc and now the expiration time of the leases make more sense!
It still went and got some for tomorrow, that I don't know how, since I deleted all the leases...but lets wait and see of this does the trick!
Many Thx! :)
-
I know, but for some weird reason, it is considering the global values...not the pool ones...
After editing the services.inc like @johnpoz said, now the leases will expire, as they should, in 3601s.
Before it was assuming the 24h of the global value, and for the number of users I have, it just doesn't do, since I've more than 8190 different users, in a 24h period.How come no one had this problem before?
Or maybe no one noticed it...:)I'll let this run for at least 24h, to see if the problem is solved.
Thx!
-
@ihugof said in DHCP - No Free Leases (pf_2.4.3-release-p1):
How come no one had this problem before?
Good question. Dono what the problem is.
My LAN :
Setting :
Or : default lease time one day, max lease time 4 days.
This is a device on LAN, an iPhone, came in this morning :
It has a lease of 4 days => ok.
On my captive portal - another LAN, another DHCP server - another pool:
default lease time = 6 hours - max lease time 12 hours.
Hummm. 12 hours, so ok.
dhcpd on my two interfaces respects de duration as I set it.
-
Maybe it's the number of different clients I have...
But it started happening again...
Some leases are being kept for 24h...:-/Most of the leases are like this...and they will not be reused until they expire, 24h later...:-/
Thx!
-
@johnpoz said in DHCP - No Free Leases (pf_2.4.3-release-p1):
etc/inc/services.inc
You checked the dhpcd conf file after edeting the source, and restarting dhpcd ?
default-lease-time xx;
max-lease-time yy;did change ? For both the defaults and pool's version ?
If you have a DHCP request coming in every 10 seconds with a lease duration of 24 hours : what about making a bigger pool , like 10k entries ?
But .... I guess we all agree, some how I can't find out what is the real issue here.
Can you wireshark, and test if some devices "hailstorm" the dhpcd with requests (DHCPDISCOVER, as mentioned above) ?
Google informed me that a theory I have really exists : https://www.information-security.fr/fonctionnement-et-protection-contre-les-attaques-dhcp-starvationrogue/ (not the right language .... but you will get the picture ) -
Yes, it did indeed change the lease times on config file, although it's inserting the configurations outside the pool {}, which is normally where they as supposed to be.
It's inserting them in the subnet {} but that seems to be fine for most clients.Really can't see where the problem is coming from, since if I start using the dhcpd of another linux box or cisco router, there's no problem...hence thinking the problem is somewhere how pfsense / freebsd configured the dhcpd behind the scenes.
I can't have a bigger pool, since this subnet it's already defined and would clash with other subnets in use.
I checked today and I've like 6000+ abandoned leases and from what I know from dhcpd, these are not used until all the expired ones are used.
The abandoned leases are the ones being marked with a 24h expiration. -
Still no luck, but figured that the problem is related to the abandoned leases and they all come from a specific wireless controller (3Com)...Yes, it's OLD! :)
But it all works well with Cisco router DHCP...so, dhcpd shouldn't have any issues, but something it's not quite the same... -
It's a bit of a radical idea... but have you considered splitting wired and wireless users into different subnets (or different DHCP scopes)? Divide & conquer to find the problem, is my motto.
Either way... I agree that the lease abandon rate is pretty high... This feels like an (OSI) layer 2 problem somewhere in the network, but I don't know enough to say what it could be.
-
@msf2000 It has all been divided...:) This is just 1 subnet, were I'm testing pfSense, and it's just for wireless guest clients.
For all the other subnets, we've other dhcp servers and no issues, hence I'm being puzzled with dhcp on pfSense acting like this...
I've never seen this dhcp behavior on any other dhcp servers...:-/
-
So the controller is doing a dhcp relay? You mention that all the leases are from the controller?
Where are you getting that listing of leases and times? The dhcp lease table should show host and mac, etc.
-
@johnpoz No, no need for dhcp relay, since the controller wireless network interface is on the same network/vlan of the interface of pfSense and yes, all the abandoned leases come from that specific wireless controller - 3Com.
I've other 2 Cisco controllers and no issues with those ones.But you may be on to something, since I've noticed that all the clients leases that reach pfSense come with the controller name, and not the PC/Device name, even with all the mac addresses being different and such, but the client ID it's always the same = controller name/model.
I got the list from the pfSense DHCP GUI and from the dhcp logs, via CLI.
-
That is odd.. I have no experience with 3com controllers.. But why would pfsense not see the host-id from the client itself? Unless the controller is doing something with the dhcpdiscover/request?
-
@ihugof said in DHCP - No Free Leases (pf_2.4.3-release-p1):
er subnets, we've other dhcp servers and no issues, hence I'm being puzzled with dhcp on pfSense acting like this...
I've never seen this dhcp behavior on any other dhcp servIs the 3COM wireless controller configured as a DHCP relay, or is it configured to just pass layer 2 traffic from pfSense to the clients?
-
^ yeah sounds like it my be relaying it??