DHCP client unable to get lease from cable provider [solved]

itpp21

I have something similar and tried this solution but that didn't work, it may however have the same root cause which is why I am continuing in this thread.
Every time the WAN dhcp expires something like this happens:

Sat Feb 27 03:15:50 2021;192.168.x.1; <30>Feb 27 03:15:50 dhclient[3244]: DHCPREQUEST on igb0 to 10.160.x.x port 67
Several hours later and log lines with the same attempt every 5 minutes........
Sat Feb 27 10:45:19 2021;192.168.x.1; <30>Feb 27 10:45:19 dhclient[3244]: DHCPREQUEST on igb0 to 255.255.255.255 port 67
Sat Feb 27 10:45:19 2021;192.168.x.1; <30>Feb 27 10:45:19 dhclient[3244]: DHCPACK from 212.142.x.x

Note that 10.160.x.x should not be alive for WAN but it is and out of my control, however after several hours of trying 10.160.x.x pfsense finally decides to do a 255.255.255.255 and as you see we get an immediate reply and are done until the next renewal.

Question: can I force a 255.255.255.255 broadcast faster and/or more frequent than every 7-12 hours ?

itpp21

I hadn't noticed the advanced toggle, but lets see if this solves this issue, value 3600,

itpp21

As many have experienced, nothing really works because the client and the service work differently.
In this case the WAN dhcp server who eventually responded hands out the wrong server for a renew request where the dhclient as a service can take hours before giving up (often 24 hours on a 72 hour lease) which can lead to a new ip address while otherwise you could have a 'stable' wan address for months.

After digging around I came up with the following solution which ain't pretty but it does the job.

A script:

#!/bin/sh
# /usr/local/libexec/force-dhcp-update

pkill -9 -u `id -u _dhcp`
rm -f /var/db/dhclient.leases.igb0
dhclient "igb0"

And in '/etc/crontab' add a line:

31	6	*	*	*	root	/usr/bin/nice -n20 /usr/local/libexec/force-dhcp-update

Gertjan

This file :

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

rm -f /var/db/dhclient.leases.igb0

is the 'memory' of the DHCP client process.
It keeps track of the IP obtained during the initial lease, and the duration of the lease etc.
Have a look at it : useful information in there !

Half way the duration of the initial lease, the DHCP client process sends a DHCP-RENEW. It tries to extend the lease, using the same duration, using the same IP. (as it is stored in /var/db/dhclient.leases.xxx where xxx is the 'driver' name of the interface).
Deleting this file is deleting one more chance that the DHCP server of the ISP attributes another IP, as no RENEW (with known IP) is launched, but a DHCP-DISCOVER (etc).
It's up to the DHCP server of the ISP to grant the same IP, hand over another IP. Its the server that decides, never the client.

If the DHCP server doesn't "understand" the RENEW, then that is most probably a problem with their server. Having an issue like that might put them out of business very fast .... (I guess).

When all goes well, as this

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

the WAN dhcp server who eventually responded hands out the wrong server for a renew request where the dhclient as a service can take hours before giving up (often 24 hours on a 72 hour lease) which can lead to a new ip address while otherwise you could have a 'stable' wan address for months.

Not very clear.
A DHCP server that hands out "the wrong server" ?
It hands out an IP, mask (network) - a gateway for sure, probably a DNS IP or two, maybe a NTP, etc. But not a "server".

Also : DHCP server replies - the entire transaction : 4 messages over UDP are being send in far less then a second. If not : a cable/wire issue ??

The DHCP client keeps on trying to renew the existing lease, as it has to "pull the plug" when the lease is over. This will disrupt all the traffic.

Just my two cents : maybe is this the method of your ISP to have you using another IP WAN every xx hours ?

itpp21

@gertjan said in DHCP client unable to get lease from cable provider [solved]:

rm -f /var/db/dhclient.leases.igb0

is the 'memory' of the DHCP client process.
It keeps track of the IP obtained during the initial lease, and the duration of the lease etc.

Yes I know but it contains wrong information which gets re-used at reboot causing the same problems.

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

the WAN dhcp server who eventually responded hands out the wrong server for a renew request where the dhclient as a service can take hours before giving up (often 24 hours on a 72 hour lease) which can lead to a new ip address while otherwise you could have a 'stable' wan address for months.

Not very clear.
A DHCP server that hands out "the wrong server" ?

/var/db/dhclient.leases.igb0:

lease {
interface "igb0";
fixed-address 62.194..;
next-server 212.142.63.;
option subnet-mask 255.255.255.0;
option time-offset 3600;
option routers 62.194..*;
option domain-name-servers 62.179.104.196,213.46.228.196;
option host-name "pfSense";
option domain-name "arnhem.chello.nl";
option interface-mtu 1500;
option broadcast-address 255.255.255.255;
option dhcp-lease-time 259200;
option dhcp-message-type 5;
option dhcp-server-identifier 10.160.92.1; This is not a dhcp server! but pfSense keeps trying this one for hours
option tftp-server-name "212.142.63.132";
renew 4 2021/3/11 15:22:02;
rebind 5 2021/3/12 18:22:02;
expire 6 2021/3/13 03:22:02;
}

Gertjan

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

option dhcp-server-identifier 10.160.92.1; This is not a dhcp server! but pfSense keeps trying this one for hours

I agree with you : that's suspect.
Its announcing a RFC 1918 "Private Address Space" = 10.0.0.0/8.
So, I guess, when it's time to renew, what does a DHCP client do : broadcast or address directly its RENEW request to "10.160.92.1", and that address doesn't answer because non-existent ?
Time to wireshark to find out.

In zo'n geval is jou oplossing .... prima, lijkt me.
Bel chello.nl op - probeer voorbij die aardige mevrouw van level 1 ** support te komen en vertel level -3 support : "Uw DHCP server denkt dat ie "10.160.92.1 is - gaat verder alles goed ?"
** of meneer.

Waarop

Wat ?? U gebruikt niet één onze eigen routers maar een volledig onbekende "pfsense" wwar wij nog nooit van gehoord hebben. En tevens gebruikt deze pfsense werelds meest bekende ISC DHCP client - die wij ook niet supporten .....

Etc etc. Ik ga voor je duimen.

Been there. Seen that.

(i'm thinking out loud here ... don't know how things 'work' in the Netherlands these days, it has been decades that I left the country. )

And that DHCP-OFFER comes from your ISP ?? Not your modem (who tend to have also an on board DHCP server so you can connect to it to set modem paramters (while its not connected with its WAN = ISP)) by an=y chance ?

Btw : my ISP's router also gives my pfSense (pfsense dhcp client) this dhcp-server-identifier option :

lease {
  interface "em0";
  fixed-address 192.168.10.3;
  option subnet-mask 255.255.255.0;
  option routers 192.168.10.1;
  option domain-name-servers 192.168.10.1;
  option domain-name "home";
  option broadcast-address 192.168.10.255;
  option dhcp-lease-time 86400;
  option dhcp-message-type 5;
  option dhcp-server-identifier 192.168.10.1;  <=============
  renew 5 2021/3/5 09:31:00;
  rebind 5 2021/3/5 18:31:00;
  expire 5 2021/3/5 21:31:00;
}

The "192.168.10.1" is the LAN IP of my ISP router = the DHCP server for my pfSense.

itpp21

@gertjan said in DHCP client unable to get lease from cable provider [solved]:

option dhcp-server-identifier 10.160.92.1; This is not a dhcp server! but pfSense keeps trying this one for hours

I agree with you : that' suspect.
Its announcing a RFC 1918 "Private Address Space" = 10.0.0.0/8.
So, I guess, when it's time to renew, what does a DHCP client do : broadcast or address directly is RENEW request to "10.160.92.1", and that address doesn't answer ..... = nobody there.

Yep exactly that and it keeps doing that for hours and hours and hours and... then it times out (the dhclient) and then it does a broadcast to which a proper dhcp server replies, I get my WAN ip and the lease file gets filled with one rfc 1918 address and after 24 hours the whole circus starts again (which is why I remove the lease file as well as its useless at boottime).

Time to wireshark I guess to find out.

In zo'n geval is jou oplossing .... prima, lijkt me.
Bel chello.nl op - probeer vorrbij die aardige mevrowu van level 1 support te komen en vertel level -3 support : "Uw DHCP denkt dat ie "10.160.92.1 - gaat verder alles goed ?"

I have tried and give them the common lie's like "I have restarted the modem 20x" and "I have reinstalled windows 20x" to get past those first and second line idiots but as expected no one knows anything.

And that DHCP-OFFER comes from your ISP ?? Not your modem (who tend to have also an on board DHCP server so you can connect to it to set modem paramters (while its not connected with its WAN = ISP)) by an=y chance ?

Nope 100% sure this comes from my ISP.

Btw : my ISP's router also gives my pfSense (pfsense dhcp client) this dhcp-server-identifier option :

lease {
interface "em0";
fixed-address 192.168.10.3;
option subnet-mask 255.255.255.0;
option routers 192.168.10.1;
option domain-name-servers 192.168.10.1;

Your in routed mode, I'm in bridge mode.

Gertjan

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

Your in routed mode, I'm in bridge mode.

Yeah.
For the sake of simplicity, I do presume the modem is 100 % transparant here (they often aren't that transpararnt, which brings in a boatload of issues) - it just translates what comes out of the WAN port of pfSense over some 'laser / radio / satellite / ADSL / POTS / whatever carrier back, after a modem at the other side - to another RJ45 plug, into the DHCP server. Your "WAN" is also some "local network" with a DHCP server, a gateway etc.

I've been looking at the man pages of the DHCP client, to see if you could 'reject' that server-identifier option.
have a look here : https://www.freebsd.org/cgi/man.cgi?query=dhclient.conf&sektion=5&n=1
You could try to supersede, append, default or prepend.

You could craft your own dhcp client conf file, and use the

option.

If not, as said : restart it.

itpp21

I've looked into all those options and the private settings file, the problem is as follows;

dhclient starts a client and honors all the options you set,
dhclient gets dhcp stuff,
dhclient demotes to a service,
:label
dhclient expires dhcp lease,
dhclient forgets all options and only honors the lease file,
if this lease file has a problem your f**ked for however long it takes for dhclient(as a service) to timeout and do a broadcast.
goto label

Atm. I don't care much if this a bug in pfSense or a shortcoming in dhclient(as a service) but I have tried ALL the menu options and even editing the .inc file where hardcoded values can be set and nothing solves this behavior (regardless if the ISP is at fault).

I am running my script now for 3 days and so far so good, direct WAN assignment or renew and no more thousands of log entries attempting to get dhcp info from a none-dhcp server and also running the risk on a new WAN address every 3 days because dhclient is stuck in a loop (see above goto label).

melslenstra

@itpp21 interesting to see that my thread is resurrected after 2 years, with replies from - by the looks of it, stand to be corrected - more Dutch(-speaking) folk

I'm not 100% clear if the situation you're having is exactly the same as what I encountered. The problem I had was that dhclient is/was basically hardcoded to NEVER set the Broadcast flag in the BOOTP options field, while that was actually the only type my broken ISP's DHCP setup was able to successfully respond to...

Are you actually seeing the DHCP RENEW and/or DISCOVER packets going out to the wrong / non-existent IP address specifically (in unicast)?

itpp21

@melslenstra Only the renew request, it does not do discover ever unless it times out with the renew or dhclient is restarted.
I did end up here after massive amounts of searches which led to no solution as the problem is inside dhclient, your patching approach looked exactly like the behavior I am seeing (ignoring settings) where I have chosen a script to solve this.
This thread may not look the same but it sounds the same (it sounds like a duck ) with the core reason&effects.

tsimmons

I have subscribed to this same thread for years because I had very similar issues where DHCP client requests from my pfsense box didn't work and a client request from my Mac did; I have also seen upstream dhcp server addresses in the 10.x.x.x range; I did some pretty exhaustive research into the DHCP options but soon discovered there wasn't any effective way to modify them in pfsense so I ended up going with a big hammer: requested a static IP from my ISP and just hard-coded it in pfsense.

This is what I saw in the two requests:

PFSENSE DCHP options (didn't work):
	bootp: 0x0000
	option 12: hostname (pf_name)
	option 50: ip (192.168.100.10)
	-option 51: ** NOT SET, MISSING **
	option 53: type (discover)
	option 55: subnet_mask, broadcast, time, classless, router, domain_name, dns, hostname, domain_search, mtu
	-option 57: ** NOT SET, MISSING **
	option 61: client identifier (mac_address)

MAC DHCP options (did work):
	bootp: 0x0000
	option 12: hostname (mac_name)
	option 50: ip (192.168.17.100)
	option 51: leasetime (90 days)
	option 53: type (request, 3)
	option 55: subnet_mask, classless, router, dns, domain_name, domain_search, private, ldap, netbios_name, netbios_node
	option 57: dhcp ()
	option 61: client identifier (dhcp_id)

If there were some way to customize the specific DHCP options in pfsense, that would be ideal but it would probably require compiling your own dhclient and I'm running an SG-3100 appliance from NetGate and don't care to modify that much out of official support.

Edit: You can see what the options are from IANA.

itpp21

@tsimmons This is what I've seen in hundreds of posts over the past 5 years, including options set not being honored.
I've got a sg-5100 but this issue has been around for other pfSense based systems as well.
I don't mind changing things to solve issues but this one being around so long without any solution is a bit strange.

This may have started as a problem here when my ISP decided to handle routed and bridge mode from the same dhcp server as their modems first boot fully into routed mode and then switch to bridge (which makes sense of what I'm seeing) but then again I have been unable to convince them of this 'mistake' and pfSense can't handle this either as in bridge mode its getting invalid information.

Gertjan

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

as their modems first boot fully into routed mode and then switch to bridge (.... but then again I have been unable to convince them of this 'mistake' and pfSense can't handle this either as in bridge mode its getting invalid information.

What pfSense can do : with these two, you can instruct the DHCP client to wait after a (WAN) interface LINK UP, which happens when the modem reboots/reset/powers on. Now, the dhcp client wait "for the dust to settle" before it starts discovering.

@tsimmons said in DHCP client unable to get lease from cable provider [solved]:

from IANA.

Could find "dhcp-server-identifier" or "server-identifier" on that list. It's "54" ?

@melslenstra said in DHCP client unable to get lease from cable provider [solved]:

DHCP RENEW and/or DISCOVER packets going out to the wrong / non-existent IP address specifically (in unicast)?

I should have checked it, by now, but didi't : I really presume that DISCOVER will broadcast
Subsequent RENEWs will send direcly to "server-identifier" if it's known.

Btw, @itpp21 :

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

option dhcp-server-identifier 10.160.92.1; This is not a dhcp server! but pfSense keeps trying this one for hours
option dhcp-server-identifier 10.160.92.1; This is not a dhcp server! but pfSense keeps trying this one for hours
option tftp-server-name "212.142.63.132";

If the "dhcp-server-identifie" is set, the DHCP client should try that IP.
The tftp-server-name seems to be set correctly.

I put my money on "the dhcp server guy @ the ISP made an error".

@tsimmons said in DHCP client unable to get lease from cable provider [solved]:

If there were some way to customize the specific DHCP options in pfsense

Yes, as mentioned already above : build your own dhcp-client config file from scratch, and use that one.
Use the man page of that file to see all the commands.
BUT : there is no command option to discard "options" send by the DHCP server, as they are mandatory. And when they are wrong, they have to be corrected on that side.

itpp21

@gertjan said in DHCP client unable to get lease from cable provider [solved]:

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

as their modems first boot fully into routed mode and then switch to bridge (.... but then again I have been unable to convince them of this 'mistake' and pfSense can't handle this either as in bridge mode its getting invalid information.

What pfSense can do : with these two, you can instruct the DHCP client to wait after a (WAN) interface LINK UP, which happens when the modem reboots/reset/powers on. Now, the dhcp client wait "for the dust to settle" before it starts discovering.

Your missing an important point, dhclient suffers from memory loss.
If it starts it works by design, once its running it does not work by design, an obvious hint at this can be found here;

Press save on Wan config page,
'interfaces.inc':
/* Make sure dhclient is not running */
kill_dhclient_process($realif);

/* fire up dhclient */

.......

I appreciate your suggestions but for years this is not doing anything for anyone if used from a running dhclient as it does not honor any setting until it restarts (and then forgets everything) and the whole shebang starts again, which basically proves the point of having to use my script on a timer.

Gertjan

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

Your missing an important point, dhclient suffers from memory loss.

Received data is kept in it's working memory -
and stored in /var/db/dhclient.leases.{interface}

When the process restarted, because of a IF DOWN / UP, it restarts, parses the files, and knows as much as before.
And because it doesn't loses it's memory, you have to wipe its memory by killing the process.
killing the file
Restart.

Because it doesn't loose its 'memory', it (dhcpclient) will wake up if it was restarted, knows from FreeBSD ( pfSense) what the WAN interface is, finds the file /var/db/dhclient.leases.{interface}, sees the {interface}, sees that the IP is the one in the file and that that IP is the WAN IP, concludes that is is a DHCP type of IP, knows when to start RENEW as the time info is also stored in the file, and - the issue : knows that the DHCP server said "option dhcp-server-identifier 10.160.92.1; " as it has received that data, so it addresses the RENEW to "10.160.92.1" as it was told to do.

Thats why your killing scripts works.

I might be missing something, please detail, as I miss to understand what I miss ;)

itpp21

Received data is kept in it's working memory -
and stored in /var/db/dhclient.leases.{interface}

Yes but not settings.

issue : knows that the DHCP server said "option dhcp-server-identifier 10.160.92.1; " as it has received that data, so it addresses the RENEW to "10.160.92.1" as it was told to do.

"as it was told to do" is where it's going wrong.
When you set for example "supersede dhcp-server-identifier 255.255.255.255" either in the config or hardcoded, the running version of dhclient does not ever use this resulting in above behavior polling the wrong server (which you know is happening so you set supersede).
Such values are only honored at starttime, at runtime they are not used.

itpp21

Some interesting read why my ISP is causing a problem,

https://tools.ietf.org/html/rfc2132
9.7. Server Identifier

https://lists.isc.org/pipermail/dhcp-users/2016-October/020348.html

There appears this space allows 4 addresses (a1-a4) and that including this option is not really recommended and that there are ways to offer different addresses for different subnets.

Gertjan

@itpp21 said in DHCP client unable to get lease from cable provider [solved]:

this space allows 4 addresses (a1-a4)

Yeah, saw that post.
a1, ...a4 where each octets.
So, basicly, an IPv4addres.
Guess there is no such thing as IPv6 when that text was written ^^

cat chello.nl > " The use of the server-identifier statement is not recommended ".
You know it.
I know it.

itpp21

@gertjan 4 years ago,
https://redmine.pfsense.org/issues/7416
But I can find the same thing beyond 4 years and it looks like its still not implemented... if anyone is willing to supply me a binary build for dhclient in pfSense FreeBSD 11.3-STABLE with this patch I'm willing to pay for it.

In fact what annoys me the most is you have full control on the LAN side but just about nothing on the WAN side for dhcp, a simple --dhcp-server-ignore xyz or --dhcp-server-replace xyz,abc(regex) would give anyone control over what an ISP is doing, a clear google shows ISP's don't always know what their doing and without 3th line access as a customer your f***d.