Dynamic DNS *NOT* Updating "Cached IP"

Gertjan

A 'cryptic' answer :
https://github.com/pfsense/pfsense/blob/332052b8bd2a5d35662be2dba773b7a9f0d50681/src/etc/inc/dyndns.class#L2846

The cache files, one for IPv4 or IPv6 (if your using IPv6) is only updated if the update (against namecheap) was 'successful_update' must be 'true'.
And if it was true, read on a line or two, and find the simple test checks if the WAN IP is NOT zero '0'.
If that's also true, then the file(s) is/are updated.

So that explains why you saw an initial "0.0.0.0|xxxxxxxxx" - and why the file didn't get updated.

At line 2324 you can see what must happen, so pfSEnse knows if the update was successful :
namecheap should return 'something with the error count (after xml parsing) set to zero, and in that case the meassage "IP Address Updated Successfully!" is returned, and successful_update is set to "true".
Or it did return (the beginning of the payload) "<?xml version="1.0" encoding="utf-16"?>". Take note that more info is returned, only the first array line is shown.

Check with https://www.namecheap.com/support/knowledgebase/article.aspx/29/11/how-to-dynamically-update-the-hosts-ip-with-an-http-request/ if your parameters are ok (password and domain name).

See for more info here (old, maybe more recent info is available).

I'm not using namecheap myself.

nikolaosinlight

@gertjan

I understand why the initial cached IP was 0.0.0.0 and I even understand why it would be green when the value that is cached (and I manually updated) corresponds to the current IP.

However, clearly there is no apparent issue with my configuration as the IP address is changing on NameCheap (there are 3 entries and all 3 get updated in NameCheap with different hostname for each entry). So no it cannot be a password issue like the post you referenced where a space was present b/c in that persons case the record was not getting updated in NameCheap.

What I don't understand is why after getting a response from NameCheap pfsense does not update the cached entry. Somehow pfsense cannot interpret the response:

Done is true so that seems fine
Could it be ResponseCount which is 0?

Ultimately, something is leading pfsense to throw its hands up in the air and say "Unknown Response". Ideas? Thoughts?

Gertjan

@nikolaosinlight

pfSense thrwos in the towel because the initial 'if' case at line 2327 and the following else cases are all 'false'.
So the last else case is executed, which prints the 'payload'.

Consider line 2326 : it would be a good thing to see the entire 'anwer' (the xml result array), not just the first line.

If namecheap changed the resulting 'xml' (just a bunch of text lines) result of what is sends back, even with a message that indicated : 'all is well' then pfSense doesn't recognize the answer any more, and concludes a 'error'.
On the namecheap side, everything is fine : the IP is updated as you mentioned.
Something mist be wrong, as line 2331 ( ps -ax | grep "unbound") is NON zero, which means there is an error condition.

Btw :
I've checked

so, according to line 1625, the data chunck that comes back from the dyndns service is written out to the logs :

2021-11-30 12:34:17.735768+01:00 	php-fpm 	26619 	/rc.dyndns.update: phpDynDNS (245809): (Success) No Change In IP Address.
2021-11-30 12:34:17.723084+01:00 	php-fpm 	26619 	/rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_wanhe-net-tunnelbroker'245809'0.cache: 82.127.34.254
2021-11-30 12:34:17.715726+01:00 	php-fpm 	26619 	/rc.dyndns.update: Dynamic DNS he-net-tunnelbroker (245809): 82.127.34.254 extracted from Check IP Service
2021-11-30 12:34:17.226359+01:00 	php-fpm 	26619 	/rc.dyndns.update: Dynamic DNS he-net-tunnelbroker (245809): _checkStatus() starting.
2021-11-30 12:34:17.221878+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Data: nochg 82.127.34.254
2021-11-30 12:34:17.217371+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header:
2021-11-30 12:34:17.212892+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header:
2021-11-30 12:34:17.208441+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Content-Type: text/html; charset=utf-8
2021-11-30 12:34:17.203979+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Connection: close
2021-11-30 12:34:17.199507+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Content-Length: 20
2021-11-30 12:34:17.195006+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Referrer-Policy: strict-origin
2021-11-30 12:34:17.190531+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: X-Content-Type-Options: nosniff
2021-11-30 12:34:17.186116+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: X-XSS-Protection: 1; mode=block
2021-11-30 12:34:17.181605+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: X-Frame-Options: DENY
2021-11-30 12:34:17.177183+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Set-Cookie: referer=Direct+Access; path=/; secure
2021-11-30 12:34:17.172680+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
2021-11-30 12:34:17.168247+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Server: Apache/2.4.18 (Ubuntu)
2021-11-30 12:34:17.163795+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: Date: Tue, 30 Nov 2021 11:34:17 GMT
2021-11-30 12:34:17.161276+01:00 	php-fpm 	26619 	/rc.dyndns.update: Response Header: HTTP/1.0 200 ok
2021-11-30 12:34:16.286608+01:00 	php-fpm 	26619 	/rc.dyndns.update: Dynamic DNS he-net-tunnelbroker (245809): _update() starting.

(log lines in reverse order)

You saw the 'nochg' : it means "no change". This message could show up when you update an IP that is already updated. Doing this to often, and the dyndns could start to behave differently (== other return message !?!) or even blocking you.

nikolaosinlight

@gertjan

Are you referring to my post b/c I don't see any of the response headers you show above and definitely do not see any "nochg" in the logs.

The response payload from Namecheap in what I get is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<interface-response>
   <Command>SETDNSHOST</Command>
   <Language>eng</Language>
   <IP>174.91.131.4</IP>
   <ErrCount>0</ErrCount>
   <errors />
   <ResponseCount>0</ResponseCount>
   <responses />
   <Done>true</Done>
   <debug />
</interface-response>

What I don't get is the 2331 that is skipped over clearly should hit as ErrCount is clearly "0"

} else if ($ncresponse['interface-response']['ErrCount'] === "0") {
         $status = $status_intro . $success_str . gettext("IP Address Updated Successfully!");
         $successful_update = true;

Instead the last else is hit which dumps the payload at line 2336:

} else {
	$status = $status_intro . "(" . gettext("Unknown Response") . ")";
	log_error($status_intro . gettext("PAYLOAD:") . " " . $data);
	$this->_debug($data);

Lastly, where are you turning on "Verbose logging"?

In the end, I really don't get why this fails, and moreover how to resolve this b/c in the last 3 days the IP's are still showing red and for sure Namecheap service is not going to be happy and will likely ban us. I have setup the next update to happen in about 30 hours so will see....

Gertjan

That looks like the issue :

@nikolaosinlight said in Dynamic DNS *NOT* Updating "Cached IP":

<interface-response>
<Command>SETDNSHOST</Command>
<Language>eng</Language>
<IP>174.91.131.4</IP>
<ErrCount>0</ErrCount>

Follow with me, starting from line 2325.

The 'payload', the xml that comes back from the namecheap dyndns server is aviable in the string $data.
First, if present, all "^M" occurrences are removed. (windows double line feed stuff)
Then $data - remember, a string xml data string, is converted to an array with the xml2array() function. This function gives back an array called $ncresponse, with the xml element separated in each array element.
Now, PHP can access each element with a nice "$ncresponse['interface-response']['ErrCount']"

All the if statements fail at this moment, line 2338 is executed : the 'payload' is shown ..... the $data string.

Or, you should have a array called $ncresponse
That array should contain, amongst others, a key called ['interface-response']
And there should be a array key insuie that element called ['ErrCount']
And the value of ErrCount should contain the value '0'.

The $data 'payload' shows that this is the case.

Still,

} else if ($ncresponse['interface-response']['ErrCount'] === "0") {

fails.

I propose :

Change the '===' in this line 2331 to '=='

If that doesn't work, undo the '==' change.

Instead of logging the $data string payload (line 2338), lets log the $ncresponse array.
Insert before line 2338 this line :

$data = print_r($ncresponse , true);

Your code looks now like :

						$status = $status_intro . "(" . gettext("Unknown Response") . ")";
						$data = print_r($ncresponse , true);
						log_error($status_intro . gettext("PAYLOAD:") . " " . $data)

and see what the logs (like the second image in your first post) tells you ....
This check shows us what the xml2array actually did (returned).

To undo this edit, just remove the line you added.

Btw : you can force a success, as we know the update was successful.
Add :

						$successful_update = true;

right after line 2339.
But that's just a work around, and not a solution to the issue.

barnettd

I am having the same issue on multiple devices running 21.05.2 using namecheap dynamic dns. I wasn't comfortable modifying code as suggested by @Gertjan so I created a new "custom" client using the name cheap update URL:
https://dynamicdns.park-your-domain.com/update?host=yoursubdomain&domain=YourDomain.tld&password=yourddnspassword

Here is my sanitized log for reference:

Dec  7 11:32:18 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS: updatedns() starting
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): 1.2.3.4 extracted from local system.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS (test.example.net): running get_failover_interface for wan. found mvneta2
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): _update() starting.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: HTTP/2 200
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: date: Tue, 07 Dec 2021 19:32:19 GMT
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: content-type: application/json
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: vary: Accept-Encoding
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: set-cookie: .s=e12b94d6aa2140f08484e67036fb699f; domain=.www.namecheap.com; path=/; httponly
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: strict-transport-security: max-age=16000000; includeSubDomains
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: strict-transport-security: max-age=16000000; includeSubDomains
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: cf-cache-status: DYNAMIC
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: server: cloudflare
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: cf-ray: 6ba02de7db6d7e58-LAX
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header:
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header:
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Data: <?xml version="1.0" encoding="utf-16"?>
Dec  7 11:32:19 el php-fpm[10117]: <interface-response>
Dec  7 11:32:19 el php-fpm[10117]:   <Command>SETDNSHOST</Command>
Dec  7 11:32:19 el php-fpm[10117]:   <Language>eng</Language>
Dec  7 11:32:19 el php-fpm[10117]:   <IP>1.2.3.4</IP>
Dec  7 11:32:19 el php-fpm[10117]:   <ErrCount>0</ErrCount>
Dec  7 11:32:19 el php-fpm[10117]:   <errors />
Dec  7 11:32:19 el php-fpm[10117]:   <ResponseCount>0</ResponseCount>
Dec  7 11:32:19 el php-fpm[10117]:   <responses />
Dec  7 11:32:19 el php-fpm[10117]:   <Done>true</Done>
Dec  7 11:32:19 el php-fpm[10117]:   <debug><![CDATA[]]></debug>
Dec  7 11:32:19 el php-fpm[10117]: </interface-response>
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): _checkStatus() starting.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: phpDynDNS (el): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec  7 11:32:19 el php-fpm[10117]: <interface-response>
Dec  7 11:32:19 el php-fpm[10117]:   <Command>SETDNSHOST</Command>
Dec  7 11:32:19 el php-fpm[10117]:   <Language>eng</Language>
Dec  7 11:32:19 el php-fpm[10117]:   <IP>1.2.3.4</IP>
Dec  7 11:32:19 el php-fpm[10117]:   <ErrCount>0</ErrCount>
Dec  7 11:32:19 el php-fpm[10117]:   <errors />
Dec  7 11:32:19 el php-fpm[10117]:   <ResponseCount>0</ResponseCount>
Dec  7 11:32:19 el php-fpm[10117]:   <responses />
Dec  7 11:32:19 el php-fpm[10117]:   <Done>true</Done>
Dec  7 11:32:19 el php-fpm[10117]:   <debug><![CDATA[]]></debug>
Dec  7 11:32:19 el php-fpm[10117]: </interface-response>
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: phpDynDNS (el): (Unknown Response)

SteveITS

@barnettd Interesting, I have a router with 21.05 at home...hadn't bothered with 21.05.1 since it was for 32 bit ARM, a routing issue, and a captive portal fix, and 21.05.02 was only for 6100s? No notes for dynamic DNS.

On 21.05 my Namecheap DDNS is updating and green. Did it used to work for you on 21.05?

Edit: I should note I believe I found a while back that pfSense only updates the dyn DNS if it thinks the IP changed, not if it is different than what the provider thinks it is. Subtle difference, but basically it isn't sending the new IP on each connection. The edge case would be that something else changed the IP, then when pfSense checks it hasn't changed from pfSense's point of view.

barnettd

@steveits Yes it used to work. All the devices it is failing on have 2 internet connections and a gateway group (Tiers) that is monitored by Dynamic DNS. No changes were made to the configs.

I have noticed that when the gateway changes due to a connection outage it seems the IP is updated at namecheap, but when the gateway changes back to the primary, the dynamic dns stays on the secondary connection. Manually clicking the "Save and Force Update" button seemed to resolve the issue, but thats not ideal.

I honestly don't know if the issue is on the pfsense side or if namecheap changed their response somehow. Since yours is still working it seems to point toward pfsense.

nikolaosinlight

@barnettd somewhat comforting (but not really) that it isn't just me. I too have a tiered gateway group and yes tested the failure of the primary and it didn't switch the DNS.

For myself as well Namecheap gets updated and like you said if it works when there is no tiered gateway group then likely not a Namecheap interface change.

How do we get this looked at and resolved? To not update the IP in pfsense and stay red is a problem for sure as we can not turn up our frequency of cron checks... BUT to have tiered gateway failover with 2 ISPs and not failover the IP when the Tier 1 fails is really bad as services will fail - essentially there is no failover unless I am monitoring and force it as you did.

nikolaosinlight

@steveits I tried manually correcting the cached entries and reloaded the page and got full green. Then I dropped the primary (Tier 1) ISP and it went red, updated Namecheap correctly, and continued to show the prior IP address in UI and in the cache files.

I think @barnettd may be on to something in that this could be a bug related to a Tiered Gateway Group (which also does not work as it should when ISP fails).

I am only about a month into pfsense. Is there a place to report a bug? What is the process?

SteveITS

Since you mention gateway groups I did notice this redmine - DDNS set to a gateway group does not update on WAN failover but Jim couldn't duplicate the issue.

That redmine site is the place to report bugs.

biggsy

I noticed the following in my syslog and suspected a temporary problem but found the lines were also there yesterday:

Dec 10 01:01:01 php[4009]: rc.dyndns.update: phpDynDNS (mail): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec 10 01:01:01 php[4009]: <interface-response>
Dec 10 01:01:01 php[4009]:   <Command>SETDNSHOST</Command>
Dec 10 01:01:01 php[4009]:   <Language>eng</Language>
Dec 10 01:01:01 php[4009]:   <IP>[redacted]</IP>
Dec 10 01:01:01 php[4009]:   <ErrCount>0</ErrCount>
Dec 10 01:01:01 php[4009]:   <errors />
Dec 10 01:01:01 php[4009]:   <ResponseCount>0</ResponseCount>
Dec 10 01:01:01 php[4009]:   <responses />
Dec 10 01:01:01 php[4009]:   <Done>true</Done>
Dec 10 01:01:01 php[4009]:   <debug><![CDATA[]]></debug>
Dec 10 01:01:01 php[4009]: </interface-response>
Dec 10 01:01:01 php[4009]: rc.dyndns.update: phpDynDNS (mail): (Unknown Response)
Dec 10 01:01:02 php[4009]: rc.dyndns.update: phpDynDNS (@): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec 10 01:01:02 php[4009]: <interface-response>
Dec 10 01:01:02 php[4009]:   <Command>SETDNSHOST</Command>
Dec 10 01:01:02 php[4009]:   <Language>eng</Language>
Dec 10 01:01:02 php[4009]:   <IP>[redacted]</IP>
Dec 10 01:01:02 php[4009]:   <ErrCount>0</ErrCount>
Dec 10 01:01:02 php[4009]:   <errors />
Dec 10 01:01:02 php[4009]:   <ResponseCount>0</ResponseCount>
Dec 10 01:01:02 php[4009]:   <responses />
Dec 10 01:01:02 php[4009]:   <Done>true</Done>
Dec 10 01:01:02 php[4009]:   <debug><![CDATA[]]></debug>
Dec 10 01:01:02 php[4009]: </interface-response>
Dec 10 01:01:02 php[4009]: rc.dyndns.update: phpDynDNS (@): (Unknown Response)

That brought me to this topic. I have only one WAN, so no gateway groups.

I updated from 2.4.5_P1 to 2.5.2 back in September and I've made no changes to the Dynamic DNS setup for well over a year.

Older syslog records show that the Namecheap DDNS records were successfully updated in Sep, Oct and Nov.

I tried editing the DynDNS settings and saving. Only the "mail" record was updated to my current public IP. The generic (@) record now shows as 0.0.0.0 in the GUI. (I can live without the generic, though, as long as the mail gets through.)

The only significant change to my pfSense since updating to 2.5.2 was to install the Wireguard package.

I haven't been able to find any indication that Namecheap have changed anything at their end.

Edit: Will be watching my friend's pfSense, which is due to update his DDNS records, also at Namecheap, tomorrow morning.

biggsy

Yep, same logs on my friend's machine.

bmeeks

While looking around for other stuff, I found this link that appears related to the issue posted in this thread: https://www.reddit.com/r/NameCheap/comments/qz1mjf/namecheap_dynamic_dns_returning_utf16_encoded/hljqzja/. This does in fact appear to perhaps be a Namecheap problem.

biggsy

@bmeeks

Many thanks for finding this. I see that the original post and the last response form Namecheap were 21 days ago.

I just hope they don't start blocking because of too-frequent updates, which are now occurring daily because of this problem.

bmeeks

@biggsy said in Dynamic DNS *NOT* Updating "Cached IP":

@bmeeks

Many thanks for finding this. I see that the original post and the last response form Namecheap were 21 days ago.

I just hope they don't start blocking because of too-frequent updates, which are now occurring daily because of this problem.

The guys over on the "other Sense" forum site are also reporting the same issue.

biggsy

@bmeeks

It's Namecheap so perhaps I shouldn't be surprised that I can't find any DDNS update "abuse" policy on their web site.

barnettd

@bmeeks That issue definitely matches what I'm seeing. Setting up a custom client using the namecheap update URL seems to be work for now.

Hopefully this is resolved soon, I have quite a few devices using Dynamic DNS from namecheap and it would be a pain to change.

nikolaosinlight

@barnettd

So setting up a custom Dynamic DNS for Namecheap only works for me if I leave Result Match blank - even if I modify the IP address and save the full response as the following it does not work - so if Namecheap fails to update I am completely hosed:

<?xml version="1.0" encoding="utf-16"?>
<interface-response>
  <Command>SETDNSHOST</Command>
  <Language>eng</Language>
  <IP>%IP%</IP>
  <ErrCount>0</ErrCount>
  <errors />
  <ResponseCount>0</ResponseCount>
  <responses />
  <Done>true</Done>
  <debug><![CDATA[]]></debug>
</interface-response>

I opened a ticket with Namecheap and this was their first response:

"You are right, unfortunately, we have such a glitch in our system currently. Our technical department is working on the resolution, however, we have no certain time frames when the issue is solved. Although, we have checked it from our side, and it does not affect the main functionality of the Dynamic DNS feature, and the IP address should be updated successfully. We see from your request, that you may face an issue with the regular propagation. The only case is to wait up to 30 minutes till the A record is propagated to a new one. Unfortunately, such a process cannot be sped up, as it is for a 100% automatic one and it is a regular procedure. "

When I responded that the above makes not sense as Namecheap fully updates but it is rather the response that is not properly formulated for pfsense to process and moreover pfsense will keep trying pointlessly they said:

"We perfectly understand your concern. The matter regarding the incorrect encoding format has been already escalated. To our regret, there is no ETA yet."

Changing the encoding of a response from UTF-16 to UTF-8 is not rocket science.

As such, I am not holding my breath and think it is time to look at other Registrars (we have been doing more and more with AWS recently so likely will go that route).

--Nikolaos

SteveITS

From the Reddit page, "They think a recent change in php started enforcing the UTF encoded verification, which means it's possible namecheap's xml response has been wrong for quite some time but not noticed until now." I have no idea but is there a way to turn that off in PHP? Not a great solution but could be a workaround. The odd thing is it doesn't look like PHP was updated in 21.05.x/2.5.x builds. There was the JIT/PCRE fix for the 3100 and 1000.