Dynamic DNS *NOT* Updating "Cached IP"

Gertjan

That looks like the issue :

@nikolaosinlight said in Dynamic DNS *NOT* Updating "Cached IP":

<interface-response>
<Command>SETDNSHOST</Command>
<Language>eng</Language>
<IP>174.91.131.4</IP>
<ErrCount>0</ErrCount>

Follow with me, starting from line 2325.

The 'payload', the xml that comes back from the namecheap dyndns server is aviable in the string $data.
First, if present, all "^M" occurrences are removed. (windows double line feed stuff)
Then $data - remember, a string xml data string, is converted to an array with the xml2array() function. This function gives back an array called $ncresponse, with the xml element separated in each array element.
Now, PHP can access each element with a nice "$ncresponse['interface-response']['ErrCount']"

All the if statements fail at this moment, line 2338 is executed : the 'payload' is shown ..... the $data string.

Or, you should have a array called $ncresponse
That array should contain, amongst others, a key called ['interface-response']
And there should be a array key insuie that element called ['ErrCount']
And the value of ErrCount should contain the value '0'.

The $data 'payload' shows that this is the case.

Still,

} else if ($ncresponse['interface-response']['ErrCount'] === "0") {

fails.

I propose :

Change the '===' in this line 2331 to '=='

If that doesn't work, undo the '==' change.

Instead of logging the $data string payload (line 2338), lets log the $ncresponse array.
Insert before line 2338 this line :

$data = print_r($ncresponse , true);

Your code looks now like :

						$status = $status_intro . "(" . gettext("Unknown Response") . ")";
						$data = print_r($ncresponse , true);
						log_error($status_intro . gettext("PAYLOAD:") . " " . $data)

and see what the logs (like the second image in your first post) tells you ....
This check shows us what the xml2array actually did (returned).

To undo this edit, just remove the line you added.

Btw : you can force a success, as we know the update was successful.
Add :

						$successful_update = true;

right after line 2339.
But that's just a work around, and not a solution to the issue.

barnettd

I am having the same issue on multiple devices running 21.05.2 using namecheap dynamic dns. I wasn't comfortable modifying code as suggested by @Gertjan so I created a new "custom" client using the name cheap update URL:
https://dynamicdns.park-your-domain.com/update?host=yoursubdomain&domain=YourDomain.tld&password=yourddnspassword

Here is my sanitized log for reference:

Dec  7 11:32:18 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS: updatedns() starting
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): 1.2.3.4 extracted from local system.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS (test.example.net): running get_failover_interface for wan. found mvneta2
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): _update() starting.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: HTTP/2 200
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: date: Tue, 07 Dec 2021 19:32:19 GMT
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: content-type: application/json
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: vary: Accept-Encoding
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: set-cookie: .s=e12b94d6aa2140f08484e67036fb699f; domain=.www.namecheap.com; path=/; httponly
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: strict-transport-security: max-age=16000000; includeSubDomains
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: strict-transport-security: max-age=16000000; includeSubDomains
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: cf-cache-status: DYNAMIC
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: server: cloudflare
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: cf-ray: 6ba02de7db6d7e58-LAX
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header: alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header:
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Header:
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Response Data: <?xml version="1.0" encoding="utf-16"?>
Dec  7 11:32:19 el php-fpm[10117]: <interface-response>
Dec  7 11:32:19 el php-fpm[10117]:   <Command>SETDNSHOST</Command>
Dec  7 11:32:19 el php-fpm[10117]:   <Language>eng</Language>
Dec  7 11:32:19 el php-fpm[10117]:   <IP>1.2.3.4</IP>
Dec  7 11:32:19 el php-fpm[10117]:   <ErrCount>0</ErrCount>
Dec  7 11:32:19 el php-fpm[10117]:   <errors />
Dec  7 11:32:19 el php-fpm[10117]:   <ResponseCount>0</ResponseCount>
Dec  7 11:32:19 el php-fpm[10117]:   <responses />
Dec  7 11:32:19 el php-fpm[10117]:   <Done>true</Done>
Dec  7 11:32:19 el php-fpm[10117]:   <debug><![CDATA[]]></debug>
Dec  7 11:32:19 el php-fpm[10117]: </interface-response>
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: Dynamic DNS namecheap (test.example.net): _checkStatus() starting.
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: phpDynDNS (el): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec  7 11:32:19 el php-fpm[10117]: <interface-response>
Dec  7 11:32:19 el php-fpm[10117]:   <Command>SETDNSHOST</Command>
Dec  7 11:32:19 el php-fpm[10117]:   <Language>eng</Language>
Dec  7 11:32:19 el php-fpm[10117]:   <IP>1.2.3.4</IP>
Dec  7 11:32:19 el php-fpm[10117]:   <ErrCount>0</ErrCount>
Dec  7 11:32:19 el php-fpm[10117]:   <errors />
Dec  7 11:32:19 el php-fpm[10117]:   <ResponseCount>0</ResponseCount>
Dec  7 11:32:19 el php-fpm[10117]:   <responses />
Dec  7 11:32:19 el php-fpm[10117]:   <Done>true</Done>
Dec  7 11:32:19 el php-fpm[10117]:   <debug><![CDATA[]]></debug>
Dec  7 11:32:19 el php-fpm[10117]: </interface-response>
Dec  7 11:32:19 el php-fpm[10117]: /services_dyndns_edit.php: phpDynDNS (el): (Unknown Response)

SteveITS

@barnettd Interesting, I have a router with 21.05 at home...hadn't bothered with 21.05.1 since it was for 32 bit ARM, a routing issue, and a captive portal fix, and 21.05.02 was only for 6100s? No notes for dynamic DNS.

On 21.05 my Namecheap DDNS is updating and green. Did it used to work for you on 21.05?

Edit: I should note I believe I found a while back that pfSense only updates the dyn DNS if it thinks the IP changed, not if it is different than what the provider thinks it is. Subtle difference, but basically it isn't sending the new IP on each connection. The edge case would be that something else changed the IP, then when pfSense checks it hasn't changed from pfSense's point of view.

barnettd

@steveits Yes it used to work. All the devices it is failing on have 2 internet connections and a gateway group (Tiers) that is monitored by Dynamic DNS. No changes were made to the configs.

I have noticed that when the gateway changes due to a connection outage it seems the IP is updated at namecheap, but when the gateway changes back to the primary, the dynamic dns stays on the secondary connection. Manually clicking the "Save and Force Update" button seemed to resolve the issue, but thats not ideal.

I honestly don't know if the issue is on the pfsense side or if namecheap changed their response somehow. Since yours is still working it seems to point toward pfsense.

nikolaosinlight

@barnettd somewhat comforting (but not really) that it isn't just me. I too have a tiered gateway group and yes tested the failure of the primary and it didn't switch the DNS.

For myself as well Namecheap gets updated and like you said if it works when there is no tiered gateway group then likely not a Namecheap interface change.

How do we get this looked at and resolved? To not update the IP in pfsense and stay red is a problem for sure as we can not turn up our frequency of cron checks... BUT to have tiered gateway failover with 2 ISPs and not failover the IP when the Tier 1 fails is really bad as services will fail - essentially there is no failover unless I am monitoring and force it as you did.

nikolaosinlight

@steveits I tried manually correcting the cached entries and reloaded the page and got full green. Then I dropped the primary (Tier 1) ISP and it went red, updated Namecheap correctly, and continued to show the prior IP address in UI and in the cache files.

I think @barnettd may be on to something in that this could be a bug related to a Tiered Gateway Group (which also does not work as it should when ISP fails).

I am only about a month into pfsense. Is there a place to report a bug? What is the process?

SteveITS

Since you mention gateway groups I did notice this redmine - DDNS set to a gateway group does not update on WAN failover but Jim couldn't duplicate the issue.

That redmine site is the place to report bugs.

biggsy

I noticed the following in my syslog and suspected a temporary problem but found the lines were also there yesterday:

Dec 10 01:01:01 php[4009]: rc.dyndns.update: phpDynDNS (mail): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec 10 01:01:01 php[4009]: <interface-response>
Dec 10 01:01:01 php[4009]:   <Command>SETDNSHOST</Command>
Dec 10 01:01:01 php[4009]:   <Language>eng</Language>
Dec 10 01:01:01 php[4009]:   <IP>[redacted]</IP>
Dec 10 01:01:01 php[4009]:   <ErrCount>0</ErrCount>
Dec 10 01:01:01 php[4009]:   <errors />
Dec 10 01:01:01 php[4009]:   <ResponseCount>0</ResponseCount>
Dec 10 01:01:01 php[4009]:   <responses />
Dec 10 01:01:01 php[4009]:   <Done>true</Done>
Dec 10 01:01:01 php[4009]:   <debug><![CDATA[]]></debug>
Dec 10 01:01:01 php[4009]: </interface-response>
Dec 10 01:01:01 php[4009]: rc.dyndns.update: phpDynDNS (mail): (Unknown Response)
Dec 10 01:01:02 php[4009]: rc.dyndns.update: phpDynDNS (@): PAYLOAD: <?xml version="1.0" encoding="utf-16"?>
Dec 10 01:01:02 php[4009]: <interface-response>
Dec 10 01:01:02 php[4009]:   <Command>SETDNSHOST</Command>
Dec 10 01:01:02 php[4009]:   <Language>eng</Language>
Dec 10 01:01:02 php[4009]:   <IP>[redacted]</IP>
Dec 10 01:01:02 php[4009]:   <ErrCount>0</ErrCount>
Dec 10 01:01:02 php[4009]:   <errors />
Dec 10 01:01:02 php[4009]:   <ResponseCount>0</ResponseCount>
Dec 10 01:01:02 php[4009]:   <responses />
Dec 10 01:01:02 php[4009]:   <Done>true</Done>
Dec 10 01:01:02 php[4009]:   <debug><![CDATA[]]></debug>
Dec 10 01:01:02 php[4009]: </interface-response>
Dec 10 01:01:02 php[4009]: rc.dyndns.update: phpDynDNS (@): (Unknown Response)

That brought me to this topic. I have only one WAN, so no gateway groups.

I updated from 2.4.5_P1 to 2.5.2 back in September and I've made no changes to the Dynamic DNS setup for well over a year.

Older syslog records show that the Namecheap DDNS records were successfully updated in Sep, Oct and Nov.

I tried editing the DynDNS settings and saving. Only the "mail" record was updated to my current public IP. The generic (@) record now shows as 0.0.0.0 in the GUI. (I can live without the generic, though, as long as the mail gets through.)

The only significant change to my pfSense since updating to 2.5.2 was to install the Wireguard package.

I haven't been able to find any indication that Namecheap have changed anything at their end.

Edit: Will be watching my friend's pfSense, which is due to update his DDNS records, also at Namecheap, tomorrow morning.

biggsy

Yep, same logs on my friend's machine.

bmeeks

While looking around for other stuff, I found this link that appears related to the issue posted in this thread: https://www.reddit.com/r/NameCheap/comments/qz1mjf/namecheap_dynamic_dns_returning_utf16_encoded/hljqzja/. This does in fact appear to perhaps be a Namecheap problem.

biggsy

@bmeeks

Many thanks for finding this. I see that the original post and the last response form Namecheap were 21 days ago.

I just hope they don't start blocking because of too-frequent updates, which are now occurring daily because of this problem.

bmeeks

@biggsy said in Dynamic DNS *NOT* Updating "Cached IP":

@bmeeks

Many thanks for finding this. I see that the original post and the last response form Namecheap were 21 days ago.

I just hope they don't start blocking because of too-frequent updates, which are now occurring daily because of this problem.

The guys over on the "other Sense" forum site are also reporting the same issue.

biggsy

@bmeeks

It's Namecheap so perhaps I shouldn't be surprised that I can't find any DDNS update "abuse" policy on their web site.

barnettd

@bmeeks That issue definitely matches what I'm seeing. Setting up a custom client using the namecheap update URL seems to be work for now.

Hopefully this is resolved soon, I have quite a few devices using Dynamic DNS from namecheap and it would be a pain to change.

nikolaosinlight

@barnettd

So setting up a custom Dynamic DNS for Namecheap only works for me if I leave Result Match blank - even if I modify the IP address and save the full response as the following it does not work - so if Namecheap fails to update I am completely hosed:

<?xml version="1.0" encoding="utf-16"?>
<interface-response>
  <Command>SETDNSHOST</Command>
  <Language>eng</Language>
  <IP>%IP%</IP>
  <ErrCount>0</ErrCount>
  <errors />
  <ResponseCount>0</ResponseCount>
  <responses />
  <Done>true</Done>
  <debug><![CDATA[]]></debug>
</interface-response>

I opened a ticket with Namecheap and this was their first response:

"You are right, unfortunately, we have such a glitch in our system currently. Our technical department is working on the resolution, however, we have no certain time frames when the issue is solved. Although, we have checked it from our side, and it does not affect the main functionality of the Dynamic DNS feature, and the IP address should be updated successfully. We see from your request, that you may face an issue with the regular propagation. The only case is to wait up to 30 minutes till the A record is propagated to a new one. Unfortunately, such a process cannot be sped up, as it is for a 100% automatic one and it is a regular procedure. "

When I responded that the above makes not sense as Namecheap fully updates but it is rather the response that is not properly formulated for pfsense to process and moreover pfsense will keep trying pointlessly they said:

"We perfectly understand your concern. The matter regarding the incorrect encoding format has been already escalated. To our regret, there is no ETA yet."

Changing the encoding of a response from UTF-16 to UTF-8 is not rocket science.

As such, I am not holding my breath and think it is time to look at other Registrars (we have been doing more and more with AWS recently so likely will go that route).

--Nikolaos

SteveITS

From the Reddit page, "They think a recent change in php started enforcing the UTF encoded verification, which means it's possible namecheap's xml response has been wrong for quite some time but not noticed until now." I have no idea but is there a way to turn that off in PHP? Not a great solution but could be a workaround. The odd thing is it doesn't look like PHP was updated in 21.05.x/2.5.x builds. There was the JIT/PCRE fix for the 3100 and 1000.

nikolaosinlight

@steveits
I pointed Namecheap to the Reddit post and this post and they say they have no ETA.

I get software interfaces are hard to update especially if they have worked well in the past so they definitely don't want to break existing functionality but seems like they will not fix is my guess. If they do... great... but from their response it does not seem likely it will be anytime soon or possibly ever.

--Nikolaos

Gertjan

@nikolaosinlight

Then the solution, even temporary, is rather easy :
change dyndns.class behavior so it accepts the 'new' answer as the 'all is good' answer.

This

... ($ncresponse['interface-response']['ErrCount'] === "0") ....

part should be 0 or '0' or - I don't know, some new UTF like 0 or '0'
Is it an integer value - a text value ? 8 bit or other type of encoding ?

Whatever, make the test 'succeed' and you will all be good.
A soon as it re breaks again, undo the edits and the story is over.

Btw : I can't play with this myself, as I'm not a namecheap customer.

edit : I hope for them that this is only a dyndns.class problem, because, if not, every customer that uses a dyndns client will hammer their Dyndns service constantly ....

SteveITS

@gertjan said in Dynamic DNS *NOT* Updating "Cached IP":

change dyndns.class behavior so it accepts the 'new' answer

I suspect the issue is that $ncresponse is considered empty since the response is invalid due to the incorrect encoding. (or something similar, I did not look into this at all, just reading posts)

Presumably from Namecheap's standpoint they want to send changes through QA/testing so as not to break things for everyone else.

bmeeks

Here is a link to the other site's issue on this Namecheap dynamic DNS problem: https://github.com/opnsense/plugins/issues/2666#issuecomment-979154701.

If you follow the comments and links, you may uncover a way to duplicate the workaround on pfSense. I highly doubt the source code nor specific file matches, but the overall gist of the fix should be applicable on pfSense. Note that the "fix" is really just a temp workaround until Namecheap fixes their core issue.

The root cause of the problem is incorrect type coding by Namecheap.