Trying to find the reason for the failed DDNS update

DaHai8

Using Cloudflare DDNS on pfsense 2.8.1 and most of the time is works and updates. Yesterday, however, it failed with a red circle X next to the Dynamic DNS Client. And at that time, the external address had indeed changed and the old IP address was invalid. Clicking on Edit and Forcing and update worked.
And like I said, most of the time it works automatically. But yesterday it did not.
I checked the logs and this is all I could find around that time:

In the log, you can see that pfsense detected 'no change in my IP address..'. Yet when I logged in on the 10th I saw the DDNS entry had a red X by it and said 'Failed'.
I already new the external IP had changed thru manually checking it myself, and saw the old address was now invalid.
When I did a 'Force update' on that client, it worked and updated the DNS to the new IP address.
But I can't find anything in the system log to indicate why it failed.
None of the other System logs seem to have anything related to DDNS.
Ideas where to look to find out why DDNS failed to auto-update?

Thanks!

Gertjan

@DaHai8

Install the pfSense cron packag, and you'll see where these daily "01:01:0x" come from :

so every day, at 01 hour 01 minutes, /etc/rc.dyndns.update executes, and does a 'dyndns' if needed.

My question : your pfSense WAN interface, is it a an RFC1918 type of IP, or a "real Internet IP" ?
If it's an RFC1918, the upstream (ISP ?) router) might have changed it's ISP WAN IP, but pfSense wasn't ware of this. That's why the cron task exists : at least ones a day it check if "everything is still ok" = the recorded WAN IP is still the real WAN ISP IP.
It does so, by default, by using : see : pfSense : Services > Dynamic DNS > Check IP Services :
http://checkip.dyndns.org
check / click for yourself ;)

So, it could be possible that your WAN ISP IP did change.

You could edit the cron task, so it executes more often, like one every hour.

If the pfSense WAN interface does 'contains' the real ISP WAN IP, and it changes, then this will provoke an pfSense interface event. During this WAN interface event, a "/etc/rc.dyndns.update" will be executed, and a changed WAN IP will be detected 'real time'.

DaHai8

@Gertjan : Appologies for not stating that its double-natted through those wonderful Carrier Grade NATs.
However, if pfsense was not aware of this change in ISP WAN, why was there a Red X Circle by the Dynamic DNS Client with 'Failed' popup when I moused-over it?
To me, that means it tried to check the ISP Wan IP through the configured 'Check IP Service' and it failed.
And, yes, as I said, I did check it myself that is how I know that it had the wrong ISP WAN IP in the 'failed' Dynamic DNS client.
So, yes, as I said, the ip did change, the check in pfSense failed and yes, I did check it manually to see it had changed, and yes, I did update it manually and, yes, it did work, and yes, it did get the correct ISP WAN IP.
So... why did it fail before I manually updated it?
Why did it have a Red X Circle next to the Dynamic DNS Client?
Where I can look to see when it executed and why it failed?
There is nothing in the System log (as shown above).

Gertjan

@DaHai8 said in Trying to find the reason for the failed DDNS update:

.... double-natted through those wonderful Carrier Grade NATs.

Yeah, that's a good example of 'pfSEnse not beeing aware that something has changed upstream' - hence the daily check.

@DaHai8 said in Trying to find the reason for the failed DDNS update:

why was there a Red X Circle by the Dynamic DNS Client with 'Failed' popup when I moused-over it?
....
So... why did it fail before I manually updated it?

Because, the DynDNS widget will compare the cached (in a file) IP with the actual "WAN IP" (using http://checkip.dyndns.org) : in human language language :

If [(IP in file) is different as (the outcome of http://checkip.dyndns.org)]
Then : show a red cross
Else : Green Ok.

Afaik (I didn't check) : the widget informs you about the mismatch, the red cross, but doesn't tries to resolve the issue when it detects a mismatch.

Go for the fast solution : edit your cron time setting, make it execute more often.
If you set it to 'every hour', then that will be the maxium delay of an DyNDNS-IP mismatch.

DaHai8

@Gertjan : So...the page widget checks the external IP when you view the page, but unlike the Cron job, it doesn't update the cache if different, it just whines about it.
Odd and dumb, but not really surprising I guess.
I'll update the Cron job and cross my fingers.
Thanks

Gertjan

@DaHai8 said in Trying to find the reason for the failed DDNS update:

So...the page widget checks the external IP when you view the page

Follow me please :

....
Euh ...
https://github.com/pfsense/pfsense/blob/master/src/usr/local/www/widgets/widgets/dyn_dns_status.widget.php :

Github is ... to busy.

Plan B : open this file on your pfSense : usr/local/www/widgets/widgets/dyn_dns_status.widget.php
The test starts at line 97.
Line 98 : the cache file is located and the content is read. Now, you know where the file is: have a look at it yourself ?^^
Line 104 : if the file didn't exist, "N/A" is shown.

Line 109 : with the function dyndnsCheckIP() (you can find it in /etc/services.inc) the "WAN ISP IP" is checked, with "http://checkip.dyndns.org" or an URL you've supplied yourself.
Line 113 : you'll understand.

Line 114 and 116 : red or green.

@DaHai8 said in Trying to find the reason for the failed DDNS update:

Odd and dumb, but not really surprising I guess.

Maybe because the dashboard is already 'slow' as it is (as a lot has to execute to collect all the GUI dashboard glitter).

@DaHai8 said in Trying to find the reason for the failed DDNS update:

I'll update the Cron job and cross my fingers.

Keep us posted.

SteveITS

@Gertjan said in Trying to find the reason for the failed DDNS update:

DynDNS widget will compare the cached (in a file) IP with the actual "WAN IP" (using http://checkip.dyndns.org)

This is probably unrelated to OP's question but notably pfSense does not compare it to the dyndns hostname result, and thus doesn't update. For instance if you also update that dyndns hostname from another location (and now the A record is "wrong") the original will not detect that it needs to update the hostname. For instance if we restore a pfSense config on a new router before installation we have to do that offline or go force the live router to update its A record.

DaHai8

Ok. I think I'm clear on this now.
Originally, I thought the widget reported on the status of the Cron job success or failure. This was because a green checkmark on the Dynamic DNS Clients page shows 'Updated' when its green and you mouse-over it, and 'Failed' when its red and you mouse-over it. I took the 'failed' status as meaning the cron job failed, and wondered if it was the Check IP Service that failed, or the update to the A failed.
Now I understand that a 'failed' X just means the Cached IP does not match what the Check IP Service reported when that widget is run when that page is viewed. It actually has nothing to do with the status of the cron job that updates the DDNS A record.
It's probably just me, but it might make more sense if the Text hover for the Red X on that page showed 'Mismatch' or 'Out of Date' instead of 'failed'. To me, failed means it didn't work/run/succeed.
So, now I know what it really means and how to mitigate an external IP address change.
Thank you all for your help!