Cloudflare dynamic dns not working
-
I didn't like how much was hardcoded, so here is what I did to get it working on mine. I am currently running 2.2 but the fork/commit is based on the current master. The majority of this was from brainlesscurls: https://github.com/brainlesscurls/pfsense/commit/4f3ce17b98919e464044cbc983e31549b595bdc3
Here is the code that I created, it does not correctly handle the response coming back or check if cached differs at all:
https://github.com/pivconcto/pfsense/commit/40400bd172ae1097b5c0db540cf4dde20e9f78d7
-
I have sorted this not after a lot of research.
I stumbled across this website: http://blog.t-m-s.co.uk/ with a blog post with exactly our issue and solution with a very good explanation on the use of the CloudFlare API.
I recommend reading through this completely to understand what is going on.
After following the steps in the website and tweaking a few things to cater for my needs it's up and running beautifully.
-
I actually worked out a fix at the dyndns.class file:
CloudFlare now requires a "record ID" in order to update an address, for such I did another curl call before pfsense change of the IP in order to get the record ID from the target.
Also the CloudFlare URL is quite different from the one in the original file of pfSense.
if you open the /etc/inc/dyndns.class, line 607, the original case looks like:
case 'cloudflare':
$needsIP = TRUE;
$dnsServer ='www.cloudflare.com';
$dnsHost = str_replace(' ','', $this->_dnsHost);
$URL = "https://{$dnsServer}/api.html?a=DIUP&email={$this->_dnsUser}&tkn={$this->_dnsPass}&ip={$this->_dnsIP}&hosts={$dnsHost}";
curl_setopt($ch, CURLOPT_URL, $URL);
break;I change it to:
case 'cloudflare':
$needsIP = TRUE;
$dnsServer ='www.cloudflare.com';
list ($host,$zone) = split('.', $this->_dnsHost, 2);
$getid = "https://$dnsServer/api_json.html?a=rec_load_all&tkn={$this->_dnsPass}&email={$this->_dnsUser}&z=$zone";
$crl = curl_init();
$timeout = 5;
curl_setopt ($crl, CURLOPT_URL,$getid);
curl_setopt ($crl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($crl, CURLOPT_CONNECTTIMEOUT, $timeout);
$ids = curl_exec($crl);
curl_close($crl);
$json_ids = json_decode($ids, true);
$jsonIterator = new RecursiveIteratorIterator(new RecursiveArrayIterator($json_ids),RecursiveIteratorIterator::SELF_FIRST);
$recid=0;
foreach ($jsonIterator as $key => $val) { if(!is_array($val)) { if ($key == "rec_id") { $recid=$val; } if ($key == "name" and $val == $this->_dnsHost) { break; } } }
$dnsHost = str_replace(' ','', $this->_dnsHost);
$URL = "https://{$dnsServer}/api_json.html?a=rec_edit&tkn={$this->_dnsPass}&email={$this->_dnsUser}&id=$recid&z=$zone&type=A&name=$host&content={$this->_dnsIP}&service_mode=1&ttl=1";
curl_setopt($ch, CURLOPT_URL, $URL);
break;In the Dynamic DNS interface all the information is pretty much the same, except the password must be the Token ID found at https://www.cloudflare.com/my-account ("Your API key is: ")
With this change he is automatically updating CloudFlare but the cached IP address still 0.0.0.0, something else must change to make this value also correct.
I hope this help to fix the problem for other people as well.
-
I actually worked out a fix at the dyndns.class file: CloudFlare now requires a "record ID" in order to update an address, for such I did another curl call before pfsense change of the IP in order to get the record ID from the target. Also the CloudFlare URL is quite different from the one in the original file of pfSense. if you open the /etc/inc/dyndns.class, line 607, the original case looks like: case 'cloudflare': $needsIP = TRUE; $dnsServer ='www.cloudflare.com'; $dnsHost = str_replace(' ','', $this->_dnsHost); $URL = "https://{$dnsServer}/api.html?a=DIUP&email={$this->_dnsUser}&tkn={$this->_dnsPass}&ip={$this->_dnsIP}&hosts={$dnsHost}"; curl_setopt($ch, CURLOPT_URL, $URL); break; I change it to: case 'cloudflare': $needsIP = TRUE; $dnsServer ='www.cloudflare.com'; list ($host,$zone) = split('\.', $this->_dnsHost, 2); $getid = "https://$dnsServer/api_json.html?a=rec_load_all&tkn={$this->_dnsPass}&email={$this->_dnsUser}&z=$zone"; $crl = curl_init(); $timeout = 5; curl_setopt ($crl, CURLOPT_URL,$getid); curl_setopt ($crl, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($crl, CURLOPT_CONNECTTIMEOUT, $timeout); $ids = curl_exec($crl); curl_close($crl); $json_ids = json_decode($ids, true); $jsonIterator = new RecursiveIteratorIterator(new RecursiveArrayIterator($json_ids),RecursiveIteratorIterator::SELF_FIRST); $recid=0; foreach ($jsonIterator as $key => $val) { if(!is_array($val)) { if ($key == "rec_id") { $recid=$val; } if ($key == "name" and $val == $this->_dnsHost) { break; } } } $dnsHost = str_replace(' ','', $this->_dnsHost); $URL = "https://{$dnsServer}/api_json.html?a=rec_edit&tkn={$this->_dnsPass}&email={$this->_dnsUser}&id=$recid&z=$zone&type=A&name=$host&content={$this->_dnsIP}&service_mode=1&ttl=1"; curl_setopt($ch, CURLOPT_URL, $URL); break; In the Dynamic DNS interface all the information is pretty much the same, except the password must be the Token ID found at https://www.cloudflare.com/my-account ("Your API key is: ") With this change he is automatically updating CloudFlare but the cached IP address still 0.0.0.0, something else must change to make this value also correct. I hope this help to fix the problem for other people as well.
I tried your fix but it just makes pfsense spit out crash reports and it still don't appear to be updating.I didn't delete my old cloudflare entry and start over so I assume that is what caused it to not load the changes. -
So after spending some time at this I came up with a working solution that uses CloudFlare's latest API. Be sure to use your account API key for the password field instead of your account password.
It updates the cached IP also and handles some basic errors.
Tested on 2.2.3-RELEASE 32 bit and 64 bit systems without errors.
If you choose "Diagnostics -> Edit File" then open the "/etc/inc/dyndns.class" file and edit the lines to the ones below.
Original
case 'cloudflare': $needsIP = TRUE; $dnsServer ='www.cloudflare.com'; $dnsHost = str_replace(' ','', $this->_dnsHost); $URL = "https://{$dnsServer}/api.html?a=DIUP&email={$this->_dnsUser}&tkn={$this->_dnsPass}&ip={$this->_dnsIP}&hosts={$dnsHost}"; curl_setopt($ch, CURLOPT_URL, $URL); break;
Patch
case 'cloudflare': $needsIP = TRUE; $dnsServer ='api.cloudflare.com'; $dnsHost = str_replace(' ','', $this->_dnsHost); $host_names = explode(".", $dnsHost); $bottom_host_name = $host_names[count($host_names)-2] . "." . $host_names[count($host_names)-1]; curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'X-Auth-Email: '.$this->_dnsUser.'', 'X-Auth-Key: '.$this->_dnsPass.'', 'Content-Type: application/json' )); // Get zone ID $getZoneId = "https://{$dnsServer}/client/v4/zones/?name={$bottom_host_name}"; curl_setopt($ch, CURLOPT_URL, $getZoneId); $output = json_decode(curl_exec($ch)); $zone = $output->result[0]->id; if ($zone){ // If zone ID was found get host ID $getHostId = "https://{$dnsServer}/client/v4/zones/{$zone}/dns_records?name={$this->_dnsHost}"; curl_setopt($ch, CURLOPT_URL, $getHostId); $output = json_decode(curl_exec($ch)); $host = $output->result[0]->id; if ($host){ // If host ID was found update host $hostData = array( "content" => "{$this->_dnsIP}", "type" => "A", "name" => "{$this->_dnsHost}", "proxiable" => false, "proxied" => false ); $data_json = json_encode($hostData); $updateHostId = "https://{$dnsServer}/client/v4/zones/{$zone}/dns_records/{$host}"; curl_setopt($ch, CURLOPT_URL, $updateHostId); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT'); curl_setopt($ch, CURLOPT_POSTFIELDS,$data_json); } } break;
Original
case 'cloudflare': // recieve multipe results $data = explode("\n",$data); $lines = count($data)-1; // loop over the lines for ($pos=0; ($successful_update || $pos == 0) && $pos < $lines; $pos++){ $resp = $data[$pos]; if (preg_match('/UAUTH/i', $resp)) { $status = "DynDNS: The username specified is not authorized to update this hostname and domain."; } else if (preg_match('/NOHOST/i', $resp)) { $status = "DynDNS: No valid FQDN (fully qualified domain name) was specified"; } else if (preg_match('/INVLDHST/i', $resp)) { $status = "DynDNS: An invalid hostname was specified. This may be due to the fact the hostname has not been created in the system. Creating new host names via clients is not supported."; } else if (preg_match('/INVLDIP/i', $resp)) { $status = "DynDNS: The IP address given is not valid."; } else if (preg_match('/DUPHST/i', $resp)) { $status = "DynDNS: Duplicate values exist for a record. Only single values for records are supported currently."; } else if (preg_match('/NOUPDATE/i', $resp)) { $status = "DynDNS: No changes made to the hostname (".strtok($resp,' ')."). Continual updates with no changes lead to blocked clients."; $successful_update = true; //success if it is the same so that it saves } else if (preg_match('/OK/i', $resp)) { $status = "DynDNS: (Success) (".strtok($resp,' ').") IP Address for Changed Successfully!"; $successful_update = true; } else { $status = "DynDNS: (Unknown Response)"; log_error("DynDNS: PAYLOAD: {$resp}"); $this->_debug($resp); } log_error($status); } break;
Patch
case 'cloudflare': $output = json_decode($data); if ($output->result->content === $this->_dnsIP){ $status = "DynDNS: (Success) {$this->_dnsHost} updated to {$this->_dnsIP}"; $successful_update = true; } elseif ($output->errors[0]->code === 9103){ $status = "DynDNS ({$this->_dnsHost}): ERROR - Invalid Credentials! Don't forget to use API Key for password field with CloudFlare."; } elseif (($output->success) && (!$output->result[0]->id)) { $status = "DynDNS ({$this->_dnsHost}): ERROR - Zone or Host ID was not found, check your hostname."; } else { $status = "DynDNS ({$this->_dnsHost}): UNKNOWN ERROR - {$output->errors[0]->message}"; log_error("DynDNS ({$this->_dnsHost}): PAYLOAD: {$data}"); } break;
-
Thank you det0nat3.
That patch pretty much covers all that I was looking for.
-
wonders what's wrong with diff …
-
Is this still broken in 2.2.3?
Shouldn't it be possible to use a "custom" dyndns entry instead of the api?
Edit:
I used @__det0nat3__ solution, thanks! -
Thank you det0nat3!!!
n1!
I'm on 2.2.3-RELEASE (amd64) and still not working as is but your fix did the trick!
Would be great if it was in 2.2.4 ;)
-
Just updated to 2.2.4 and now getting this error:
/rc.dyndns.update: DynDNS: PAYLOAD: E_INVLDINPUT
DynDNS was previously working flawlessly after following this thread, any suggestions? Do I just redo the edit?
-
Just updated to 2.2.4 and now getting this error:
/rc.dyndns.update: DynDNS: PAYLOAD: E_INVLDINPUT
DynDNS was previously working flawlessly after following this thread, any suggestions? Do I just redo the edit?
Yeah, just happend to me too, seems like the update did an overwrite of the file.
Is there an ETA of a fix for this? -
Any manual patches need to be re-applied after upgrading the OS. https://github.com/pfsense/pfsense/pull/1916
You can use this URL with System Patches package: https://github.com/doktornotor/pfsense/commit/9aa75e0d5e6b75c0e138408a80ce9615b4136391.patch
-
Has anyone had any luck on the 2.2.5 update? Aparently CloudFlare DNS was fixed.
When I try this I get "Invalid HostName" - I have tried the zone ID, hostname but no luck.
-
Has anyone had any luck on the 2.2.5 update? Aparently CloudFlare DNS was fixed.
When I try this I get "Invalid HostName" - I have tried the zone ID, hostname but no luck.
On 2.2.5, Works no problem for me as of right now. I used the API. Never had it set up before. Got to this article when using my password returned 0.0.0.0
use your CloudFlare API
-
On 2.2.5, Works no problem for me as of right now.
use your CloudFlare API
To add onto what humungus said about the API. On 2.2.5 when trying to setup a DDNS entry with CloudFlare instead of entering in the password for your CloudFlare account put in the API Key from the CloudFlare My Account page.
-
Yep I have been using the API key as password - Not getting any luck.
If I wrongly set my username or password fields I get an authentication failure - So these details are correct.
The issue I am having is I am confused about what to put for the hostname - neither the domain name, zone ID or host ID seem to work, I get the error below:
- Zone or Host ID was not found, check your hostname.
-
I've just set Dymanic DNS up with Cloudflare and it seems to work fine, the updated IP address does take a few minutes to show on Cloudflare however, which initially lead me to believe Dymanic DNS wasn't working.
Service Type: Cloudflare
Interface to monitor: LAN
Hostname: office.mydomain.com
MX: Leave blank
Wildcard: No wildcard
Username: Email address used to log in to my Cloudflare account
Password: Your Cloudflare API keyObviously ensure the A record you want to update already exists for the domain in question on your Cloudflare account.
-
Yep exactly same settings as I have set but still no luck after enabling it again.
I get the error: "php-fpm[40812]: /services_dyndns_edit.php: DynDNS (removed domain name): ERROR - Zone or Host ID was not found, check your hostname."
But the custom script I has setup works fine…
If you need to see the custom script let me know - I found it on this forum to get cloudflare working.
-
So here is the latest script in PHP I use to get this working without needing to mess with pfsense = it will always work. You need to serve it from the server in pfsense (don't think 2.3 does php anymore though) or a local server in your network. If you don't have a local server but an external, you can simply modify it to put it on an external server to use the connecting IP to update the DNS record. First off, you need to know your records and their respective default data such as record ID, easiest is to just curl the data:
curl https://www.cloudflare.com/api_json.html -d 'a=rec_load_all' -d 'tkn=thiiissiiisssyoouuuuurAPIkeeeeey' -d 'email=admin@example.com' -d 'z=example.com'
Now as hinted earlier in this post, when using Cloudflare as a CDN for your websites as I do, I have a minimum of 1 dynamic origin record going to my pfSense (which is the one I'm interested in updating) and 1 public record that routes my website through the CDN platform. You don't want your dynamic IP to go through Cloudflare and need to make sure you don't activate it and need to use service-mode 0 for that record every time you update it. If you don't you'll break stuff such as OpenVPN as the Cloudflare proxies are for web traffic only. In the event where I need to offroute from the CDN I also update my www record as well, but keep it in service mode 1 so it still goes through the CDN.
//////////////////////////////////////start fetching IP $url = "http://whatismyip.akamai.com/"; $headers = array( "GET HTTP/1.1", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.6,en;q=0.5", "Cache-Control: no-cache", "Pragma: no-cache", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:44.0) Gecko/20100101 Firefox/44.0" ); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return page in string curl_setopt($ch, CURLOPT_ENCODING , "gzip"); curl_setopt($ch, CURLOPT_TIMEOUT, 60); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects $data = curl_exec($ch); if (curl_errno($ch)) { print "Error: " . curl_error($ch); } else { curl_close($ch); } ///////////////////////////////////////end fetching IP //everyhing above can be removed if you put the script on an external server and change $data to $_SERVER['REMOTE_ADDR'] //set up the array $request = array(); //the action you are taking $request['a'] = 'rec_edit'; //your API key $request['tkn'] = 'add_your_tkn_value_here'; //your zone, example.com $request['z'] = 'add_your_zone_value_here'; //admin@example.com $request['email'] = 'add_your_email_value_here'; //the rr-id derived from the CURL command $request['id'] = 'add_your_rr_id_value_here'; //the record type you are setting $request['type'] = 'A'; //the value of the record e.g. a subdomain such as www $request['name'] = 'add_your_rr_name_value_here'; //this is your new IP, if you want to put the script on an external server //change $data to $_SERVER['REMOTE_ADDR'], make sure that the request isn't proxied $request['content'] = $data; //this means you will stick to bypass, so you don't accidently turn on the CDN $request['service_mode'] = '0'; //set the time to live of the record as preferred, keep it short so you don't need to wait to be able to //reconnect any of your services such a website proxied through Cloudflare or OpenVPN in the event of a new IP $request['ttl'] = '120'; //the actual API call to update the data per input above $response = @json_decode(file_get_contents('https://www.cloudflare.com/api_json.html?' . http_build_query($request)), true); if(!$response || $response['result'] != 'success'){ print 'Failed to update DNS. :-('; var_dump($response); } else{ echo "Successfully updated!"; } //this next call is for my www record, notice "service mode" is missing, which means it will default to 1 and keep it in proxied mode $request = array(); $request['a'] = 'rec_edit'; $request['tkn'] = 'add_your_tkn_value_here'; $request['z'] = 'add_your_zone_value_here'; $request['email'] = 'add_your_email_value_here'; $request['id'] = 'add_your_rr_id_value_here'; $request['type'] = 'A'; $request['name'] = 'add_your_rr_name_value_here'; $request['content'] = $data; $request['ttl'] = '120'; $response = @json_decode(file_get_contents('https://www.cloudflare.com/api_json.html?' . http_build_query($request)), true); if(!$response || $response['result'] != 'success'){ print 'Failed to update DNS. :-('; var_dump($response); } else{ echo "Successfully updated!"; } echo " ".$data; ?>
Now to get this to work with pfSense you need to set your Dynamic DNS settings to Service Type: Custom and then set the update URL further down the page. Obviously make sure that WAN is the interface to monitor.
-
As Cloudflare is deprecating their v1 API here is my updated code for v4.
First get the relevant zone id's with curl:
curl -X GET 'https://api.cloudflare.com/client/v4/zones' \ -H 'X-Auth-Email: 'mail@example.com \ -H 'X-Auth-Key: 'API-key \ -H 'Content-Type: application/json'
Use the zone ID from the curl command above to get all the records for the zone, make note of the record id's.
curl -X GET 'https://api.cloudflare.com/client/v4/zones/<zone_id>/dns_records' \ -H 'X-Auth-Email: 'mail@example.com \ -H 'X-Auth-Key: 'API-key \ -H 'Content-Type: application/json'</zone_id>
Now replace the e-mail, API-key, zone ID + resource id in the URL in the PHP script and you are good to go:
$url = "http://whatismyip.akamai.com/"; $headers = array( "GET HTTP/1.1", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.6,en;q=0.5", "Cache-Control: no-cache", "Pragma: no-cache", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:44.0) Gecko/20100101 Firefox/44.0" ); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return page in string curl_setopt($ch, CURLOPT_ENCODING , "gzip"); curl_setopt($ch, CURLOPT_TIMEOUT, 60); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects $ip = curl_exec($ch); if (curl_errno($ch)) { print "Error: " . curl_error($ch); } else { // Show me the result curl_close($ch); } $headers2=array( 'X-Auth-Email: mail@example.com', 'X-Auth-Key: API-key', 'Content-Type: application/json' ); ///////////////////// $curl = curl_init("https://api.cloudflare.com/client/v4/zones/<zone_id>/dns_records/<record_id>"); $data = array( "type" => "A", "name" => "origin.example.com", "content" => $ip, "proxied"=> false ); $json = json_encode($data); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "PUT"); curl_setopt($curl, CURLOPT_HEADER, false); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_HTTPHEADER, $headers2); curl_setopt($curl, CURLOPT_POSTFIELDS, $json); // Make the REST call, returning the result $response = curl_exec($curl); if (!$response) { die("Connection Failure.n"); } //////////////////////////////////////// ?></record_id></zone_id>