Issues with Cloudflare Dynamic DNS
-
I have been getting the following error for maybe the last month in the 2.3.3 Development Snapshots as well as the 2.4 Snapshots. I tried tracking the issues down and from what I could find, it seems like it isn't parsing the JSON received back from CloudFlare correctly. When this full JSON string (not contained in the logs unfortunately) is put into a JSON syntax checker, it comes out just fine, but in dyndns.class, json_last_error_msg() returns syntax error right after it tries to get the domain ID. This, of course, prevents the array from even being created, hence the unknown error.
Anyone have any ideas as I have hit a brick wall? My log is below (with hostnames changed).
Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic DNS: updatedns() starting
Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic DNS cloudflare (3g.hostname.org): <currentnicip>extracted from local system.
Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic DNS (3g.hostname.org): running get_failover_interface for opt2. found ppp0
Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic DNS cloudflare (3g.hostname.org): <currentnicip>extracted from local system.
Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic Dns (3g.hostname.org): Current WAN IP: <currentnicip>Cached IP: <oldnicip>Dec 15 01:01:05 php-cgi rc.dyndns.update: DynDns (3g.hostname.org): Dynamic Dns: cacheIP != wan_ip. Updating. Cached IP: <oldnicip>WAN IP: <currentnicip>Dec 15 01:01:05 php-cgi rc.dyndns.update: Dynamic DNS cloudflare (3g.hostname.org): _update() starting.
Dec 15 01:01:06 php-cgi rc.dyndns.update: Dynamic DNS cloudflare (3g.hostname.org): _checkStatus() starting.
Dec 15 01:01:06 php-cgi rc.dyndns.update: phpDynDNS (3g): PAYLOAD: {"result":[{"id":"CLOUDFLAREIDAWHOLEBUNCHOFCHARACTERS","name":"hostname.org","status":"active","paused":false,"type":"full","development_mode":-803711,"name_servers":["cleo.ns.cloudflare.com","roxy.ns.cloudflare.com"],"original_name_servers":["ns-cloud-a1.googledomains.com","ns-cloud-a2.googledomains.com","ns-cloud-a3.googledomains.com","ns-cloud-a4.googledomains.com"],"original_registrar":null,"original_dnshost":null,"modified_on":"2016-12-07T11:13:00.513004Z","created_on":"2016-10-12T20:05:42.101483Z","meta":{"step":4,"wildcard_proxiable":false,"custom_certificate_quota":0,"page_rule_quota":3,"phishing_detected":false,"multiple_railguns_allowed":false},"owner":{"type":"user","id":"THISISMYUSERIDDONTLOOK","email":"MYEMAIL@EMAIL.COM"},"permissions":["#analytics:read","#billing:edit","#billing:read","#cache_purge:edit","#dns_records:edit","#dns_records:read","#lb:edit","#lb:read","#logs:read","#organization:edit","#organization:read","#ss
Dec 15 01:01:06 php-cgi rc.dyndns.update: phpDynDNS (3g): UNKNOWN ERROR - [/quote]</currentnicip></oldnicip></oldnicip></currentnicip></currentnicip></currentnicip> -
Add a little php code that writes the response to a file? Check if thats 'complete' and checks as valid json. And matches at least one of the 3 criteria.. content/errors[0]/success
file_put_contents("/tmp/cf-response.json", $data); -
Maybe your going at it the wrong way? I found this: https://www.google.com/url?sa=t&source=web&rct=j&url=/amp/s/scotthelme.co.uk/replacing-dyndns-cloudflare-ddns/amp/&ved=0ahUKEwizjuKYqJPRAhXFgVQKHW6JC6wQFgg6MAI&usg=AFQjCNGEPBEtX_kd8XCL-4CjUhqcBkNDLQ
Looks like he made a simple solution and who doesn't love cron..
-
Maybe your going at it the wrong way? I found this: https://www.google.com/url?sa=t&source=web&rct=j&url=/amp/s/scotthelme.co.uk/replacing-dyndns-cloudflare-ddns/amp/&ved=0ahUKEwizjuKYqJPRAhXFgVQKHW6JC6wQFgg6MAI&usg=AFQjCNGEPBEtX_kd8XCL-4CjUhqcBkNDLQ
Looks like he made a simple solution and who doesn't love cron..
I would use this, but I might as well fix the functionality in pfSense so people with the same issue can fix it as well.
Add a little php code that writes the response to a file? Check if thats 'complete' and checks as valid json. And matches at least one of the 3 criteria.. content/errors[0]/success
file_put_contents("/tmp/cf-response.json", $data);I'll check this out and get back to you.
EDIT: I found the issue. For some reason, curl_exec($ch) is returning header contents as well and it seems to be messing up the JSON parsing. See below. I'll definitely need to modify the curl query in order for it to only return the body of the response.
HTTP/1.1 200 OK Date: Tue, 27 Dec 2016 11:39:59 GMT Content-Type: application/json Transfer-Encoding: chunked Connection: keep-alive Set-Cookie: __cfduid=SOMEIDTHATYOUSHOULDNTSEE; expires=Wed, 27-Dec-17 11:39:59 GMT; path=/; domain=.cloudflare.com; HttpOnly Expires: Sun, 25 Jan 1981 05:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Strict-Transport-Security: max-age=31536000 X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Server: cloudflare-nginx CF-RAY: NOOOOOPPPPEE-DFW {"result":[{"id":"MYCLOUDFLAREID","name":"hostname.org","status":"active","paused":false,"type":"full","development_mode":-1857245,"name_servers":["cleo.ns.cloudflare.com","roxy.ns.cloudflare.com"],"original_name_servers":["ns-cloud-a1.googledomains.com","ns-cloud-a2.googledomains.com","ns-cloud-a3.googledomains.com","ns-cloud-a4.googledomains.com"],"original_registrar":null,"original_dnshost":null,"modified_on":"2016-12-07T11:13:00.513004Z","created_on":"2016-10-12T20:05:42.101483Z","meta":{"step":4,"wildcard_proxiable":false,"custom_certificate_quota":0,"page_rule_quota":3,"phishing_detected":false,"multiple_railguns_allowed":false},"owner":{"type":"user","id":"MYUSERID","email":"MYEMAILDONTTOUCH"},"permissions":["#analytics:read","#billing:edit","#billing:read","#cache_purge:edit","#dns_records:edit","#dns_records:read","#lb:edit","#lb:read","#logs:read","#organization:edit","#organization:read","#ssl:edit","#ssl:read","#waf:edit","#waf:read","#zone:edit","#zone:read","#zone_settings:edit","#zone_settings:read"],"plan":{"id":"0feeeeeeeeeeeeeeeeeeeeeeeeeeeeee","name":"Free Website","price":0,"currency":"USD","frequency":"","is_subscribed":true,"can_subscribe":false,"legacy_id":"free","legacy_discount":false,"externally_managed":false}}],"result_info":{"page":1,"per_page":20,"total_pages":1,"count":1,"total_count":1},"success":true,"errors":[],"messages":[]} -
The problem was 'probably' introduced with/around this commit: https://github.com/pfsense/pfsense/commit/0e0f580d3905a228f5d0f11629b68d5605e3f071
However there does seem to be code in place to cut off the header returned by curl before entering the _checkStatus function:$response = curl_exec($ch); $header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE); $header = substr($response, 0, $header_size); $data = substr($response, $header_size); $this->_checkStatus($ch, $data, $header);Would be nice to know what header_size and header it finds there.. Perhaps something aint working like it should?
p.s. your running latest snapshot version of pfSense?
-
The problem was 'probably' introduced with/around this commit: https://github.com/pfsense/pfsense/commit/0e0f580d3905a228f5d0f11629b68d5605e3f071
However there does seem to be code in place to cut off the header returned by curl before entering the _checkStatus function:$response = curl_exec($ch); $header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE); $header = substr($response, 0, $header_size); $data = substr($response, $header_size); $this->_checkStatus($ch, $data, $header);Would be nice to know what header_size and header it finds there.. Perhaps something aint working like it should?
p.s. your running latest snapshot version of pfSense?
That would make sense. I'll have it save (or at least log) the header size.
p.s. Yes I am. (2.4.0.b.20161228.0040)
EDIT: Wait, I just realized it fails right before this in the CloudFlare case block. See below.
// This is line 698 curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'X-Auth-Email: '.$this->_dnsUser.'', 'X-Auth-Key: '.$this->_dnsPass.'', 'Content-Type: application/json' )); // Get zone ID $getZoneId = "https://{$dnsServer}/client/v4/zones/?name={$this->_dnsDomain}"; curl_setopt($ch, CURLOPT_URL, $getZoneId); $output = json_decode(curl_exec($ch)); <----- FAILS RIGHT HERE. We need to strip headers before this. $zone = $output->result[0]->id; -
Could you try and apply this patch and see if that fixes the issue? You can use the pfSense SystemPatches package for it with a strip count of 2:
https://github.com/PiBa-NL/pfsense/commit/d21097526668ed010b0cbdab424a250211a151ab.patch -
Could you try and apply this patch and see if that fixes the issue? You can use the pfSense SystemPatches package for it with a strip count of 2:
https://github.com/PiBa-NL/pfsense/commit/d21097526668ed010b0cbdab424a250211a151ab.patchI pretty much did the same thing except I used curl_setopt($ch, CURLOPT_HEADER, 0); instead of the header stripping code. At the end of the CloudFlare block it is set back to 1. It is finally working using these additions. See below. Yours would work the same I imagine, with an exception of a few more lines of code.
curl_setopt($ch, CURLOPT_HEADER, 0); // +++++++++++++++ ADDED LINE 699 curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'X-Auth-Email: '.$this->_dnsUser.'', 'X-Auth-Key: '.$this->_dnsPass.'', 'Content-Type: application/json' )); // Get zone ID $getZoneId = "https://{$dnsServer}/client/v4/zones/?name={$this->_dnsDomain}"; curl_setopt($ch, CURLOPT_URL, $getZoneId); $output = json_decode(curl_exec($ch)); $zone = $output->result[0]->id; if ($zone) { // If zone ID was found get host ID $getHostId = "https://{$dnsServer}/client/v4/zones/{$zone}/dns_records?name={$this->_FQDN}&type={$recordType}"; curl_setopt($ch, CURLOPT_URL, $getHostId); curl_setopt($ch, CURLOPT_HEADER, 0); //+++++++++++++++ ADDED LINE 715 $output = json_decode(curl_exec($ch)); $host = $output->result[0]->id; if ($host) { // If host ID was found update host $hostData = array( "content" => "{$this->_dnsIP}", "type" => "{$recordType}", "name" => "{$this->_dnsHost}" ); $data_json = json_encode($hostData); $updateHostId = "https://{$dnsServer}/client/v4/zones/{$zone}/dns_records/{$host}"; curl_setopt($ch, CURLOPT_HEADER, 1); // +++++++++++++++ ADDED LINE 726 curl_setopt($ch, CURLOPT_URL, $updateHostId); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT'); curl_setopt($ch, CURLOPT_POSTFIELDS, $data_json); } }I could stuff this into a commit request on GitHub if that would be easier. Let me know if you have any questions.
P.S. On an unrelated note, would it be possible to add a checkbox or something to only use the domain part for updates, because the way it is now it is impossible to update the root domain A record (aka, hostname.org 's A record).
-
What about this? I think that might be the better overall option: https://github.com/PiBa-NL/pfsense/commit/15dcf1320c08eb9339eda3e6fdf04599c51694b7.patch
-
What about this? I think that might be the better overall option: https://github.com/PiBa-NL/pfsense/commit/15dcf1320c08eb9339eda3e6fdf04599c51694b7.patch
This looks like it would work. I'll have to apply it later to find out. That patch seems to just remove headers by default unless it's needed, right?
-
Yes that is the general idea behind it, get the headers just before they are 'required', and quite a bit 'cleaner' than my previous solution :D. its even including some cleanup and a bugfix for another provider. If you could verify that it works properly for you as i don't use CloudFlare which seems to be the most curl involved option in that location.. Ill send a pullrequest to get it integrated.
-
Yes that is the general idea behind it, get the headers just before they are 'required', and quite a bit 'cleaner' than my previous solution :D. its even including some cleanup and a bugfix for another provider. If you could verify that it works properly for you as i don't use CloudFlare which seems to be the most curl involved option in that location.. Ill send a pullrequest to get it integrated.
Sorry I didn't test this sooner but this worked! I'll have to make a new thread requesting to add a checkbox or something to only use the domain part of the update. I did fix it myself by adding a condition that if there is only an underscore in the hostname, then only use the domain part.