Cannot reach api server from pfsense
-
For context see topic 158465
Recently I noticed pfsense acme my certificates have stopped renewing and I cannot get past this problem:
The acme script presented me with a TXT record to add to my domain for validation, but each time I ran the renewal script, I received a new TXT record.
You say you're using RCF 2136 style updates, so I changed to that.
The API calls are supposed to be sent to https://api.namecheap.com/xml-response? yet no matter what I do, the pfsense acme script says it doesn't get a response from that server.
If I try manually, I get a response (from the command line in the pfsense gui).
I have tried:
https://api.namecheap.com/xml-response?
https://api.namecheap.com/xml-response
https://api.namecheap.com
api.namecheap.comCan anyone shed some light on this please?
The output is here:
Primary Renewing certificate account: fastzanet server: letsencrypt-staging-2 /usr/local/pkg/acme/acme.sh --issue --domain 'fw.fast.za.net' --dns 'dns_nsupdate' --domain 'fw-1a.fast.za.net' --dns 'dns_nsupdate' --domain 'fw-1b.fast.za.net' --dns 'dns_nsupdate' --home '/tmp/acme/Primary/' --accountconf '/tmp/acme/Primary/accountconf.conf' --force --reloadCmd '/tmp/acme/Primary/reloadcmd.sh' --log-level 3 --log '/tmp/acme/Primary/acme_issuecert.log' Array ( [path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/ [PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/ [NSUPDATE_SERVER] => /tmp/acme/Primary/fw.fast.za.netnsupdate [NSUPDATE_KEYNAME] => [NSUPDATE_KEYALGO] => [NSUPDATE_KEY] => /tmp/acme/Primary/fw.fast.za.netnsupdate [NSUPDATE_ZONE] => ) [Wed Jun 2 17:28:35 SAST 2021] Using CA: https://acme-staging-v02.api.letsencrypt.org/directory [Wed Jun 2 17:28:35 SAST 2021] Multi domain='DNS:fw.fast.za.net,DNS:fw-1a.fast.za.net,DNS:fw-1b.fast.za.net' [Wed Jun 2 17:28:35 SAST 2021] Getting domain auth token for each domain [Wed Jun 2 17:28:42 SAST 2021] Getting webroot for domain='fw.fast.za.net' [Wed Jun 2 17:28:42 SAST 2021] Getting webroot for domain='fw-1a.fast.za.net' [Wed Jun 2 17:28:43 SAST 2021] Getting webroot for domain='fw-1b.fast.za.net' [Wed Jun 2 17:28:43 SAST 2021] Adding txt value: h83cYEE-AQkVS3KgfSLFlUkQU93bM9u-PjHNfl8xQ88 for domain: _acme-challenge.fw.fast.za.net [Wed Jun 2 17:28:43 SAST 2021] adding _acme-challenge.fw.fast.za.net. 60 in txt "h83cYEE-AQkVS3KgfSLFlUkQU93bM9u-PjHNfl8xQ88" ; Communication with 104.219.249.152#53 failed: timed out could not reach any name server [Wed Jun 2 17:29:04 SAST 2021] error updating domain [Wed Jun 2 17:29:04 SAST 2021] Error add txt for domain:_acme-challenge.fw.fast.za.net [Wed Jun 2 17:29:04 SAST 2021] Please check log file for more details: /tmp/acme/Primary/acme_issuecert.log
-
@lifeboy said in Cannot reach api server from pfsense:
fw.fast.za.net
acme wants to commicate with "fw.fast.za.net" (104.219.249.152) to add a zone called "_acme-challenge" and a TXT record.
"fw.fast.za.net" (104.219.249.152) isn't allowing it.
@lifeboy said in Cannot reach api server from pfsense:
You say you're using RCF 2136 style updates, so I changed to that.
You've added the right access credentils ?
-
@gertjan I don't think that is what it's trying to do. It's trying to add a TXT record to that fw.fast.za.net zone at api.namecheap.com which is 104.219.249.152.
fw.fast.za.net is the pfSense server.
-
@lifeboy said in Cannot reach api server from pfsense:
fw.fast.za.net is the pfSense server.
That is : " fw.fast.za.net" has an A record that points to your WAN IPv4 ?
The master name server should be at "104.219.249.152."
edit : that is, there should be some (at least 2) NS records that tell who/where the name servers are.Normally, these should be found like this :
dig fw.fast.za.net NS +short
but there are none ??
Example, One of my domains :
dig test-domaine.fr NS +short ns3.test-domaine.fr. ns1.test-domaine.fr. ns2.test-domaine.fr.
where ns1.test-domaine.fr. is the master.
-
@gertjan said in Cannot reach api server from pfsense:
Normally, these should be found like this :
dig fw.fast.za.net NS +shortbut there are none ??
The NS records are for the domain (fast.za.net), not for the host (A record) fw.
$ dig +short fw.fast.za.net 197.214.119.130 $ dig fw.fast.za.net NS fast.za.net. 2755 IN SOA freedns1.registrar-servers.com. hostmaster.registrar-servers.com. 1622642442 43200 3600 604800 3601 $ dig +short fast.za.net SOA freedns1.registrar-servers.com. hostmaster.registrar-servers.com. 1622642442 43200 3600 604800 3601 $ dig +short freedns1.registrar-servers.com 45.58.122.82 $ dig +short api.namecheap.com 104.219.249.152
So there is a zone fast.za.net at freedns1.registrar-servers.com that has a host fw.fast.za.net.
The error clearly says that the address that is not responding / cannot be reached is the address of namecheap's api server, 104.219.249.152.
But when I try to reach that api server manually from the pfSense hosts, it responds. So either the diagnostic log is completely misleading or something else is causing the script to not a get a response from the api server. -
if I have to summarize these :
namecheap pfsense acme letsencryptI see that there is a $50 "fee".
Changing DNS or even moving the domain name registration elsewhere.edit :
Should you use this : https://github.com/acmesh-official/acme.sh/blob/master/dnsapi/dns_namecheap.sh as a guide line, right ?
This https://github.com/acmesh-official/acme.sh/blob/master/dnsapi/dns_nsupdate.sh is for name servers that support nsupdate (RCF 2136 ) - and I'm not sure name cheap supports that most common DNS update protocol.
-
Specific instructions for Namecheap at docs.netgate.com solves the mystery.
I'm trying it that way now.
-
Ah, oops, I thought you were already using the manual.
-
This doesn't seem good..
"The Namecheap DNS API requires that the client read all records and then write them all back when making any change"
Wouldn't it be easier to just manually create the record via namecheap dns interface? Yeah sucks you would have to do it every 90 days. But depending on the number of records you have - something went wrong and all of them got messed up with the write back of all entries..
edit: I take it you meet their requirements for api access.
edit2: Namecheap is just a registrar - they don't have to handle the dns. You could just move the dns to some other provider that provides better api. I have a few domains with namecheap, but I don't use their dns for all of them. The domain using for acme with namecheap, I use cloudflare dns, which works easy with the acme stuff. Sure don't have to rewrite every record on a simple change of 1 record.
-
@johnpoz, yes indeed. I'm engaging Namecheap about this now.
As to writing only one record manually: Of course it would be easier, but that is where my whole journey started. acme doesn't read the TXT record and then creates a new TXT to add. :-( Frustrating. I would thing once one has validated the domain with a TXT record, it should not have to be changed on a renewal.
-
@lifeboy said in Cannot reach api server from pfsense:
I would thing once one has validated the domain with a TXT record, it should not have to be changed on a renewal.
I believe they create new upon every renewal for security reasons. So yeah every time you go to renew that TXT record needs to be updated with the whatever the new TXT is.
If you set acme to manual mode for dns.. It should present you with the value for the TXT record and you could just manually create the record via the namecheap dns interface, then once it resolves. Have acme validate it.
If your having issues with namecheap - you can just migrate the dns for this domain to cloudflare.. They do have FREE dns.. And have gone through a few renewals with acme using it, only thing I ran into is had to up dns-sleep time from 120 to 180.
-
@johnpoz After changing the settings to Namecheap the certificate renewal ran without a hitch!
Also, I have been in contact with Namecheap and the limitation on their API does not apply anymore.
So finally, two ticks and I can continue with other things
-
limitation for access - or limitation of having to read and write all records for update?
Glad you got it sorted.
-
@lifeboy said in Cannot reach api server from pfsense:
acme doesn't read the TXT record and then creates a new TXT to add
Letenscrypt generates a random 'code' - this will become the content of the TXT record, hand over this content to the acme.sh script - as it asks for it. acme.sh knows how to set it up, as, for example, a DNS TXT record : you have to choose the 'method'. When done - a time wait can be needed know, as DNS slaves have to sync with the DNS master server you changed, it signals Letsencryt that's it's done.
Now, Letenscrypts test the presence of this of this TXT record on any (or all now ?) of your domain's name servers .
If the test == proof that you control the domain name, succeeds, Letsencrypt will cache the result for a week or so : renew you cert the next day, and you'll see there is no DNS TXT hassle any more.
Also : at the end of the acme.sh script, with a positive result, or not, acme.sh will remove the added TXT record, thus leaving no trace in the zone / DNS structure.