ACME with Siteground

Amarand

I had used DNS-manual method before I realized this was a manual thing that required you to create/download a certificate every three months. Not ideal.

So basically, I have a SOHO firewall appliance (Netgate SG-3100) running the latest version of pfsense (2.4.5-RELEASE-p1 (arm)) and have just updated the ACME package to the latest version (0.6.9_2).

I use SiteGround for my DNS, and am familiar with creating TXT records for verification.

My problem is, I have no idea what the ideal set-up/configuration is so that ACME will automatically pull down certificates every three months for Let's Encrypt.

I was fiddling with my production certificate (expired) and that timed out (too many failed attempts). So I deleted all of my previous configuration and created a new entry just for STAGING server while I try to figure it out.

Obviously, if I want something to work automatically, unattended, DNS-manual is not the way to go.

I looked at method "DNS-NSupdate / RFC 2136" but that just throws errors, as I'm likely missing one of the steps. I'm not entirely sure this is the method I want to use.

I don't have a web-site in my SOHO (my actual domain is hosted at SiteGround), so I can't just place a .html verification file, at least I don't think I can with the firewall appliance.
I do have full control over my DNS by signing into SiteGround and using cPanel to go into the Advanced DNS configuration tool.

So what's the best/ideal way to get my certificates auto-renewing properly?

I feel silly asking, as I have decades of experience with computers, networking, web-sites, SSL...but I'm used to a much longer period of time for certificates (one year at least), and I usually process those manually.

So, while I love the idea of free certificates through Let's Encrypt, the auto-renewal isn't entirely intuitive when you're using an appliance.

As an aside, back when I had a CentOS server sitting as a dual-homed firewall and web-server, I was easily able to configure cron/certificates to pull certs (it would check every day), and install them properly. I switched to the Netgate appliance, because I wanted something lower powered, and easier to configure. So far, it's been awesome...but the Let's Encrypt/ACME stuff hasn't been that intuitive. (I realize all of those things are third-party, and free, and you get what you pay for but...I'd like it to kinda "just work.")

I don't mind spending time trying to figure it out, but it might be nice if there were specific directions - with examples - for the ideal method, given my configuration.

Any ideas?

Oh, and I'm not sure if I mentioned this or not, but basically, the reason why I want to use the Let's Encrypt with my SOHO firewall, is so that I can browse to the URL in any browser, without having to use a site exception.

Right now, I can connect to my site using the IP address, and add an exception for that, but it throws errors, and I don't know how long browsers are going to allow for site exceptions, with the way Chrome is dealing with security these days. I like to be able to browse to my firewall using the DNS name, but it (Firefox/Chrome) definitely isn't having that.

Anyway, hope I've given enough information. Please let me know if I've missed anything that would help.

Gertjan

Hi,

@amarand said in ACME with Siteground:

I looked at method "DNS-NSupdate / RFC 2136"

You can't pick the (example) nsupdate method. The chice of method depands complely what your registrar offers you. SiteGround should have manuals/FAQ/etc that inform you how you can access the zone info of your domain. The "how' here is not the classic GUI access (edit : your cPanel) , but something that's called an API.
Depending on the API, you could choose a acme method from here https://github.com/acmesh-official/acme.sh/tree/master/dnsapi

@amarand said in ACME with Siteground:

the auto-renewal isn't entirely intuitive

Actually, it is.

Read this : https://letsencrypt.org/how-it-works/ and understand also : "without human intervention". Not on your side - neither at their side (this means billions of economy).

You'll - that is : acme -be using the API of your registrar. This enables you to use a script on your side (== acme) to connect to a script on their side (the API).
The method used depends on how the API is implemented on the registrar's side. Some authentication has to be sued so the API knows it's YOU (== your script).

DNS-NSupdate / RFC 2136 exists because bind, worlds best know domain name DNS server, implemented it. It's probably the oldest method.

Btw : the manual method is a bare bone method that always works always, as it depends only on you having access to the zone info of your domain name using some GUI.
The issue is that it is : you'll be doing the zone update manually.
Letsencrypt adds to that : every 90 days.

Now : the real question - and I could not find the answer : does SiteGround has some API available ?
I know, using their cPanel stuff, you can obtain a cert. But's that's not what you are looking for.

Edit : example !

Take www.ovh.com : a big europeen registrar / web hoster / datacenter owner / etc etc.
Go to their site : at the bottom you find API - then go to "Explore the API".
Look for "/domain/zone/{zoneName}/record" - click on it.
Now you see what parameters are needed.

Now, check the method fire : the source : https://github.com/acmesh-official/acme.sh/blob/master/dnsapi/dns_ovh.sh
Line 155 : there you see the method being used.
The TTL is hard set to 60 seconds.
The fieldType is also hard coded to TXT, as that is what acme/Letsenscypt uses.

This method need :
Your domain.tld name
The subDomain, which will be "_acme-challenge"
The TXT filed will contain a challenge code to be put into the TXT field. This code is give to the acme script by Letsensrypt. For example : 'bmDWOCHFZRtOOCr_vU-mEfTIqA6i9ib0R3V6-RMF3FE'.

When the script method actually managed to communicate with the API, after a typical 60 to 360 seconds of delay (to be set in acme by you - don't make it to small ! DNS slaves have to synchronise with the now update zone file on the domain master first) Letsencrypt will check if "_acme-challeng.your-domain.tld'" has a TXT record, and if so, if this TXT contains 'bmDWOCHFZRtOOCr_vU-mEfTIqA6i9ib0R3V6-RMF3FE'.
If that's the case, there is proof that the acme script controls the DNS of the domain name : a cert can be handed over, because "you", with a script, proofed you're the owner of the domain.

Amarand

@gertjan

As mentioned, I have used this with a CentOS Linux system many times in the past. I would enter the TXT DNS entry into my domain, and when Let's Encrypt went to verify, the code was there.

With my current system, it's totally different.

There are a few components to this:

I have a Netgate appliance, sitting on the Internet, as a dual-homed firewall. It doesn't act as a web-server, so I can't provide .html verification.
I am able to manually edit my domain and add the TXT entry, which I've done before. Do I need to change this TXT entry every time I want to renew every 90 days?
Why do I need to use an API to talk to my registrar (SiteGround), if I can enter the information myself?

I'm confused about this specific process, how to set-up my Netgate appliance to work with ACME/Let's Encrypt, NOT with how I've made Let's Encrypt work in the past with CentOS Linux and my home server.

I don't understand why ACME/Let's Encrypt needs an API hook into my DNS/SiteGround/registrar.

Gertjan

@amarand said in ACME with Siteground:

I don't understand

Take a mirror, and put yourself in front of it.
Now, ask the guy in the mirror :

Do you want to do this manual procedure every 90 - typically 60 - days ?
If yes : you're ok, you know what to do.
If no : automatize it (or have some one else doing it for you). Check the word Automatic here. Automatic does not mean your registrar and/ir web host can obtain certificates for your domain. That will be them using acme to inject certificates in the web server you use. That's for kids, mums, and grand pa.
A real (see this list again) registrar has an API. The API is not only usefull for DNS matters. It could be used to control everything that ius domain/DNS/mail/server/billing related. See the API example I showed.

When you go for a domain name, a shared web host / dedicated server / what ever, it's not the price neither the GUI or extranet that control these things. Many of use have many of these services and it becomes impossible to click click click click click.
That's where these API kicks in.
Most registrars offer an 'API' type access these days, as the click click click click click is just to tedious.

acme.sh has a manual method, as it is perfect to help understand what happens at what moment.
DNS servers have for dedicated method to update the zone records of a domain, just think about the dyndns concept (I presume you know what that is) : it's identical ! Dyndns changes the IP of an 'A' record, that's all it does.
With acme.sh you're creating TXT records (and after the check is done, they are deleted). This can and should be scripted. That's why Letsencrypot was created in the first place.

See also https://docs.netgate.com/pfsense/en/latest/packages/acme/settings-validation.html

Dyndns : a pfsense article from the past : https://docs.netgate.com/pfsense/en/latest/services/dyndns/rfc2136.html

Amarand

@gertjan

The crux of this is, check the list, if my DNS service provider (SiteGround) is not on the list, there is not currently an automated script that will talk to my DNS provider to perform the regular ritual that proves I own my domain.

As an aside, I did a search for "SiteGround DNS API" and nothing useful came back, so I suspect they don't have one. "SiteGround" is not listed as a script in the acme.sh/dnsapi directory you shared.

So at this point, I am just giving up, and continuing to use a local site exception in my browser for my firewall because this probably isn't going to work, and I've already wasted days trying to figure it out.

Also, your replies are extremely condescending. I've been doing this since the 70's, and I've never received a reply in a forum that wanted to make me give up on a simple technical problem. Bravo.

Gertjan

@amarand said in ACME with Siteground:

if my DNS service provider (SiteGround) is not on the list, .....
I did a search for "SiteGround DNS API" and nothing useful came back, so I suspect they don't have one. "SiteGround" is not listed as a script in the acme.sh/dnsapi directory you shared.

I was hoping that documents, manuals, and other materials in your possession, as you are a client, would mention the access needed for acme.sh (or certbot, or ..... ). See the LE site for details.
Is there not some users / support forum ? You're never the only one asking the same question ?

@amarand said in ACME with Siteground:

and I've already wasted days trying to figure it out.

Days ?
A mail or ticket tothe support would do.
Something like :
Do you support : https://letsencrypt.org/about/
They will confirm that they do .... for their proucts like web servers on hosts, shard hosts etc.

You want something different : a domain name with an accessible (API) DNS service.
That's could be something that SiteGround does not offer ....

edit : [The final answer](site:community.letsencrypt.org SiteGround).

Not really an issue as you can take away your domain name, place it with another registrar, and keep the other services that you have at SiteGround.

@amarand said in ACME with Siteground:

Also, your replies are extremely condescending.

Ah ... the mirror I guess.

I fully understand that you do want something better as manually updating.
But, it's up to you.
And you know now - you just wrote it, what needs to be done to make it work.

Again :

I don't understand why ACME/Let's Encrypt needs an API hook into my DNS/SiteGround/registrar.

when you click-through the manual procedure with acme.sh, create your TXT recods etc, a log is created here /tmp/acme/[acme.sh-acountname]/ a log file.
Have a look at that file, and you see what' being done so LE decides to give you a certificate.

It is a rather complex procedure : if humans have to do this every time, failures can happen quickly.
One might say : Certificates are "security" and complex things. Obtaining one is "as hard".
Why the LE certificates are only valid 90 days is explain : one of the raeasons is : they have to be free of charge.

It all boils down to : if you want to use LE, you have to script it.
That's why the pfSense package comes in.
You'll be needing a domain from a registrar that offers API access - or another type of method that acme.sh accepts.

I didn't meant to be condescending, I just want you to find out the what and especially the why parts. I can't bring you a turn key solution. You can.

Amarand

@gertjan

Going back to your original analogy (DynDNS), why can't it be as simple as place a TXT file in the DNS manually, and then all future/subsequent requests go through automatically? With DynDNS, all it has to do is sign in with your credentials, and say "okay, update my DNS entry with this IP address." That uses a username and password scheme. It also can be a boot-time script or a cron job.

The "proof" that I own the domain should be as simple as placing something ONCE in my DNS, and that "proof" is valid as long as it's there.

Is that not a feature that Let's Encrypt supports? Is the only way to "prove" my domain, to have a SCRIPT place a one-time-use TXT entry via API? Or are there other automated methods?

For example: with web-sites, you can use PKI (public key/private key) to confirm you are who you are, and, of course, you could put a "works forever" key into the DNS as well.

My concern isn't the scripting part - my concern is that the script "must" (?) change a value in my DNS, which isn't something I feel comfortable granting. And even if I did feel comfortable with it, I don't think SiteGround offers a DNS API hook.

So are there other automated methods (obviously I don't want to be doing anything manual more than once) that will work, if I don't have a DNS API with SiteGround, assuming I don't want to leave my current service provider, as I have a pre-paid contract until 2022?

Gertjan

@amarand said in ACME with Siteground:

Is that not a feature that Let's Encrypt supports?

Your quiet close.
It's :
@gertjan said in ACME with Siteground:

The TXT filed will contain a challenge code to be put into the TXT field. This code is give to the acme script by Letsensrypt. For example : 'bmDWOCHFZRtOOCr_vU-mEfTIqA6i9ib0R3V6-RMF3FE'.

This bmD....................RMF3FE thing is generated randomly, and will be unique for every certificate request.
This proofs that you control right now - and not some time X in the past.
Note that, ones this test passed, it stays valid for one week.