Acme (Let's Encrypt) w/ High Availability - disable cert sync?

[eduardr]

I configured both fw1 and fw2 with the acme service and later realized fw1 was syncing the fw1 Let's Encrypt cert to fw2, overwriting it in the process.

What is the recommended approach?

1. Disable HA Sync -> Certificate/CA option?
2. Only set up acme package on fw1, have the cert there contain both domains fw1.___ and fw2.___?

[Derelict]

That's what I do. The CN is the name pointing to a CARP VIP with SANs for the names pointing to the interface addresses (fw, fw1, fw2). Do that on the primary and they will sync over.

[jimp]

Here is what I recommend:

• Only install ACME package on the primary node
• Setup a cert with entries for the CARP VIP, and each individual node (As Derelict mentioned)
• Allow the primary to sync this cert over
• Set this cert as the GUI cert on both nodes

The only downsides to this approach are:

• The cert renew will require a manual GUI restart on the secondary
• The cert auth has to happen using a method where the primary can speak on behalf of the secondary, such as a DNS-01 method (nsupdate, any of the DNS providers). Things like standalone or ftp webroot are not practical in this scenario.

[Derelict]

Yes, I am doing all of my ACME using DNS TXT records.

[eduardr]

Thanks for the suggestions, will look at following recommended approach on the next renew cycle.

Overall the process of getting familiar with the acme package setup and getting all the TXT records manually added and configuring firewalls at our 3 locations took most of one day. Got some errors from the let's encrypt service a few times that I couldn't debug (big and not easy to follow acme pkg log files) and wound up having to delete and redo the TXT records and acme cert setup a few times.

Don't know that I'd recommend Let's Encrypt as a $ saving approach since it's difficult and time consuming and requires some care, feeding and attention to make sure things are still working every 90 days the certs expire. The real argument for putting in the effort of automating this process are the security benefits of having short lived certs, assuming those benefits real and worthwhile for "normal" sites.

I'll be curious to see how eagerly admins take up Let's Encrypt over time, or if pragmatism/laziness rules and people will stick with buying regular 1-2 year certs once in a long while and not worry about it the rest of the time.

[Derelict]

Let's see. It's a little tricky because you have to set it up to get the string to place into your DNS.

After some trial and error I settled on:

1. Set up the DNS TXT records at the DNS server (I'm using HE.NET) for _acme-challenge.hostname.example.com. I used a 300 second TTL there so if I hosed it up I could try again shortly. In reality this can be something like 86400 but since it will only be queried at renewal time it really doesn't matter. Let's Encrypt queries fresh every time anyway from what I can tell.  Do this for every CN and SAN. Use a placeholder for the record value like "text."

2. Create the ACME request in pfSense - My HA VM config attached

3. Click Issue

4. Look at the resulting logs and copy/paste the TXT value: attribute values for each FQDN into the DNS server's TXT records for that FQDN.

5. Click Renew.

6. Assign the new certificate to the webgui.


[ypanier]

@jimp:

The only downsides to this approach are:

• The cert renew will require a manual GUI restart on the secondary

Hello,

Is there some tricks to automate the reload of some processes on the slave instance when the certificates changed ? (Gui, HaProxy, VPN etc.. )

I had planned to managed the SSL offloading of some applications in one place but this behaviour have changed my plan. (maybe it is better like that  ?)

Thanks to the developpers of the core project and to the package developpers for their efforts.

Best regards,

[eduardr]

Looks like I'll need to implement the suggestion to have acme service on fw1 only sooner than later. Even with HA Sync -> certificate sync turned off, the fw1 acme cert is still being synced to fw2 causing problems.

[eduardr]

Setting up acme service on fw1 only, and having HA sync the certs to fw2 is working fine now.

A few other hints:

• When adding the TXT records to your DNS, first check that each TXT record is live with these two tools:

https://toolbox.googleapps.com/apps/dig/#TXT/
    $ dig -t txt _acme-challenge.fw.yourcomain.something

Note: it's safest to wait at least as long as the DNS timeout set on the TXT records. For ex. if you set the timeout to 7200, this means 2 hours. Any less than that and the old data may still be cached and cause an Acme verification failure.

Once all the TXT records are live, go ahead and hit the Renew button on the acme cert.

If the records are not properly set or not live yet you will get an error like this:
    Verify error:DNS problem: NXDOMAIN looking up TXT for _acme-challenge.fw.yourcomain.something

If you get this error, you'll have to hit Issue on the cert and delete then add the TXT records with their new values given by the acme service and wait long enough for the old TXT records to be deleted from DNS and the new ones to be added. It will not work to hit Renew once you get the verification error. Hitting Renew will just keep generating the error below and eventually you'll be rate limited by the acme web service and have to wait some time before Issuing a new cert.

Unable to update challenge :: The challenge is not pending