How to use acme cert generation with carp

lifeboy

I'm running into a problem to renew my LE certificate and due to the bug in acme.sh and dns_maib I am now using the "webroot local folder" method instead of DNS validation. This works fine for the primary (active) server.

I have more than one pfsense machine though, managed by CARP, and to be able to create one certificate that has SAN's for all the servers, I need to set multiple entries in the domain SAN list table. The first two domains (the CARP address, plus the first pfsense machines' ip) get created fine. However, creating the SAN of the second server fails.

Primary
Renewing certificate 
account: fastzanet 
server: letsencrypt-production-2 

/usr/local/pkg/acme/acme.sh  --issue  --domain 'fw.fast.za.net' --webroot pfSenseacme --domain 'fw-1a.fast.za.net' --webroot pfSenseacme --domain 'fw-1b.fast.za.net' --webroot pfSenseacme --home '/tmp/acme/Primary/' --accountconf '/tmp/acme/Primary/accountconf.conf' --force --always-force-new-domain-key --reloadCmd '/tmp/acme/Primary/reloadcmd.sh' --log-level 3 --log '/tmp/acme/Primary/acme_issuecert.log'
Array
(
    [path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
    [PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
    [SSL_CERT_DIR] => /etc/ssl/certs/
    [folder] => /usr/local/www/.well-known/acme-challenge/
)
[Thu Aug 29 19:59:45 SAST 2024] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Thu Aug 29 19:59:45 SAST 2024] Using pre generated key: /tmp/acme/Primary/fw.fast.za.net/fw.fast.za.net.key.next
[Thu Aug 29 19:59:45 SAST 2024] Generate next pre-generate key.
[Thu Aug 29 19:59:45 SAST 2024] Multi domain='DNS:fw.fast.za.net,DNS:fw-1a.fast.za.net,DNS:fw-1b.fast.za.net'
[Thu Aug 29 19:59:51 SAST 2024] Getting webroot for domain='fw.fast.za.net'
[Thu Aug 29 19:59:51 SAST 2024] Getting webroot for domain='fw-1a.fast.za.net'
[Thu Aug 29 19:59:51 SAST 2024] Getting webroot for domain='fw-1b.fast.za.net'
[Thu Aug 29 19:59:51 SAST 2024] fw.fast.za.net is already verified, skip http-01.
[Thu Aug 29 19:59:51 SAST 2024] fw-1a.fast.za.net is already verified, skip http-01.
[Thu Aug 29 19:59:51 SAST 2024] Verifying: fw-1b.fast.za.net
[Thu Aug 29 19:59:51 SAST 2024] Found domain http api file: /tmp/acme/Primary/httpapi/pfSenseacme.sh

challenge_response_put Primary, fw-1b.fast.za.net
FOUND domainitemwebroot
put token at: /usr/local/www/.well-known/acme-challenge//HMFFoIaQyUdcXqR6130WOXOtZFXW__40fpytqUpyYyc
[Thu Aug 29 19:59:52 SAST 2024] Pending, The CA is processing your order, please just wait. (1/30)
[Thu Aug 29 19:59:55 SAST 2024] Found domain http api file: /tmp/acme/Primary/httpapi/pfSenseacme.sh
[Thu Aug 29 19:59:55 SAST 2024] Invalid status, fw-1b.fast.za.net:Verify error detail:197.214.119.132: Invalid response from https://fw-1b.fast.za.net/.well-known/acme-challenge/HMFFoIaQyUdcXqR6130WOXOtZFXW__40fpytqUpyYyc: 404
[Thu Aug 29 19:59:55 SAST 2024] Please check log file for more details: /tmp/acme/Primary/acme_issuecert.log
[Thu Aug 29 19:59:55 SAST 2024] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 3
[Thu Aug 29 19:59:56 SAST 2024] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 3

I have the acme plugin installed and I even created the same acme account keys and certificate entries on the second machine, but the lookup of the local webfolder fails with a curl error 3. (http 404)

I suspect the file doesn't get created by the acme script on the second firewall, because it runs on the first one. Correct?

Is there a way to fix this and get it to work?

lifeboy

I guess one way to possibly get the directory with the validation info to the second (and more) server is to use CARP, which is already in place.

.well-known/acme-challenge/HMFFoIaQyUdcXqR6130WOXOtZFXW__40fpytqUpyYyc

There is no option in the GUI to add a custom item to CARP, but would it be possible in principle? If so, then I supposed it's a matter of adding such an item to the GUI list.

Is there another way maybe?

lifeboy

Would it be a sensible solution to mount the .well-known/acme-challenge directory via NFS on the second server? How would one do that in pfSense without breaking something?

coreybrett

I was able to get this working with my HA setup.

I am using the Duck-DNS method and the cert has SANs for each of the routers. The ACME package is configured on the primary.

I use DNS alias mode and have a few extra CNAMES in my normal DNS zone pointing to the Duck-DNS domains.

lifeboy

@coreybrett Yes, I used a similar method previously with NameCheap, but now I moved our DNS hosting inhouse using Power-Mail-in-a-Box and only realised 3 months down the line that a bug in DNS-MIAB prevent me from updating the certificate with all the SAN's needed. So as this stage I can't use the DNS API method, I must use the Webroot local folder option.

BTW, I have made an attempt to fix the bug and submit the change via a pull-request to Github, but I need to run automated test to do that, which I have not been able to figure out yet...

lifeboy

I have actually figured out how to setup the automated unit tests to run, so my code changes have passed all the required tests and I'm just waiting for the fix to be accepted. Hopefully the patch will be rolled out soon.