@jimp said in Feature Idea: scheduled rule for acme.sh certificate process:
In an ideal world, everyone would use DNS validation instead, but...
I'm completely with you in that. Doing that myself even on my home setup (moved the domain to cloudflare for that - very successfully). So much easier with the added perk of providing your own "dynamic" DNS adresses with your own domain, eliminating one more external dependency.
But the SFTP one is something I have to ponder over, could work for quite a few users that fall under the initial description. Thanks for the reminder
If you want to offload SSL to HAProxy on pfSense and let ACME handle the validation, you can. You do not need to import anything from your other servers, let the ACME package create and request the certificates for you. Even though they already exist, as long as the validation passes it will be OK.
From the look of that last error you do not have the proper settings for DNS validation.
You could instead follow the example at https://forum.netgate.com/topic/90643/let-s-encypt-support/31 and configure it so HAProxy can assist in handling the validation instead of using DNS.
Well I don't know WTF changed, but this problem has automagically resolved itself after a reboot. My guess is some stale routing state somewhere.
The GUI changes in latest ACME package work well, though!
@napsterbater Thanks for the response. I found that post before posing my question here. The issue is that this solution required the installation of a different certificate manager.
What follows is not a complaint but an observation. It is now clear to me that the pfSense Certificate Manager is designed to import and export certificates needed by the router. It's a great router. We really shouldn't need it to be a CA as well.
So I installed OpenSSL and used it to recreated all my certs, replacing the old ones as needed. We no longer generate certificates in the pfSense Certificate Manager.
I opened https://redmine.pfsense.org/issues/8682 to add a proper check on the AJAX response for this, and just pushed the change in the latest version of the ACME package. An update should show up shortly. It won't solve your other issue (which I can't reproduce) but it will at least hopefully indicate success or failure properly. It did in my testing, but I couldn't induce a server side failure to test a real-world failure, only a locally faked one.
In the ACME general settings, check Write Certificates, and then have your script check in /conf/acme/ and copy them wherever you want. The script doesn't need to hook on an update, it could check the file modification time or use some other method. Calling it from cron once a day some time after the ACME update would be sufficient.
That would be a matter of configuring plesk to accept updates via the nsupdate method. Using their API is going to be completely different unless it implements that somehow.
nsupdate is an implementation of a specific protocol, RFC2136.
This is a guide on what is required to get a bind server configured to accept updates:
An alternate strategy might be to set up a bind server like in that link that serves as the master of a dynamic zone with the plesk as the slave but that would preclude managing the zone in plesk which is likely undesirable.
The heavy lifting for this probably needs to be done on the plesk. Instead of accepting updates via their proprietary API they should have a standard method such as RFC2136.
@gertjan - Thanks so much!
Your suggestion led me to a realization. I dropped to the pfSense shell and SSH'd into my Serv-U instance. First thing I noticed (I suspected this would be the case) was that the SSH crypto key wasn't recognized in the host list, and I didn't know if the ACME package forced acceptance the first time it connected so I added the key to the list manually from the shell and then realized something else...
I had originally assumed that the FTP Webroot connection was coming from Let's Encrypt issuing servers, but I remembered reading somewhere in their forums that they don't use FTP at all for challenges, rather this is a function of the pfSense ACME package. I had been using one of my DDNS hostnames for the SFTP setup in the ACME package and I realized that this meant that if the FTP connection was coming from the pfSense box then the DDNS URI would be trying to use reflection to resolve the IP address, which probably wouldn't work. Now that I understood the FTP connection was coming from pfSense and not Let's Encrypt I should change the URI to use the actual LAN IP of the Serv-U host. I did this and the FTP Webroot challenge worked like a charm.
I'm posting this response simply to add to the information here on the configuration of the acme plugin to successfully create/renew a Let'sEncrypt certificate. I had quite a struggle to get it to work and also got a timeout error message.
It seems essential that port 80 is open for the pfSense web interface. Under "System / Advanced / Admin Access" the WebGUI redirect" tickbox must not be ticked, to allow port 80 to be redirected to port 443. If this is ticked, port 80 does not respond and the certbot script to fails.
Under "Services / Acme / Certificate options: Edit" it's easy to miss the small little "+" for RootFolder under Domain SAN list.
Here's the spot!
Ensure that the directory is specified.
Lastly, I have created a firewall rule that allows port 80 access to "this firewall" in the WAN rules. I did this before I discovered point 1 above, so it may not be required, but I'm not going to delete my cert now to test it again :-)
Hope that helps future finders of this thread.
i still can't fix it. i am using simple dns plus dns server.
and why i can't input add EC PRIVATE KEY in custom key?
-----BEGIN EC PRIVATE KEY-----
-----END EC PRIVATE KEY-----
i try change to -----BEGIN PRIVATE KEY----- get log:
getCertificatePSK updating custom key
/usr/local/pkg/acme/acme.sh --renew -d 'xi.net' -d '*.xi.net' --home '/tmp/acme/xi.net/' --accountconf '/tmp/acme/xi.net/accountconf.conf' --force --reloadCmd '/tmp/acme/xi.net/reloadcmd.sh' --yes-I-know-dns-manual-mode-enough-go-ahead-please --dns --ocsp-must-staple --log-level 3 --log '/tmp/acme/xi.net/acme_issuecert.log'
[path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
[PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
[Sun Jun 24 02:27:14 CST 2018] Renew: 'xi.net'
[Sun Jun 24 02:27:18 CST 2018] Multi domain='DNS:xi .net,DNS:*.xi.net'
unable to load Private Key
34380776392:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:1200:
34380776392:error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:374:Type=X509_ALGOR
34380776392:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:700:Field=pkeyalg, Type=PKCS8_PRIV_KEY_INFO
34380776392:error:0907B00D:PEM routines:PEM_READ_BIO_PRIVATEKEY:ASN1 lib:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/pem/pem_pkey.c:142:
[Sun Jun 24 02:27:18 CST 2018] Create CSR error.
[Sun Jun 24 02:27:18 CST 2018] Please check log file for more details: /tmp/acme/xiaoyu.net/acme_issuecert.log
[Sun Jun 24 02:27:18 CST 2018] The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.
I was able to make it work by adding only one SNI: ".example.com". If both "example.com" and ".example.com" are present in the list, it will fail. I guess this is related to the fact that verification records are to be added and removed on the same domain for both, confusing the client.
I ended up doing this because this is what it was suggested in the the blog post I read. I hope docs will be updated to include clean info on how to deal with this, so it would save others time.
@sgw if you've got problem importing certs or CAs into things like switches etc. make sure which format they need. Some won't like normal PEM style format and want sth like PKCS8 or PKCS12 format. Others want key+cert in the same file. :)
Thanks for the reply
well-known/acme-challenge/my_token at http://www.mydomain.com/ did not give back the token.
do you mean a token should be returned after the colon in the following example
errordetail='Invalid response from http://www.mydomain.com/.well-known/acme-challenge/F-diHXNvud92akJo7Va8450ZS-6MHt23A9n2KjEBBsFc: '
web root file not present
No file existed at /tmp/haproxy_chroot/.well-known/acme-challenge
Is this an authorization issue
Why on Earth would you do it that way vs just handling ACME on the server? If the certificate has nothing to do with pfSense, and the proxy or web server is capable of handling the request, just handle it there with a local ACME client (certbot, acme.sh, dehydrated, etc)
Exposing the firewall web server, adding users to scp keys around… Lots of things here are using insecure practices, or at least less-than-ideal ones.
After further diagnosis, this appears to be an upstream routing or firewall issue. pfsense cannot ping route53.amazonaws.com and traceroute gets hung up 1 hop away with our ISP–working with them on that.
Let's Encrypt certificates are only ever valid for 90 days. The field in the GUI controls how long it will wait to force a renewal, usually it's at 60 days IIRC but you can turn that down. You can't effectively go over 90 since that would mean it never renews.
I'm also considering Google Cloud DNS as a possible service to switch to, and based on the claim below that adding a dns api script should be "easy" and the extensive Google Cloud DNS API, I won't rule out Google Cloud DNS yet. Something I plan on looking into over the next few weeks. I don't know yet whether the Google Cloud DNS api relies on installing certain Google scripts/libraries which may or may not be feasible to run on pfSense machines.
"If your DNS provider is not on the supported list above, you can write your own DNS API script easily. If you do, please consider submitting a Pull Request and contribute it to the project."
That "someone" was me : a certificate for the portal.(mydomane.tld) and another certificate for pfsense.(mydomane.tld) hosted on another NIC.
True, I could have combined these two into one certificate.
These days I simplified maintenance, and I use a (one) wildcard cert.
Btw : not related to acme :
Restarting the GUI is completely harmless - I'm the only "user" anyway.
Restarting the portal does have an impact as explained above.
i found some interesting stuff applying some echo lines on datetimes:
Let's encrypt generated certificate is always 90 days valid
pfsense WebUI "Services/Acme/Certificate options/Certificate renewal after" option does not affect certificate lifetime generated by Let's encrypt. It does affect acme_command.sh;
Even a 1 day certificate is valid for 90 days but the option set "Certificate renewal after" correctly set the end date checked by acme_command.sh. So i trust that it could do a good job within 90 days time frame. Any value grater than 90 would let you drop in an unmanged time frame where your certificate is outdated but the script things "Renewal number of days not yet reached".
I would suggest a bug fix in pfsense UI to discard bad values set up in certificate edit page and help users.
You should consider the second gap: since cron job run once a day, you may run the job just 1 hour before a certificate may ends, then you have to wait next job 24 later to get an updated certificate; in the case a webserver's certificate you can get users warned by browser security features for 23/24 hours.
We will plan to examine better the code and patch it with such as a provision feature to issue a new certificate if it will be replaced soon
Easy as we speak
just adding the following line in acme.inc it is possible to renew certificates on the edge of 24 hours
$nextrenewalex = $nextrenewal->sub(new \DateInterval('PT24H'));
in the function issue_certificate right after:
$nextrenewal = $lastrenewal->add(new \DateInterval('P'.$renewafterdays.'D'));
With this patch cron job would be more efficent while renewing certificates giving no downtime of services where certificates are applied to
Yes. It requires a real, valid domain name. And using webroot or standalone mode on pfSense requires that the domain name point to your WAN IP address and that your firewall expose port 80 and/or 443 (depending on the mode) to the world, which is not good.
Get a real domain name, pick one of the providers that offers a DNS update method supported by the ACME package (there is a list in the certificate options), and then use that to update. You don't have to publicly expose anything on your firewall for DNS updates.
acme package installed (v0.1.34)
has been replaced ags ago with
acme package installed (v0.2.2)
Packages like acme should be kept on the latest version - no exceptions.
You have probably upgrade and update problems, handle them asap.
edit : oops : first post dates from 24/11/2017 ….
Any, update, a new one came out ;)