Will do. Thanks

If you want to offload SSL to HAProxy on pfSense and let ACME handle the validation, you can. You do not need to import anything from your other servers, let the ACME package create and request the certificates for you. Even though they already exist, as long as the validation passes it will be OK.

From the look of that last error you do not have the proper settings for DNS validation.

You could instead follow the example at https://forum.netgate.com/topic/90643/let-s-encypt-support/31 and configure it so HAProxy can assist in handling the validation instead of using DNS.

ok. Installed cert-bot on the mailserver and set up that ACME-challenge-dir etc // seems to work now. Thanks so far!

The GUI does not have support for EC keys at this time.

Ok worked fine with HAProxy. I would suggest you to drop the Squid packages since many features are deprecated or not working properly :)
Thank you for all your amazing stuff.

Thanks for the reply, gotcha i thought it was a bug on pfSense will update it the package

edit: solved thank you @jimp

Well I don't know WTF changed, but this problem has automagically resolved itself after a reboot. My guess is some stale routing state somewhere.

The GUI changes in latest ACME package work well, though!

@napsterbater Thanks for the response. I found that post before posing my question here. The issue is that this solution required the installation of a different certificate manager.

What follows is not a complaint but an observation. It is now clear to me that the pfSense Certificate Manager is designed to import and export certificates needed by the router. It's a great router. We really shouldn't need it to be a CA as well.

So I installed OpenSSL and used it to recreated all my certs, replacing the old ones as needed. We no longer generate certificates in the pfSense Certificate Manager.

I opened https://redmine.pfsense.org/issues/8682 to add a proper check on the AJAX response for this, and just pushed the change in the latest version of the ACME package. An update should show up shortly. It won't solve your other issue (which I can't reproduce) but it will at least hopefully indicate success or failure properly. It did in my testing, but I couldn't induce a server side failure to test a real-world failure, only a locally faked one.

In the ACME general settings, check Write Certificates, and then have your script check in /conf/acme/ and copy them wherever you want. The script doesn't need to hook on an update, it could check the file modification time or use some other method. Calling it from cron once a day some time after the ACME update would be sufficient.

That would be a matter of configuring plesk to accept updates via the nsupdate method. Using their API is going to be completely different unless it implements that somehow.

nsupdate is an implementation of a specific protocol, RFC2136.

This is a guide on what is required to get a bind server configured to accept updates:

https://www.netgate.com/docs/pfsense/dns/rfc2136-dynamic-dns.html

An alternate strategy might be to set up a bind server like in that link that serves as the master of a dynamic zone with the plesk as the slave but that would preclude managing the zone in plesk which is likely undesirable.

The heavy lifting for this probably needs to be done on the plesk. Instead of accepting updates via their proprietary API they should have a standard method such as RFC2136.

@gertjan - Thanks so much!

Your suggestion led me to a realization. I dropped to the pfSense shell and SSH'd into my Serv-U instance. First thing I noticed (I suspected this would be the case) was that the SSH crypto key wasn't recognized in the host list, and I didn't know if the ACME package forced acceptance the first time it connected so I added the key to the list manually from the shell and then realized something else...

I had originally assumed that the FTP Webroot connection was coming from Let's Encrypt issuing servers, but I remembered reading somewhere in their forums that they don't use FTP at all for challenges, rather this is a function of the pfSense ACME package. I had been using one of my DDNS hostnames for the SFTP setup in the ACME package and I realized that this meant that if the FTP connection was coming from the pfSense box then the DDNS URI would be trying to use reflection to resolve the IP address, which probably wouldn't work. Now that I understood the FTP connection was coming from pfSense and not Let's Encrypt I should change the URI to use the actual LAN IP of the Serv-U host. I did this and the FTP Webroot challenge worked like a charm.

I'm posting this response simply to add to the information here on the configuration of the acme plugin to successfully create/renew a Let'sEncrypt certificate. I had quite a struggle to get it to work and also got a timeout error message.

It seems essential that port 80 is open for the pfSense web interface. Under "System / Advanced / Admin Access" the WebGUI redirect" tickbox must not be ticked, to allow port 80 to be redirected to port 443. If this is ticked, port 80 does not respond and the certbot script to fails.

Under "Services / Acme / Certificate options: Edit" it's easy to miss the small little "+" for RootFolder under Domain SAN list.
Here's the spot!
Domain SAN list small plus
Ensure that the directory is specified.

Lastly, I have created a firewall rule that allows port 80 access to "this firewall" in the WAN rules. I did this before I discovered point 1 above, so it may not be required, but I'm not going to delete my cert now to test it again :-)

Hope that helps future finders of this thread.

i still can't fix it. i am using simple dns plus dns server.

and why i can't input add EC PRIVATE KEY in custom key?

-----BEGIN EC PRIVATE KEY-----
MHQCAQEEIIJtk7xEZdevLY597iBUD59GQra/Uh/hzoQg9DCIAUy9oAcGBSuBBAAK
oUQDQgAE6atp4nEZ1LapCAHdwY6REzljZHUZI0HYH16lCOOGQ+uh
+z1ZmWWXuqSEEThQvpZjESy66GcGWQ==
-----END EC PRIVATE KEY-----

i try change to -----BEGIN PRIVATE KEY----- get log:
getCertificatePSK updating custom key

/usr/local/pkg/acme/acme.sh --renew -d 'xi.net' -d '*.xi.net' --home '/tmp/acme/xi.net/' --accountconf '/tmp/acme/xi.net/accountconf.conf' --force --reloadCmd '/tmp/acme/xi.net/reloadcmd.sh' --yes-I-know-dns-manual-mode-enough-go-ahead-please --dns --ocsp-must-staple --log-level 3 --log '/tmp/acme/xi.net/acme_issuecert.log'

Array
(
[path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
[PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
)
[Sun Jun 24 02:27:14 CST 2018] Renew: 'xi.net'
[Sun Jun 24 02:27:18 CST 2018] Multi domain='DNS:xi .net,DNS:*.xi.net'
unable to load Private Key
34380776392:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:1200:
34380776392:error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:374:Type=X509_ALGOR
34380776392:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/asn1/tasn_dec.c:700:Field=pkeyalg, Type=PKCS8_PRIV_KEY_INFO
34380776392:error:0907B00D:PEM routines:PEM_READ_BIO_PRIVATEKEY:ASN1 lib:/builder/ce-243/tmp/FreeBSD-src/crypto/openssl/crypto/pem/pem_pkey.c:142:
[Sun Jun 24 02:27:18 CST 2018] Create CSR error.
[Sun Jun 24 02:27:18 CST 2018] Please check log file for more details: /tmp/acme/xiaoyu.net/acme_issuecert.log
[Sun Jun 24 02:27:18 CST 2018] The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.

@sgw if you've got problem importing certs or CAs into things like switches etc. make sure which format they need. Some won't like normal PEM style format and want sth like PKCS8 or PKCS12 format. Others want key+cert in the same file. :)

Thanks for the reply

well-known/acme-challenge/my_token at  http://www.mydomain.com/ did not give back the token.

do you mean a token should be returned after the colon in the following example

errordetail='Invalid response from http://www.mydomain.com/.well-known/acme-challenge/F-diHXNvud92akJo7Va8450ZS-6MHt23A9n2KjEBBsFc: '

web root file not present

No file existed at /tmp/haproxy_chroot/.well-known/acme-challenge

Is this an authorization issue

error='"error":{"type":"urn:acme:error:unauthorized"

You will have to use a DNS-based validation method. ACME requires port 80. TLS validation on port 443 is also no longer supported.

Why on Earth would you do it that way vs just handling ACME on the server? If the certificate has nothing to do with pfSense, and the proxy or web server is capable of handling the request, just handle it there with a local ACME client (certbot, acme.sh, dehydrated, etc)

Exposing the firewall web server, adding users to scp keys around… Lots of things here are using insecure practices, or at least less-than-ideal ones.

After further diagnosis, this appears to be an upstream routing or firewall issue.  pfsense cannot ping route53.amazonaws.com and traceroute gets hung up 1 hop away with our ISP–working with them on that.