@-flo-:

Gertjan, can you elaborate on how you set this up?

Never did so myself.
"DNS-Manual" means that you have to go through the same procedure every 90 days or less.

You need a domain name, and you need to have access to "zone information" of this domain name. I guess every registrar gives you this kind of access when you rent a domain name.
So, it's rather easy to set a TXT record with the key info letsencrypt gave you when asking for a certificate or renewing a certificate.
When you add this key, probably using the GUI used by the registrar to administer your domain name, know that you have to wait several minutes or even more, because the zone info has to be synced among at least one other name server that 'hosts' you domain name.
Only when that has been done, you can proceed with the acme interface (pfSense) to ask for a (re) new certificate.

I'm using my own dedicated server, and I'm using my own DNS master server that hosts my domain name (actually more then 10).
acme used by pfSEnse has been set up to "talk" to my DNS server, so it can add these TXT records itself in the zone file (the file with all the info related to a domain name).
This is the so called "nsupdate" method, and is fully automated.

@B-C:

like 8443 - I can create the LE cert for one of these VMs, just not clear on how the VM gets the cert installed to use?
using a Service Desk Plus specifically running on debian.

There is no such thing as a buildin script that copies a certificate (certificate files, or the whole bunch in a 'chained' file) from one device, pfSense, to another device, your server.
The files have to get moved over, the service - the web server - has to be restarted.
It is possible of course, but for your setup you need your script.

When I renew my certificate for my pfsense (pfsense.mynetwork.tld) I also renew for diskstation.mynetwork.tld, printer1.mynetwork.tld printer2.mynetwork.tld, etc. I have to copies the needed  files over to the diskstation, printer1, printer2 etc - most of them do not even have a telnet or ssh access, so scripting is impossible.

Best is to run some letsenscrypt support from these devices, if it is possible.

Thank you, this seems to have done the trick. A little bit of extra work because manual DNS is a pita and the local http server can't bind to port 80 becauase of local running ngix - hence a port forwarding is necessary. Also I didn't think of this thread you've mentioned because some renewals still worked with tls, some others didn't so this made it harder for me to specify an exact error scheme.

Julian

Should that URL be open to the world? I can't reach it on port 80 over IPv4 or IPv6 right now. Perhaps the validation servers at Let's Encrypt also can't reach it?

Since it's a timeout, I would focus on firewall rules or other access rules, maybe even routing upstream, anything that could prevent LE from reaching your web server on port 80. Maybe you have something like pfBlocker filtering access or geoblocking?

Of course this isn't a general discussion thread, my mistake .

Ahh, bummer. Thanks!

Done, thanks: https://redmine.pfsense.org/issues/8211

COMPLETED… TY PiBa

Ah ha! I found the issue! jimp, was HAProxy updated with the upgrade to 2.4.2?

Using the standalone method, I created a backend in the HAProxy UI with a single server bound to localhost on port 8082, with no health checks, or timeout/retry settings.
(I didn't use health checks was because this backend is "down" whenever the acme.sh script isn't running. So I just ignored the overhead of doing a health check.)

The problem here is there is an IPv4 and IPv6 address for localhost, so in the newest version of HAProxy, it actually created two servers even though the UI only had one specified:

backend 0_HTTP_ACME_Standalone_http_ipvANY mode http log global timeout connect 30000 timeout server 30000 retries 3 server pfsense_0 127.0.0.1:8082  server pfsense_1 ::1:8082 

The ACME client only binds to the IPv4 interface using socat:

[2.4.2-RELEASE][root@pfsense]/tmp/acme: sockstat -l46 | grep 8082 root    socat      96563 5  tcp4  *:8082                *:*

(I ran this sockstat command during the execution of certificate creation/renewal and it only ever listens on 127.0.0.1:8082.)

And the nail in the coffin is that with the None option specified in the load balancing section of the backend, it defaults to Round Robin:

@HAProxy:

The load balancing algorithm of a backend is set to roundrobin when no other
algorithm, mode nor option have been set. The algorithm may only be set once
for each backend.

So LE was successfully reaching in to my infrastructure (as I noted with the packet capture) on the 127.0.0.1:8082 server, but when it attempted to reach in again, it would be Round Robin'd to the ::1:8082 server, to which the ACME client wasn't bound. This would then timeout and cause the validation process to fail.

–-

I've adjusted the backend to only listen on 127.0.0.1, not localhost, but I would be more satisfied if I knew the proper knobs to turn such that the HAProxy backend would timeout quickly, and a new request would be issued to the next server in the list. Alas, I will have to figure that out another day.

Thanks for your help, jimp!

–-

PS - I actually tried to pull the HEAD of acme.sh's repo to execute on the pfSense box, but as soon as I saw the output I knew there had to be adjustments made to the source code that made it compatible with pfSense. I gave that up quickly.

I also only ran it from the command line because the ACME UI actually prints out the full command of what it's executing under the covers. I thought it was safe to copy and paste it so I could have control over execution during debugging.  :)

Thanks again

I’m in the middle of switching my DNS servers from Namecheap to Cloudflare, just waiting for the changes to take effect (up to 24 hrs). Plan to try authentication using txt record.

Not sure why you’re seeing multiple A records? I have one A record and a couple of CNAME’s. Maybe something due to the changing DNS servers? I do see a bunch of MX records which seems strange since I’m not running any email on this site. Currently just planning to use it for VPN. Maybe the MX records are just placeholders?

thanks for that informative feedback, will pick up some of your suggestions as soon as I find the time to continue that project.

Edit:

You use SSL-Offloading for all VMs, OK, same as here. My additional wish is to encrypt the traffic from HAproxy to the backends as well, with a separate SSL-cert with long lifetime, ideally also generated/refreshed on pfsense. So the config of these backends is my current issue.

AFAIK the NIST guideline sounded like
"Drop the algorithmic complexity song and dance
No more arbitrary password complexity requirements needing mixtures of upper case letters, symbols and numbers. Like frequent password changes, it’s been shown repeatedly that these types of restrictions often result in worse passwords."
Yes it was about that you can not require a complex password with short length over much more longer password but without any complexity. The length is limited by freeDNS and I am sure in case of limited length complexity always wins  ;)

Nice Jimp!  Quick work, thanks.

I remember posting somewhere on this forum about this ACME package that even a space " " in the name (field) will give a situation were the account key exists, but can't be show or edited anymore …

ACME only updates TXT records, it has nothing to do with dynamic DNS for regular A/AAAA records. The package does not share any code or functionality with dynamic DNS in the base system.

The ACME package only asks for what the acme.sh dnsapi script for Route53 wants (dns_aws.sh) and that is only the AWS Access Key ID and secret.

No love.

As I mentioned, I confirmed  www.foo.com/.well-known/acme-challenge isn't redirecting, but the renewal is still failing.

Oddly, the error.log shows this:

AH00112: Warning: DocumentRoot [/var/lib/letsencrypt/tls_sni_01_page/] does not exist
[Fri Oct 27 09:47:19.831094 2017] [ssl:warn] [pid 35459] AH01906: b0e858dd145cedac69bf9f2ff813bdce.4aff4cc0f637d578fdb7e19834ea33dc.acme.invalid:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[Fri Oct 27 09:47:19.831626 2017] [mpm_prefork:notice] [pid 35459] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured – resuming normal operations
[Fri Oct 27 09:47:19.831636 2017] [core:notice] [pid 35459] AH00094: Command line: '/usr/sbin/apache2'
[Fri Oct 27 09:47:26.617630 2017] [mpm_prefork:notice] [pid 35459] AH00171: Graceful restart requested, doing restart
[Fri Oct 27 09:47:26.657069 2017] [mpm_prefork:notice] [pid 35459] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured – resuming normal operations
[Fri Oct 27 09:47:26.657091 2017] [core:notice] [pid 35459] AH00094: Command line: '/usr/sbin/apache2'

If I can figure out how to force a renewal, I'll test on my other host and see if I can replicate it.

Thank you. And please do not understand me wrong - i'm only want to help community.

What exactly did you enter for the SFTP server? It should be sftp://x.x.x.x not just a bare IP address. See https://doc.pfsense.org/index.php/ACME_package#FTP_Webroot