ACME 0.5.8 Breaks Letencrypt webroot local folder setup



  • I went to add a new SSL cert setup for my machines today however I noticed that even though I am selecting webroot local folder and doing the same thing that I have on other certificate entries I am now getting:

    RootFolder: /tmp/haproxy_chroot/haproxywebroot/.well-known/acme-challenge/
    , Key Algorithm: HMAC-MD5, API Endpoint: portal.nexcess.net

    Even though the older entries have only:

    RootFolder: /tmp/haproxy_chroot/haproxywebroot/.well-known/acme-challenge/
    , Key Algorithm: HMAC-MD5

    I checked the config file for pfsense and it appears that the entry for the newly created one and or ones that I copy and save have only a few things in the <item> area versus the 100+ lines that show up in the ones that do still work.

    This seems to cause the renewal process to break and for the certificates to not validate. I am afraid to make any additional changes now even though I have some deadlines coming not to mention this really puts a damper on functionality.

    Is there a way that I can install 0.5.7 to see if it has the same issue and or does anyone have a quick fix that I can apply?

    Suggestions highly appreciated.


  • Rebel Alliance Developer Netgate

    The difference in the settings isn't relevant. The changes there were to make it more efficient. It doesn't store unnecessary empty values now. And the "API Endpoint" bit is a default value from another mode that isn't relevant to webroot.

    So there isn't any detail in your post about what might have led to the failure. Post the logs from the renewal attempt.



  • This is what I get when I go against the Staging Server. Keep in mind this worked fine when I originally set it up earlier this year before I added another host. Also it works fine on the other certificates that I am doing that have not been edited recently. I even deleted this one and set it to only do one domain which has the following output:

    letsEncrypt-Testing
    Renewing certificate
    account: letsEncrypt-Staging
    server: letsencrypt-staging-2

    /usr/local/pkg/acme/acme.sh --issue -d 'testing.mydomain.com' --webroot pfSenseacme --home '/tmp/acme/letsEncrypt-Testing/' --accountconf '/tmp/acme/letsEncrypt-Testing/accountconf.conf' --force --reloadCmd '/tmp/acme/letsEncrypt-Testing/reloadcmd.sh' --log-level 3 --log '/tmp/acme/letsEncrypt-Testing/acme_issuecert.log'

    Array
    (
    [path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
    [PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
    [folder] => /tmp/haproxy_chroot/haproxywebroot/.well-known/acme-challenge/
    )
    [Thu Jul 25 10:38:41 CDT 2019] Single domain='testing.mydomain.com'
    [Thu Jul 25 10:38:41 CDT 2019] Getting domain auth token for each domain
    [Thu Jul 25 10:38:44 CDT 2019] Getting webroot for domain='testing.mydomain.com'
    [Thu Jul 25 10:38:44 CDT 2019] Verifying: testing.mydomain.com
    [Thu Jul 25 10:38:44 CDT 2019] Found domain http api file: /tmp/acme/letsEncrypt-Testing//httpapi/pfSenseacme.sh

    challenge_response_put letsEncrypt-Testing, testing.mydomain.com
    FOUND domainitemwebroot
    put token at: /tmp/haproxy_chroot/haproxywebroot/.well-known/acme-challenge//SxxJRS8OVpD6ewnMa9xhrp72cYPk2Edju5flJRg0O5c
    [Thu Jul 25 10:38:47 CDT 2019] Pending
    [Thu Jul 25 10:38:50 CDT 2019] Pending
    [Thu Jul 25 10:38:52 CDT 2019] Pending
    [Thu Jul 25 10:38:55 CDT 2019] Pending
    [Thu Jul 25 10:38:58 CDT 2019] Found domain http api file: /tmp/acme/letsEncrypt-Testing//httpapi/pfSenseacme.sh
    [Thu Jul 25 10:38:58 CDT 2019] testing.mydomain.com:Verify error:Fetching http://testing.mydomain.com/.well-known/acme-challenge/SxxJRS8OVpD6ewnMa9xhrp72cYPk2Edju5flJRg0O5c: Timeout during connect (likely firewall problem)
    [Thu Jul 25 10:38:58 CDT 2019] Please check log file for more details: /tmp/acme/letsEncrypt-Testing/acme_issuecert.log

    I can post the log that it talks about but its huge so if you have a pertinent thing to look for I will gladly do so.


  • Rebel Alliance Developer Netgate

    @criley said in ACME 0.5.8 Breaks Letencrypt webroot local folder setup:

    [Thu Jul 25 10:38:58 CDT 2019] testing.mydomain.com:Verify error:Fetching http://testing.mydomain.com/.well-known/acme-challenge/SxxJRS8OVpD6ewnMa9xhrp72cYPk2Edju5flJRg0O5c: Timeout during connect (likely firewall problem)

    That's really the important part. It tried to reach your server on port 80 but couldn't. It timed out. So either port 80 is being blocked, or there is a problem in haproxy. (Or, I suppose, a DNS problem with the A record for that domain)



  • The actual domain name is testing.stltoday.com I try to not identify the actual domain on public message boards.


  • Rebel Alliance Developer Netgate

    Looks like port 80 is open on there, so the next check would be to look at haproxy and make sure it's not having a problem running the integration script.

    Still doesn't look like an ACME problem to me.



  • Alright after banging my head against the wall I tried something totally looney and disabled the pfBlockerNG plugin where I am blocking all of Europe, and suddenly it is working again for all six certificates. I then re-enabled pfBlockerNG and once again everything is still working as long as the certificate has not yet expired and has been created.

    My question therefore is does letEncrypt have European Servers that may come into play and how can I add a firewall rule since they don't publish an IP List that allows automatic upgrades to take place. As far as I know I cannot just add a firewall rule with pfsense that allows traffic from anywhere to a directory.

    Thank you for the assistance.


  • Rebel Alliance Developer Netgate

    Let's Encrypt won't publish a list of possible sources as that would let someone game the system to obtain certs for domains they do not own from systems they have compromised in subtle ways (e.g. port forward all LE servers to fakeserver, but let other connections go through to realserver)

    They could reach you from anywhere in the world, there is no way to predict the source. You have to allow connections from anywhere during that timeframe.

    If that bugs you, then switch to a DNS-based method that does not require any inbound access whatsoever.


Log in to reply