Account key registration throwing curl error 52

[benjamin_riggs]

Any idea why my attempts to register new account keys all seem to fail with a curl error 52, indicating no response from the webserver? I can curl the staging server without issue from the command line. From what I can suss from /tmp/acme/_registerkey/acme_issuecert.log, the POST to /acme/new-nonce succeeds, but then the POST to /acme/new-acct fails with error 52.

And, as mentioned here, the GUI silently swallows the error.

Here's the log output:

[Thu Jul 19 16:49:03 CDT 2018] Registering account
[Thu Jul 19 16:49:03 CDT 2018] url='https://acme-staging-v02.api.letsencrypt.org/acme/new-acct'
[Thu Jul 19 16:49:03 CDT 2018] payload='{"termsOfServiceAgreed": true}'
[Thu Jul 19 16:49:03 CDT 2018] Use cached jwk for file: /tmp/acme/_registerkey//ca/acme-staging-v02.api.letsencrypt.org/account.key
[Thu Jul 19 16:49:03 CDT 2018] base64 single line.
[Thu Jul 19 16:49:03 CDT 2018] payload64='ey<snip>V9'
[Thu Jul 19 16:49:03 CDT 2018] _request_retry_times='1'
[Thu Jul 19 16:49:03 CDT 2018] Get nonce. ACME_NEW_NONCE='https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce'
[Thu Jul 19 16:49:03 CDT 2018] HEAD
[Thu Jul 19 16:49:03 CDT 2018] _post_url='https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce'
[Thu Jul 19 16:49:03 CDT 2018] body
[Thu Jul 19 16:49:03 CDT 2018] _postContentType='application/jose+json'
[Thu Jul 19 16:49:03 CDT 2018] curl exists=0
[Thu Jul 19 16:49:04 CDT 2018] wget exists=127
[Thu Jul 19 16:49:04 CDT 2018] _CURL='curl -L --silent --dump-header /tmp/acme/_registerkey//http.header  -g '
[Thu Jul 19 16:49:04 CDT 2018] _ret='0'
[Thu Jul 19 16:49:04 CDT 2018] _headers='HTTP/1.1 204 No Content
Server: nginx
Replay-Nonce: Lq<snip>HA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Thu, 19 Jul 2018 21:49:04 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 19 Jul 2018 21:49:04 GMT
Connection: keep-alive
^M'
[Thu Jul 19 16:49:04 CDT 2018] _CACHED_NONCE='Lq<snip>HA'
[Thu Jul 19 16:49:04 CDT 2018] nonce='Lq<snip>HA'
[Thu Jul 19 16:49:04 CDT 2018] protected='{"nonce": "Lq<snip>HA", "url": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct", "alg": "RS256", "jwk": {"e": "AQAB", "kty": "RSA", "n": "qQ<snip>-M"}}'
[Thu Jul 19 16:49:04 CDT 2018] base64 single line.
[Thu Jul 19 16:49:04 CDT 2018] protected64='ey<snip>X0'
[Thu Jul 19 16:49:04 CDT 2018] base64 single line.
[Thu Jul 19 16:49:04 CDT 2018] _sig_t='NA<snip>ls='
[Thu Jul 19 16:49:04 CDT 2018] sig='NA<snip>ls'
[Thu Jul 19 16:49:04 CDT 2018] body='{"protected": "ey<snip>X0", "payload": "ey<snip>V9", "signature": "NA<snip>ls"}'
[Thu Jul 19 16:49:04 CDT 2018] POST
[Thu Jul 19 16:49:04 CDT 2018] _post_url='https://acme-staging-v02.api.letsencrypt.org/acme/new-acct'
[Thu Jul 19 16:49:04 CDT 2018] body='{"protected": "ey<snip>X0", "payload": "ey<snip>V9", "signature": "NA<snip>ls"}'
[Thu Jul 19 16:49:04 CDT 2018] _postContentType='application/jose+json'
[Thu Jul 19 16:49:04 CDT 2018] Http already initialized.
[Thu Jul 19 16:49:04 CDT 2018] _CURL='curl -L --silent --dump-header /tmp/acme/_registerkey//http.header  -g '
[Thu Jul 19 16:51:04 CDT 2018] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 52
[Thu Jul 19 16:51:04 CDT 2018] _ret='52'
[Thu Jul 19 16:51:04 CDT 2018] original
[Thu Jul 19 16:51:04 CDT 2018] responseHeaders='HTTP/1.1 100 Continue
Expires: Thu, 19 Jul 2018 21:49:04 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
^M'
[Thu Jul 19 16:51:04 CDT 2018] response
[Thu Jul 19 16:51:04 CDT 2018] code='100'
[Thu Jul 19 16:51:04 CDT 2018] Register account Error: 

[jimp]

I can't seem to reproduce that issue here. I can make a new key and register it against either the v1 or v2 servers and do not receive an error.

What version of pfSense are you running, and what version of the ACME package?

[benjamin_riggs]

I'm on the latest 2.4.3_1.

My suspicion is the 100 Continue response is throwing a wrench in things, but I have no idea why, much less a cause.

[jimp]

No, I see a 100 continue as well and then it keeps going. There may be something about the parameters you are submitting it doesn't like, perhaps. The cURL error you get means the server sent back an empty response which is unusual.

[benjamin_riggs]

I'm thoroughly baffled by this. I'm just using the web form and not entering any unusual data, from what I can tell. From my cursory look at the crypto generated it seems reasonable.

Any other thoughts at where I can even begin to troubleshoot?

[jimp]

Is this the only one you've tried? How many times have you tried it?

I wonder if maybe you've hit one of their rate limits. Though the staging servers shouldn't have rate limits.

Try using the v1 servers and not v2, see what happens. If that fails, try leaving off the e-mail address.

[benjamin_riggs]

I've tried all 4 servers to no avail. While I have tried multiple times, it's never been more than 10 a day, and only that many on the staging servers.

I am behind a NAT, but nothing else here should be talking to the servers, and certainly not all 4 of them.

Removing or changing the email address doesn't seem to change anything.

I have to think it's something with my config, but don't have a clue what could be affecting this.

[jimp]

And you are certain whatever is ahead of you doesn't interfere with web requests in any way? No proxy or MITM interception type thing going on?

Looking at the portion of the log you posted vs what I get, they are identical (Except for the randomized parts) until yours hits the cURL error.

[benjamin_riggs]

My gateway is load-balanced between two DSL modems, I have a NAT, DDNS, and incoming forwarding of http/s & ssh to a server. Since the traffic is coming from the router and not the LAN, maybe the firewall is treating it differently in some way? I'm not sure why the first curl would work but not the second.

[jimp]

You mentioned Load Balancing. Is it possible your first request goes out WAN1, then the second out WAN2? ACME may drop that since the request didn't come from the same address.

[benjamin_riggs]

I have the 'sticky connections' flag set in the configuration, so I think that should prevent this scenario. I even turned the 'tracking timeout' up to 120 seconds just to be certain (which matches the curl timeout).

Is there a chance this could be failing because the traffic is coming from the router and not the LAN?

[benjamin_riggs]

Does the plugin run as a user on the OS? If so, I would assume all the traffic from the router itself would just use the default OS route, which is simply one of the pppoe interfaces. I do see that both my pppoe interfaces have the same gateway (despite wildly different IPs), but netstat -rn only shows pppoe0 in use.

Incidentally, in the Status / Gateways /Gateways GUI, I think both interfaces having the same Monitor IP (by virtue of both having the same gateway) is causing an incorrect status to be displayed for one of the gateways.

[benjamin_riggs]

Well I don't know WTF changed, but this problem has automagically resolved itself after a reboot. My guess is some stale routing state somewhere.

The GUI changes in latest ACME package work well, though!