SFTP/FTPS error connecting for challenge aquisition
-
I run an apache server on Windows and pfSense in a VMware Workstation shared VM on the same always-on PC. Some time ago I set up the ACME package in pfSense with some "tricks" to get the aquired certs from the pfSense VM to the Windows apache setup. My Windows box uses the Serv-U FTP server and I configured it for standard FTP as well as SFTP on port 22 and FTPS on port 21 (explicit SSL) and port 990 (implicit SSL). I used the Webroot FTP challenge because challenge methods which rely on port 80 access would conflict with my Apache servier on the same port, and I can't use nameserver methods. Serv-U is set up to allow a user named "LetsEncrypt" to access (and write to) the ACME challenge path on my server root. Rather that mess around trying to get pfSense to write the Let's Encrypt certs to the Windows Apache config path on a successful cert issue, I instead configured Windows Scheduler to attempt to FTP any new certs from pfSense every night at about 3am. If there are no new certs available then nothing happens, but if the scheduled task successfully FTPs new certs from pfSense then it overwrites the existing ones on the Windows box and restarts Apache. Athough it may sound complicated, it was working perfectly.
Then a few weeks ago my pfSense configuration got trashed and I had to reinstall pfSense and set up the ACME package again. This time, however, I just can't get the package to connect to my SFTP server. I can't find anything in the log regarding the attempted SFTP connection so I have no idea why it won't connect. I have set all my pfSense port forwarding correctly, and can connect to SFTP and FTPS from the outside world using clients like WinSCP and FTP Voyager, but the ACME package just fails to connect every time using EITHER sftp:// or ftps://.
Does anybody have any idea what is going on? Here is the pfSense WebUI listing of the cert renewal attempt using the Let's Encrypt staging server (indentifying entries redacted)...
=====================================================================
Renewing certificate
account: My ACME Staging Account
server: letsencrypt-staging/usr/local/pkg/acme/acme.sh --issue -d '<my domain>' --home '/tmp/acme/<name>/' --accountconf '/tmp/acme/<name>/accountconf.conf' --force --reloadCmd '/tmp/acme/<name>/reloadcmd.sh' --webroot pfSenseacme --log-level 3 --log '/tmp/acme/<name>/acme_issuecert.log'
Array
(
[path] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
[PATH] => /etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/
[ftpserver] => sftp://xxx.xxx.xxx
[username] => LetsEncrypt
[password] => <pswd>
[folder] => /<my domain>/.well-known/acme-challenge
)
[Sun Jul 8 11:16:46 EDT 2018] Single domain='<my domain>'
[Sun Jul 8 11:16:46 EDT 2018] Getting domain auth token for each domain
[Sun Jul 8 11:16:46 EDT 2018] Getting webroot for domain='<my domain>'
[Sun Jul 8 11:16:46 EDT 2018] Getting new-authz for domain='<my domain>'
[Sun Jul 8 11:16:48 EDT 2018] The new-authz request is ok.
[Sun Jul 8 11:16:48 EDT 2018] Verifying:<my domain>
[Sun Jul 8 11:16:48 EDT 2018] Found domain http api file: /tmp/acme/<name>//httpapi/pfSenseacme.shchallenge_response_put <name>, <my domain>
FOUND domainitemFTP
FTP Attempt Failed: Could not connect with sftp to xxx.xxx.xxx on port 22.
[Sun Jul 8 11:17:51 EDT 2018] Found domain http api file: /tmp/acme/<name>//httpapi/pfSenseacme.sh
[Sun Jul 8 11:17:51 EDT 2018] <my domain>:Verify error:Invalid response from http://<my domain>/.well-known/acme-challenge/MDSSHm91vhvhdIhBRG6koUKqzWO5QnZ359QHBDMaa2I:
[Sun Jul 8 11:17:51 EDT 2018] Please check log file for more details: /tmp/acme/<name>/acme_issuecert.log -
P.S. pfSense version is 2.4.3-RELEASE-p1 (amd64) and ACME package is 0.3_1.
-
P.P.S. I can REALLY access my sftp server from outside (i.e. from smartphone with WiFi turned off), not just via NAT reflection.
-
@greenmr said in SFTP/FTPS error connecting for challenge aquisition:
FTP Attempt Failed: Could not connect with sftp to xxx.xxx.xxx on port 22.
You should be able to SSH into xxx.xxx.xxx using port 22 from pfSense - with the credentials used by acme.
acme is telling you it isn't able to do so.You should be able to see SSH/SFTP logs on the xxx.xxx.xxx device stating the same thing.
Isn't xxx.xxx.xxx a local device / IP (your web server) ?
-
@gertjan - Thanks so much!
Your suggestion led me to a realization. I dropped to the pfSense shell and SSH'd into my Serv-U instance. First thing I noticed (I suspected this would be the case) was that the SSH crypto key wasn't recognized in the host list, and I didn't know if the ACME package forced acceptance the first time it connected so I added the key to the list manually from the shell and then realized something else...
I had originally assumed that the FTP Webroot connection was coming from Let's Encrypt issuing servers, but I remembered reading somewhere in their forums that they don't use FTP at all for challenges, rather this is a function of the pfSense ACME package. I had been using one of my DDNS hostnames for the SFTP setup in the ACME package and I realized that this meant that if the FTP connection was coming from the pfSense box then the DDNS URI would be trying to use reflection to resolve the IP address, which probably wouldn't work. Now that I understood the FTP connection was coming from pfSense and not Let's Encrypt I should change the URI to use the actual LAN IP of the Serv-U host. I did this and the FTP Webroot challenge worked like a charm.
