Automating Certificate imports with letencrypt script
-
Working model. tested and confirm certificate usage. * had to include the key, as Lets encrypt changes the default key on new generation/request.
it requires php5 (CLI) on lets encrypt server.
Remotescript.sh
#!/bin/bash PHP=`which php` # Letencrypt certificate and chain filename (and path) # # CHANGE THESE TO YOUR CERT/CHAIN PATH # CRT=/etc/letsencrypt/live/DNS/fullchain.pem KEY=/etc/letsencrypt/live/DNS/privkey.pem # pfsense server name or IP (using ssk keys for passwordless login) # SERVER=192.168.2.1 # use php base64_encode function to convert certificate and chain # ENCRT=`$PHP -r '$cert = file_get_contents( $argv[1] , true); echo base64_encode("$cert");' $CRT` ENKEY=`$PHP -r '$key = file_get_contents( $argv[1] , true); echo base64_encode("$key");' $KEY` # replace the placeholder string in the pattern template with certificate encoded information. # redirect it out to the sub file so it can be scp to the pfsense server # cat pattern.template | awk '$1=$1' FS="CRTPLACEHOLDER" OFS="$ENCRT" | awk '$1=$1' FS="KEYPLACEHOLDER" OFS="$ENKEY" > pattern.sub # scp the pattern file to the pfsense system # scp pattern.sub $SERVER:/tmp/ # execute sed replace against the config.xml and reload the configuration # ssh $SERVER 'cp /conf/config.xml /tmp/config.xml && sed -f /tmp/pattern.sub < /tmp/config.xml > /conf/config.xml && rm /tmp/config.cache && /etc/rc.restart_webgui'the above code, reads in the cert and chain files, encodes them with php (which makes sense when dealing with the special characters in the certs). we then use a sed template to replace the Placeholder with the base64 encoding. once that pattern.sub is ready, we scp and run it with sed.
create the template file, Change LetsEncrypt to the description name you used to import. I would recommend keeping this unique or understand this script is looking for any'NAME'any with <crt…crt>in the next line.
pattern.template
/.*LetsEncrypt.*/{ N /<crt>.*<\/crt>/{ s//<crt>CRTPLACEHOLDER<\/crt>/ } N /<prv>.*<\/prv>/{ s//<prv>KEYPLACEHOLDER<\/prv>/ P D } }</prv></prv></crt></crt>Of course I know there is some clean up and maybe a few things that could be done more efficiently, so please feel free to improve and optimize. this will get me by for now but would be open to removing php base64_encode function out and use base64 command line (more of a time issue right now to research)… pretty sure the sed -f command could be cleaner too with inplace rewriting on the file.</crt…crt>
-
Just curios why you're using a dedicated Letsencrypt server? There is a an sh script which runs perfectly on pfsense with no dependencies. I used this myself with success. If someone could add the key insertion script into the cron job we'd be all set. Check it out at:
https://github.com/Neilpang/acme.shRegarding the resistance to supporting Lets Encrypt in pfsense I don't see how not using a letsencrypt cert makes anyone safer. If the skeptics are right that illegitimate LE certs are easy to obtain (which I'm not convinced of), that affects us all whether we use LE Certs or not. Fact is the LE CA is trusted (by default) whether they like it or not. Someone correct me if I'm missing something.
-
In my environment it makes sense to use a dedicated server (which I will convert to a docker container). My ssl termination all happens behind pfsense. Don't get me wrong pfsense I know has lots of features, and have use haproxy amoung other services on it prior. However I like using pfsense for true firewall and nat rules (with dedicate resources for that purpose). I use other scale-able systems in the background to ensure services availability. and in my case a dedicated process in order to renew the LE certs. this keeps my systems clean of other dependencies.
I am glad to hear you were able to get it running on pfsense. Like I said pfsense is feature rich and I know a lot of chatter out there on getting this component native or as an install package (and I am for it) Thats what makes pfsense fit for many use cases and setups.
-
Firstly: Thank you @tylerhoadey for the sed/php scripting you shared 8)
Secondly: Thank you @dig1234 for pointing to NeilDang's acme.shhttps://gitlab.com/snippets/26220
Now just my modifications to make it run on the pfSense it self:
- install acme.sh, I used:
curl https://get.acme.sh | sh-
mkdir /root/le and put script.sh and pattern.template (below) in there.
Script.sh added some test for file existence, and pattern.template I somehow found the need to remove the tabs/spaces to make it work on pfSense/FreeBSD's sed. -
chmod +x script.sh ;)
-
I disabled the webConfigurator's use of port 80, but allowed port 80 on the rules to use the standalone mechanism
5) ```
/root/.acme.sh/acme.sh --issue --standalone -d <domainname></domainname>6) The important part here is to now to add this certificate in the Certificate manager with the name atleast containing "LetsEncrypt" Use /root/.acme.sh/<domainname>/fullchain.cer for the certifiacte and the private key would be /root/.acme.sh/<domainname>/<domainname>.key. The string LetsEncrypt in what is used by the sed pattern below to find the right certificate to update. 7)``` /root/le/script.sh <domainname></domainname>My next steps would be to run /root/.acme.sh/.acme.sh –renewAll weekly/monthly from cron, followed by /root/le/script.sh <domainname>script.sh
#!/bin/sh # # Thanks to tylerhoadey # https://forum.pfsense.org/index.php?action=profile;u=282215 PHP=`which php` # Letencrypt certificate and chain filename (and path) # # CHANGE THESE TO YOUR CERT/CHAIN PATH # DNS=$1 CRT=/root/.acme.sh/${DNS}/fullchain.cer KEY=/root/.acme.sh/${DNS}/${DNS}.key # use php base64_encode function to convert certificate and chain # ENCRT=`$PHP -r '$cert = file_get_contents( $argv[1] , true); echo base64_encode("$cert");' $CRT` ENKEY=`$PHP -r '$key = file_get_contents( $argv[1] , true); echo base64_encode("$key");' $KEY` # replace the placeholder string in the pattern template with certificate encoded information. # redirect it out to the sub file so it can be scp to the pfsense server # cat pattern.template | awk '$1=$1' FS="CRTPLACEHOLDER" OFS="$ENCRT" | awk '$1=$1' FS="KEYPLACEHOLDER" OFS="$ENKEY" > pattern.sub # scp the pattern file to the pfsense system # cp pattern.sub /tmp/ # execute sed replace against the config.xml and reload the configuration # cp /conf/config.xml /tmp/config.xml && \ sed -f /tmp/pattern.sub < /tmp/config.xml > /root/le/config.xml-tmp && \ cp /root/le/config.xml-tmp /conf/config.xml && \ rm /tmp/config.cache && /etc/rc.restart_webguipattern.template
/.*LetsEncrypt.*/{ N /<crt.* {<br="">s//<crt>CRTPLACEHOLDER<\/crt>/ } N /<prv.* {<br="">s//<prv>KEYPLACEHOLDER<\/prv>/ P D } }</prv></prv.*></crt></crt.*> ```</domainname></domainname></domainname></domainname> -
I altered your script to work on the PFSense server itself. It makes use of dehydrated (which was formerly known as letsencrypt.sh .. but they updated the name of the repo not yet)
It uses dehydrated to generate new certificates…. The certificate must be manually inserted in the cert manager with a known prefix once .... In my case sslcertificate- <domain>.... and your script automagically replaces the cert and privkey in the config.xml
In this way i am able to use a cron job to generate new certfificates with dehydrated/ldehydrated. And use your tool to renew the certs and privkeys in the confix.xml
https://github.com/lukas2511/dehydrated is the dehydrated generated tool
And your script is the tool that updates the certs on teh PFSense Server itself.
It's available on https://github.com/robinvanleeuwen/update-ssl-certs-on-pfsense.gitThe Certs have to be made manually once with a placeholder in the name for the first time to recognize it to be updatyed : for example sslcertificate- <mydomin>to correctly replace it with the pattern.template.... But it works....
I just editted your files and uploaded it to my own github repo.... If you want your name mentioned or some other kind of info about you, please let me know. I'll be happy to add it since your script was my first initial an best starting point!....
Tnx!</mydomin></domain>
-
Just curious, did you see the new LE pFsense package? Would seem to replace any need for scripts. You can find it under Available Packages..
-
https://github.com/rigtersys/update-certificates-on-other-servers/blob/master/README.md
It's a simple script to generate new certs on pfsense and rsync them to places behind your PFSense / HAProxy setup. I Needed since i placed a catchall on /.well-known/acme-challenge/* on all my IP's in ha-proxy, so i could not generate certrs inside my network with LE-certbot. I now let dehydrated genererate the certs and ssh/rsync them to the server i ran the script from…. It's a bit hacky tool... but maybe someone can get some hints/tips from it... If you have any suggestions how to improve it, let me know! so it can better all suit our needs... :)
-
@hvisage said in Automating Certificate imports with letencrypt script:
enewAll weekly/monthly from cron, followed by
Hi @hvisage thats script still alive? it work on recent vertions?? i'm stating to automate the copy of my acme generated certificates on one pfsense/dns server to other servers web, mail on my network, i'm looking for some solutions scripting for that end...
ofther runn my script it give to me some error related permission over config.xml and config.cache so thats permissions cant be changed by a regular user.
has ben pass 2 years of this post but never is late to make the work fine ... thanks -
I realize this topic is old and about importing LE certs -- but I have the opposite problem. In 2.4.4, if in the Acme package for creating LE certs, in Settings, General Settings, I select the option to "Write ACME certificates to /conf/acme/ in various formats for use by other scripts or daemons which do not integrate with the certificate manager," the certs do NOT get added to /conf/config.xml.
My use case is that I WANT the certs to be in both /conf/config.xml and on the file system. I distribute the LE certs to other devices that need a cert on my LAN.
Does anyone have a suggestion on how I can get LE certs that are created by the Acme package into BOTH the Certificate Manager AND the file system?
Thanks.
-
See https://forum.netgate.com/topic/150603/certs-not-written-to-conf-config-xml-if-write-certificates-selected-in-general-settings/2
Ask yourself this question : if you re install pfSense 'from scratch' and you import your settings using the config.xml file, thinks like certs and CA"s are also imported.
And yes, when the package Acme received a cert from LE after a successful request, it load (replace an older cert with the same "ID") it into the System > Certificate Manager > Certificates list. -
Hello, I just wanted to add to this topic, since I was looking for the same info, and found another possible solution.
Instead of trying to edit the config.xml with a regex/sed, it seems simpler to use the approach featured in this github repo. Use a php script and the built in functions for editing the config.
Check out
https://github.com/zxsecurity/pfsense-import-certificateYou will need to install the script on each firewall, and then upload your certs, and then call the script. For centralized letsencrypt managment this seems like it could be a good approach. I have 30 firewalls and I don't really want each one running acme, I would rather run a central letsencrypt, and deploy the certs to each firewall.
