Squid SSL serverkey.pem problem

afbkk

Hello,

I'm quite new to pfsense, just playing around before production assignment. I have a weird problem with Squid SSL. I have my own intermediate CA, correctly imported to pfsense together with its parent CA (both private key and public pem). In Squid SSL configuration, intermediate CA was selected. After that, when I try to start Squid, it fails:


FATAL: No valid signing SSL certificate configured for HTTP_port 192.168.202.1:3128
Squid Cache (Version 3.5.23): Terminated abnormally.
CPU Usage: 0.036 seconds = 0.036 user + 0.000 sys
Maximum Resident Size: 52192 KB
Page faults with physical i/o: 0

I figured out it uses /usr/local/etc/squid/serverkey.pem CA as default. So I compared it to my intermediate CA and I found out pfsense somehow generated it with Windows end-of-line chars, eg.:


-----END RSA PRIVATE KEY----------BEGIN CERTIFICATE-----^M
MIIF8DCCA9igAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwgYoxCzAJBgNVBAYTAkNa^M
MQ4wDAYDVQQHDAVQcmFoYTEXMBUGA1UECgwOQUYgQktLLCBzLnIuby4xOTA3BgNV^M

When I replaced serverkey.pem with my intermediate CA without those line endings, Squid works just fine. The problem is
it overwrites it each time the configuration has changed or the server was rebooted. Is this a bug or is there any workaround?

Thanks,
JK

doktornotor

There's no intermediate cert in Squid/SSL. Are you talking about the reverse proxy? Re-import the certificate via GUI without the DOS linebreaks crap.

afbkk

@doktornotor:

Re-import the certificate via GUI without the DOS linebreaks crap.

Actually I have checked the /cf/conf/config.xml to see how the intermediate CA was stored, no ^M there. Just nice and clean cert.

doktornotor

Again, are you talking about the reverse proxy? There is no intermediate anything in Squid SSL proxy. You are looking in the wrong place in config.xml.

afbkk

@doktornotor:

Again, are you talking about the reverse proxy? There is no intermediate anything in Squid SSL proxy. You are looking in the wrong place in config.xml.

No, I'm not talking about reverse proxy, I'm talking about plain proxy server with SSL support. Well, I suppose CA I've selected in Squid configuration is just a "link" to a cert
in Certificate manager. And certs in Certificate manager are stored in config.xml. Or did I get it wrong?

squid.png_thumb

squid_ssl.png_thumb

doktornotor

There is no place to select anything intermediate there.

afbkk

@doktornotor:

There is no place to select anything intermediate there.

Well it's really not a difference whether I choose root CA or intermediate CA, is it?

doktornotor

Yes, but you are confusing the hell out of me talking about a field that does not exist.

The code takes the key and cert "as is" from the cert manager. Re-import the CA(s) and fix the private key in the Cert Manager, or get the feature fixed so that it replaces the linebreaks on import.

https://github.com/pfsense/FreeBSD-ports/blob/devel/www/pfSense-pkg-squid/files/usr/local/pkg/squid.inc#L1123
https://github.com/pfsense/FreeBSD-ports/blob/devel/www/pfSense-pkg-squid/files/usr/local/pkg/squid.inc#L1167

It works just fine with CAs generated on pfSense itself.

doktornotor

To make it really clear and readable, this is what Squid does with the CA/private key:


require_once("certs.inc");
$settings = $config['installedpackages']['squid']['config'][0];
$crt_pk = "/tmp/serverkey.pem";
$srv_cert = lookup_ca($settings["dca"]);
if (base64_decode($srv_cert['prv'])) {
	file_put_contents($crt_pk, base64_decode($srv_cert['prv']) . base64_decode($srv_cert['crt']));
}
printf(file_get_contents($crt_pk));

Now, you can run it from Diagnostics - Command Prompt and see what you get. (The file will end up in /tmp/serverkey.pem so you can download it as well and look at what it produced.)

IOW: This Cert Manager code should replace the DOS line endings \r\n crap with \n. In fact, every textarea input on pfSense should do the same, automatically, without any need to do anything at all with the stuff stored in config.xml for it to be usable. Tired of hacking around it in packages.


$ grep sq_text_area_decode /usr/local/pkg/squid*.inc | wc -l
      20

Absurd. I thought I'd file a bug, but I already did. Rotting there for 1,5 years. https://redmine.pfsense.org/issues/5306

afbkk

OK, so the php code you've supplied produced the very same \r\n as squid does. Now I've tried to reinsert CA in cert manager, with double-checking there are no such nonsense. It didn't help though. Is there any workaround yet? There is already this function there in squid.inc:


/* Handle base64 encoding and linebreaks in textarea configuration fields */
function sq_text_area_decode($text) {
	return preg_replace('/\r\n/', "\n", base64_decode($text));
}

Also, I've checked the bug #5306 you filed some time ago now, you have the link there, but it's not functional anymore:

https://github.com/doktornotor/pfsense-packages/blob/patch-2/config/squid3/34/squid.inc#L85

Thanks!

doktornotor

The link was to what I once again linked above. The code in Squid takes CA cert/key as stored in config.xml, base64_decode()s it (incl. any DOS linebreaks) and plops the key and cert together in one file.

If you want to fix the CA manager, put the preg_replace to the (once again already linked) CA Manager code:


base64_encode(preg_replace('/\r\n/', "\n", $pconfig['cert']));
base64_encode(preg_replace('/\r\n/', "\n", $pconfig['key']));

Otherwise, you might want to use a usable Windows editor, such as Notepad++, to fix linebreaks and paste the cert/key.

afbkk

Thanks.

FYI, I have Ubuntu on my desktop, and all copy-paste has been done from vim :-)

Anyway, thanks for your time.

JK

doktornotor

Huh, was the certificates generated on Windows, or what?

afbkk

Nope, all certificates have been generated by openssl on centos6.

JK

malinmad

Hi,

I know this is a old topic but there was no answer and that is a very frustrating bug that I've also faced… I have 17 PFSENSE firewalls on the field (mostly same version 2.3.2-RELEASE (amd64) ) and this bug appears randomly on some of them.

Saving the "Squid" config will break the certificate file "serverkey.pem" by adding "^M" at the end of each line and squid will refuse to launch...

Recreating the certificate in "certificate manager" does not fix the issue. (Tried copying certdata from notepad++ and other means...)

Here is how I temporarily fixed my issue:
1-I modify the "serverkey.pem" file with vi to remove all "^M" and validate that SQUID can start
2-I run the following command to make the file immutable (cannot be changed) (in SHELL):
chflags schg serverkey.pem
3-When I click "SAVE" in "SQUID CONFIGS", now PFSENSE can't modify the file anymore and can't break SQUID and prevent it from starting...

This is OK for me as my certificate will only expire in 10years... Also, I've also documented where I had to put that workaround... However, a definitive fix would be awesome... Maybe there is something I am missing...

Kind regards,