HAProxy 1.5 and OCSP

michaelschefczyk

Dear PiBa,

in terms of the certificates, I do bundle the intermediary certificate with the final certificate by just appending it after the final certificate when importing it. If I do that, I do get an A+ in ssltest. Not including the intermediary certificate leads to chain issues and therefore an ssltest of B only. I did not find any way to effectively bundle the intermediary certificate with the root certificate. In fact, my final certificate is marked as "external" despite loading the root certificate also. The recalculate certificate chain button does not change this in any way.

If I do use the plain final certificate without the intermediary certificate, I get the following GUI errors after enabling OCSP stapling and trying to activate the frontend:

Errors found while starting haproxy
[ALERT] 149/233340 (91202) : parsing [/var/etc/haproxy_test/haproxy.cfg:67] : 'bind 192.168.0.2:443' : '/var/etc/haproxy_test/WAN2.pem.ocsp' is present and activates OCSP but it is impossible to compute the OCSP certificate ID (maybe the issuer could not be found)'.
[ALERT] 149/233340 (91202) : Error(s) found in configuration file : /var/etc/haproxy_test/haproxy.cfg
[ALERT] 149/233340 (91202) : Proxy 'WAN2-merged': no SSL certificate specified for bind '192.168.0.2:443' at [/var/etc/haproxy_test/haproxy.cfg:67] (use 'crt').
[ALERT] 149/233340 (91202) : Fatal errors found in configuration.

To me that looks like OCSP stapling does not work without the intermediary certificate. Also under those circumstances, the recalculate certificate chain button does not help.

Regards,

Michael

PiBa

Hi Michael,

If you add a file '/usr/local/www/show_cert.php' to pfSense with the following content:

include_once('certs.inc');
include_once('config.lib.inc');
echo '```
';
foreach($config['ca'] as $cert) {
	echo "\n\nCA          : ".$cert['descr'];
	echo "\nCA   Subject: ".cert_get_subject($cert['crt']);
	echo "\nCA   Issuer : " . cert_get_issuer($cert['crt']);
}
foreach($config['cert'] as $cert) {
	echo "\n\nCert          : ".$cert['descr'];
	echo "\nCert Subject: ".cert_get_subject($cert['crt']);
	echo "\nCert Issuer : " . cert_get_issuer($cert['crt']);
}
?>

And then open that page with a webbrowser, does it show a matching issuer & subject text for 'your' and the 'intermediate' certificate ? So something like this ? (yes all other cas and certs will get output as well and the order is not important..).

But as you can see for each cert the issuer in another CA there is the the exact same text for the subject:

Cert          : *.mydomain.tld
Cert Subject: OU=PositiveSSL Wildcard, OU=Domain Control Validated, CN=*.mydomain.tld
Cert Issuer : ST=Greater Manchester, O=COMODO CA Limited, L=Salford, CN=COMODO RSA Domain Validation Secure Server CA, C=GB

CA          : COMODO RSA Domain Validation Secure Server CA
CA   Subject: ST=Greater Manchester, O=COMODO CA Limited, L=Salford, CN=COMODO RSA Domain Validation Secure Server CA, C=GB
CA   Issuer : ST=Greater Manchester, O=COMODO CA Limited, L=Salford, CN=COMODO RSA Certification Authority, C=GB

CA          : COMODO RSA Certification Authority
CA   Subject: ST=Greater Manchester, O=COMODO CA Limited, L=Salford, CN=COMODO RSA Certification Authority, C=GB
CA   Issuer : OU=AddTrust External TTP Network, O=AddTrust AB, CN=AddTrust External CA Root, C=SE

CA          : AddTrust External CA Root
CA   Subject: OU=AddTrust External TTP Network, O=AddTrust AB, CN=AddTrust External CA Root, C=SE
CA   Issuer : OU=AddTrust External TTP Network, O=AddTrust AB, CN=AddTrust External CA Root, C=SE

michaelschefczyk

Dear PiBa,

thank you very much - the script was very helpful in focusing my attention to the remaining parameters to vary! With this help, I did get OSCP stapling to work!

The first important step is to not have the StartCom root certificate loaded under CA certificates. That may lead to an extra certificate in the .issuer file (once it does get created) and that will be reported negatively by ssltest. I never used to load the CA certificate, but did so in trying to resolve this.

The second important step is to load the StartCom intermediate certificate under CA certificates. Then, the final certificates are not listed as external but rather as dependents of the intermediary certificate. Previously, I did not detect the need to load intermediate certificates separately - now I do.

The third important step is to load the final certificate under certificates, but not bundled with the intermediate certificate.

Regards,

Michael

PiBa

Hi Michael,

Ok, good we got that part resolved, ill remove the root-ca from the .pem written for use by haproxy that should resolve the ssllab 'chain issue' if root is present in the CA list, will be in 0.25..

Now for the actual OCSP functionality.. Do you think it needs to be changed/improved in any way?

p.s. Check that you can see in pfSense system log that the ocsp responses are resolved every hour.. If not reinstall the whole package a 2nd time.

Any other questions/improvements regarding ocsp or haproxy package? Let me know :D.

Regards,
PiBa-NL

michaelschefczyk

Dear PiBa,

My log shows that every full hour, OSCP data gets collected for all HAProxy front ends configured.

From my point of view, the OCSP stapling implementation is completely fine :).

Also the pfSense HAProxy package is very strong at large. One item that should be on the agenda sometime is to compile HAProxy with ALPN-support, I think, because NPN will be replaced by ALPN next year.

Regards,

Michael

PiBa

Hi Michael,
There doesn't appear to be any build-flag to enable ALPN.. It seems like it should be 'auto-detected' by haproxy when the library supports it. Not sure what i can do there..
I think the openssl library used by FreeBSD ports does not yet contain the define 'TLSEXT_TYPE_application_layer_protocol_negotiation'.. http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f017a8f9db3a79f3a3406cf8d493ccd346db691 Even though that commit is 2 years back..

If you have any idea's i'm open to suggestions.
Greets PiBa-NL

josh4trunks

Thanks Michael for blazing the path for me. I also have an SSL cert from StartSSL and just appended it to my domain cert. When I decided to try OCSP today I got the same error. I'll work through what you did tonight hopefully

And thanks as always PiBa for your continued work on this awesome plugin!

michaelschefczyk

Dear PiBa, dear all,

While I did get OCSP to work some months ago, this is no longer so, unfortunately. I am uncertain, if issues in the haproxy-1_5 stable package towards 0.30 are causing this or if it is issues at the certificate provider.

In the qualys ssl test, I see two certificate paths, one leading to the SHA1 CA cert and the other leading the the SHA256 CA cert. While OSCP fails at different levels of the certificate chain, problems are always identical for both paths.

Yesterday, qualys ssl test did yield (for the SHA1 path as a example):

1 Sent by server analyticum.com
Fingerprint: ad2b82ac78767c062980f20fc403facdff0a8ddf
RSA 4096 bits (e 65537) / SHA256withRSA
OCSP ERROR: Request failed with HTTP status: 500 [http://ocsp.startssl.com/sub/class2/server/ca]

2 Sent by server StartCom Class 2 Primary Intermediate Server CA
Fingerprint: 064969b7f4d6a74fd098be59d379fae429a906fb
RSA 2048 bits (e 65537) / SHA256withRSA

3 In trust store StartCom Certification Authority Self-signed
Fingerprint: 3e2bf7f2031b96f38ce6c4d8a85d3e2d58476a0f
RSA 4096 bits (e 65537) / SHA1withRSA
Weak or insecure signature, but no impact on root certificate

Today (with no changes on my side), OCSP fails for the intermediary certificate instead of the server certificate:

1 Sent by server analyticum.com
Fingerprint: ad2b82ac78767c062980f20fc403facdff0a8ddf
RSA 4096 bits (e 65537) / SHA256withRSA

2 Sent by server StartCom Class 2 Primary Intermediate Server CA
Fingerprint: 064969b7f4d6a74fd098be59d379fae429a906fb
RSA 2048 bits (e 65537) / SHA256withRSA
OCSP ERROR: Request failed with HTTP status: 500 [http://ocsp.startssl.com/ca]

3 In trust store StartCom Certification Authority Self-signed
Fingerprint: 3e2bf7f2031b96f38ce6c4d8a85d3e2d58476a0f
RSA 4096 bits (e 65537) / SHA1withRSA
Weak or insecure signature, but no impact on root certificate

Can someone please point me at how to avoid this?

Regarding the future issue of making ALPN available, I did read that openssl 1.0.2 would be required (https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation - I could not find the release notes quoted under (6) in the wikipedia article or anything like this in the openssl release notes which I did find, though.) while pfSense does provide OpenSSL 1.0.1l-freebsd 15 Jan 2015 now. Does anyone know whom to notify to change that well before the end of 2015?

Regards,

Michael

PiBa

Hi Michael

Looks to me like the issue is on the startssl side?: https://forum.startcom.org/viewtopic.php?f=15&p=22411&sid=fe8d39fe277387117a769acfe59cf534

With haproxy 0.30 my ocsp still works properly for me with certificate from another CA..

As for openssl 1.0.2, its a bit tricky.. haproxy is build agains openssl from ports.. If you currently check haproxy -vv output youl see that newest dev4 package is using showing the folowing..:

Built with OpenSSL version : OpenSSL 1.0.2d 9 Jul 2015
Running on OpenSSL version : OpenSSL 1.0.1m 19 Mar 2015 (VERSIONS DIFFER!)

Ive not noticed problems caused by this yet.. Also i have discussed with pfSense devs, and its deemed difficult to fix until the new pkgng package system will be used..

Best regards
PiBa-NL

michaelschefczyk

Dear PiBa,

Thank you very much. The OCSP issue was indeed caused by a problem in StartCom's infrastructure, as StartCom's friendly certmaster did confirm. It is gone at present and I hope that it will remain OK in the future.

Thanks for following up with the OpenSSL issue. It would be good to move from npn to alpn before major providers and their browsers (Chrome) will make the switch, probably by the end of 2015, if they stick to their announcements. Nevertheless all of us should practically take it as it is.

Regards,

Michael