HAProxy OSCP stapling possibly broken

michaelschefczyk

Dear PiBa, dear all,

I suspect that the packages haproxy 0.59_4 (as well as haproxy-devel 0.59_3) have an issue with OSCP stapling with SSL Offloading that the previous versions did not have.

The error is that the OSCP data is not stapled correctly as can be seen in Qualys SSL Test, via error mes-sages when modifying the haproxy configuration and in Firefox via "MOZIL-LA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING ".

This does occur only under the following circumstances:

Letsencrypt certificate (generated with ACME on pfsense - I did not try any other scenario) with "OCSP Must Staple" enabled. Without this flag, no errors will occur. In previous versions, everything was fine with this flag enabled, however.
SNI situation where the certificate is not the primary certificate. On the primary certificate, "OCSP Must Staple" does not create any issues. Just secondary certificates in a shared frontend where the certificate is listed under "Additional certificates" in the section "SSL Offloading" do suffer from the issues described above.

Can anyone please confirm this? Is there a workaround that does not required disabling "OCSP Must Staple"? Can this be corrected in the package?

Regards,

Michael Schefczyk

PiBa

@michaelschefczyk
Hi michael,
I dont seem to see this.. Ive just reinstalled 0.59_3 with 1.8.12-8a200c7 and ssl labs tells me "OCSP stapling Yes"

In the system log of pfSense do you find the following?:

Aug 2 20:45:10	php-cgi		HAProxy OCSP socket update successful for frontend x..result: Next Update: Aug 8 01:00:00 2018 GMT
Aug 2 20:45:10	php-cgi		HAProxy Retrieving OCSP for frontend x..

though hmm i don't have 'must staple' ..
and probably i only checked my primary frontend.. give me a few minutes , ill check again :).

michaelschefczyk

Dear PiBa,

First of all: Thank you very much for your contribution to the HAProxy package and your commitment!!

I did create the issue again, if you want to observe it live: Please check [...].

My log content after marking the certificate "OCSP Must Staple" again is:

Aug 2 21:17:41 php-cgi haproxy: startup error output!: [WARNING] 213/211741 (2150) : Loading '/var/etc/haproxy/....pem.ocsp': Unable to parse OCSP response. Content will be ignored....

Again: No problem on the primary certificate, problems with secondary SNI certificates. This did work until the most recent package update.

Regards,

Michael

PiBa

@michaelschefczyk
You correct, and i'm a fool :). i should have properly read the complete problem description before hoping it might be 'user error'..

Anyhow i can reproduce the issue indeed with a secondary frontend. Will likely be sending a PR later this evening..

PiBa

@michaelschefczyk
Can you try this patch?: https://github.com/PiBa-NL/FreeBSD-ports/commit/c9eef0900f45448194cc8c83335e88eefd7e2e2b

michaelschefczyk

Dear PiBa,

I would be glad to. Can you please provide some hints regarding how to apply patches to packages?

Regards,

Michael

PiBa

@michaelschefczyk
Well trying it myself, its a bit of an issue that the patch is against 0.59_6 .. so it doesn't apply 'cleanly' against 0.59_4 ..

For now i think the easiest method is copying the whole function from here, and manually putting it into the haproxy.inc on your pfSense installation:
https://github.com/PiBa-NL/FreeBSD-ports/blob/c9eef0900f45448194cc8c83335e88eefd7e2e2b/net/pfSense-pkg-haproxy-devel/files/usr/local/pkg/haproxy/haproxy.inc#L1434
(could use diagnostics\editfile for that, or some scp program if youve enabled ssh)

PiBa

@michaelschefczyk
Made a PR for the whole shebang. https://github.com/pfsense/FreeBSD-ports/pull/547 as i'm positive the change will fix this issue..

michaelschefczyk

@piba
Dear PiBa,

the new version does solve the issue, as far as I can see. Thank you very much!!

When changing my configuration, I am getting warnings, however, which I did not get before (multiple of the following type):

[WARNING] 215/200912 (42317) : parsing [/var/etc/haproxy/haproxy.cfg:110] : a 'http-request' rule placed after a 'redirect' rule will still be processed before.

Do you spontaneously see the issue behind this?

Regards,

Michael

PiBa

@michaelschefczyk
i'm not sure where you put the 'redirect' rule.? Probably as a advanced text option? You could possibly make that a 'http-request redirect' action instead? If that doesn't help ill need to see the haproxy.cfg to say more without guessing .

michaelschefczyk

Dear PiBa,

I find the following http-requests at various places in my frontends:

http-request set-header		X-Forwarded-Proto http if !https
http-request set-header		X-Forwarded-Proto https if https

http-request set-var(txn.txnhost) hdr(host)
http-request set-var(txn.txnpath) path

The first two seem to be a consequence of the "Use "forwardfor" option" under "Advanced settings" in the frontend. I have no clue about the other two. Do you want me to transmit the haproxy.cfg file? It might be to sensitive to paste the real production file here, however.

Regards,

Michael

PiBa

@michaelschefczyk
Yes the complete haproxy.cfg should help tell where the redirect warning comes from.
But indeed do obfuscate passwords/public-ip's/domainnames you don't want to disclose.

michaelschefczyk

Dear PiBa,

Sorry, it is long as I selfhost everything from grandmother to children, non-profit assocations, research ...

Regards,

Michael

Automaticaly generated, dont edit manually.

Generated on: 2018-08-04 20:18

global
maxconn 100000
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 4096
server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats refresh 10
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend WAN1-merged
...

redirect scheme https code 301 if !{ ssl_fc }

    ...

PiBa

@michaelschefczyk
Looks like you configured this:

redirect scheme https code 301 if !{ ssl_fc }

And this same effect can be configured with a 'http-request redirect' action.

0_1533505324917_2018-08-05 23_37_31-Services_ HAProxy_ Frontend_ Edit - pfSe.localdomain.png

That would generate the line below with same effect, and it wont 'complain'. Also gives you more control about when its used exactly in relation to other http-request rules..

http-request redirect scheme https code 301  if  !{ ssl_fc }

As for the http-request set-var(txn.txnhost) hdr(host) its setting a variable txn.txnhost that is used by the acl's: acl survey-int var(txn.txnhost) -m str -i survey.domeine.de Where previously the acl would use the hdr(Host) header directly, but would never work when used for reply-traffic with http-response rules.
And allows also to use such acl's to perform actions on reply-traffic, where the host header would not have been present.. Its a change in the package to allow for more flexibility 'out of the box'..

michaelschefczyk

Dear PiBa,

Again, thank you very much! The complaint did not exist in previous versions. Your way does work. Placing the statement in the "Advanced pass thru" box does work also. I would not have understood this without your explanation!

Regards,

Michael