SSL certs handling and HAproxy

[lewis]

I checked the incoming headers and for some reason, http is still being forwarded to https so I think the problem is with the web server at this point.
It's odd because there is no forwarding configured, at all on the web server so I'll spend some time on this and see what I can discover.

It's darn close now that I've rebuilt the haproxy config.

[kiokoman]

@lewis
that error is usually from a redirect in the web server, haproxy try to comunicate with the web server via http but the server answer back via https but since haproxy is expecting HTTP traffic, it keeps resending the same request, resulting in a redirect loop

nginx could have something like -> return 301 https://$server_name$request_uri;
apache2 could have something like -> RewriteEngine / RewriteCond / RewriteRule

[lewis]

So what seems to be happening is that because the site was built with all urls being https, no matter if I hit it using http, the urls are all htps so the browser keeps getting https.

I dumped the application db, edited all the links to be http and it 'sort of' works.
I can fix the problems but it's not going to solve the main problem which is the sites cannot all be rebuilt to be http.
So, all this for nothing ti seems unless there's some way to get https pages working with haproxy as well.

[stephenw10]

Hmm, if you only see that when connecting externally though that's only when the connection is going through HAProxy.

What I expected would be a redirect in the HAProxy front end. That should only ever be for http to https for client connections but it would be possible to have ot set the other way around accidentally.

[stephenw10]

HAProxy will work fine using https to the backends if you need it to. I have no idea if you can add Varnish to that though.

[lewis]

Well, I showed all of my configuration and I can see haproxy sending to http.
Even after changing all of the urls to http, the page is still showing up broken to public clients but perfectly to internal.

[lewis]

@stephenw10 said in SSL certs handling and HAproxy:

HAProxy will work fine using https to the backends if you need it to. I have no idea if you can add Varnish to that though.

Right, that was the entire goal so if I can't do that, then this is all moot.

[stephenw10]

Well if the backend servers can only respond correctly to https connections then Varnish needs to use https to connect to them.

You might try using a very simple site that you know works fine with http as a test until you have the other parts working.

[lewis]

You might try using a very simple site that you know works fine with http as a test until you have the other parts working.

That's what this site was, for testing.
Other than this post, I can't find much more information so I guess I don't get to set this up the way I need it.

[stephenw10]

By very simple I mean just an index page. Nothing with anything linking to https.

[lewis]

@stephenw10 said in SSL certs handling and HAproxy:

By very simple I mean just an index page. Nothing with anything linking to https.

Yes, I already proved to myself that haproxy is sending http now but the site is built in https.
I dumped the db, renamed all of the https urls to http and it worked fine other than some small things I'd have to fix.

The problem is, I cannot do that to all of the apps that would be in play here, it's simply too much work.
The only other option I have is to not use haproxy on pfsense, but use acme only to have that centralized part I wanted.

Setups could be a proxy/varnish on a server. Not sure if pfsense can send pure http after acme to the LAN.

[stephenw10]

I'd be surprised if Varnish can't also use https. As long as the front and backend sessions are terminated there it can still see and cache all the content.

But, yes, if you're running Varnish on some separate server then it might be easier to do it all there.

[lewis]

These are some headers I have set in the Apache configuration.
I'm reading that haproxy also needs to have custom headers for some of these to work.

        # Add X-Forwarded-For header to log the original client IP
        RequestHeader set X-Forwarded-For %{REMOTE_ADDR}e

        # Add X-Real-IP header
        RequestHeader set X-Real-IP %{REMOTE_ADDR}e

        # Add X-Forwarded-Proto header to identify the protocol used by the client
        RequestHeader set X-Forwarded-Proto https

        # Use Vary header for content negotiation
        <Location />
            Header append Vary Accept-Encoding
        </Location>

I would need to add custom configs in haproxy, like under the frontend or backend sections, depending on specific requirements.
The actual syntax for these directives will typically look like http-request set-header X-Forwarded-For %[src] for X-Forwarded-For, as an example.
But that's the problem, I can't find enough information to even understand what I would need to add to haproxy.

Maybe I'm closer than I think but lack of knowledge and examples is making it impossible.

[lewis]

What's the chances?

[stephenw10]

You are accessing that via the proxy?

[lewis]

@stephenw10 said in SSL certs handling and HAproxy:

You are accessing that via the proxy?

I was searching Google which gave a link to these forums and this is what I got, repeatedly.
When we forward a domain, we typically maintain the old domain's cert also, just for this reason.

[kiokoman]

@lewis
must be an old entry because the forum is forum.netgate.com and not forum.pfsense.com
or they forgot to add the DNS

[stephenw10]

Oh well spotted! Yeah that's just an old link.

[lewis]

I wish I could figure this thing out. I very badly need a cache server for all of the web sites on the back end.
I appreciate the help you've all provided.

[lewis]

And today, another random thing happens on pfsense which I'm sure no dev will say 'oh ya, we're working on that one' to.

When I created my first acme cert and generated it, it should the dates of the cert start/end in Last renewed.
Today, I create a new cert, generate it and see nothing, just 'Issued Certificate Dates;' and nothing.