SSL certs handling and HAproxy

[lewis]

I'll work on this shortly but the post is getting a bit confused.

I wanted to work on one single host, not load balanced, just using ACME/HAProxy setup to see it working.
Then I could move on to what is happening with this load balancing thing that's come up.

In terms of the load balanced ones, I did run a quick test for the load balanced servers.
As soon as I disabled the rule, traffic stopped to the web servers.
I monitored two of them and both stopped receiving connections.

As mentioned, I have an alias that contains the three web servers LAN IPs.
That's what you see in the rule, FoxHAP

Now, I'm moving back to the tests I've been doing using just one server.
This is the rule when I enable it;

Then the front end of HAProxy;

Then the back end;

The load balanced servers aren't using ACME, they share their SSL certs over a SAN share.

What I'd like to accomplish is to see the single server I initially wanted to learn from working.
So, that one server has a virtual IP.
I have HAProxy set up with frontend and backend.
The ACME certificate is installed on pfsense. I also have them installed on the web server itself. I enable/disable while testing.

On the frontend, I have the external IP (virtual IP) selected and I've tried 80 and 443, SSL offloading on/off.
In the ACL, I have a unique name, I've set to 'Host matches' and the value is the fqdn.
In the actions, table, I have s unique ACL name I created.
In the Default Backend, I have the backend name.
On the backend, I do have the client certificate selected.
In Additional certificates, I have the ACME cert selected for this domain. and 'Add ACL for certificate subject alternative names' selected.

These are from what ever I've been reading that keeps changing. One article says this, another says that, nothing has been consistent enough to get this working.

That's why I posted asking if someone might have a current, up to date, known to work article/doc that I could following because after a while, it becomes a bunch of mixed up settings that will never work, which is where I'm at now. Again, not thinking about the load balanced servers yet.

[lewis]

@stephenw10 said in SSL certs handling and HAproxy:

Make sure there are firewall pass rules to allow the incoming traffic to access HAProxy.

Wait, do you mean some other kind of rule and not NAT rule?

Check the HAProxy status to see what's happening.

What am I looking for here, there's a lot of information.

[lewis]

I was going to edit the previous but my extra notes might be missed.

Yes, I do see haproxy status page.
I also notice that other than many columns, there's little data in there which makes me think haproxy is in fact being bypassed.
I'm not sure what part of that page would be useful to share.

[kiokoman]

@lewis
normal firewall rules on WAN. not a NAT
destination should be "this firewall (self)" if you want haproxy to intercept the traffic
like my print screen

[lewis]

Ok, to be sure I don't screw anything up here.
I do see the NAT firewall rule I created in the WAN view as well.
You're saying I need to remove the rule from the NAT view then add it back into the WAN view and make it self which would make sense.
The difference in your rule and my setup would be that I'm using one of the VIPs so would need to configure that as the incoming WAN IP.

I think source will be any and destination will be self as you said which is when HAProxy takes over since it knows the VIP it needs to monitor for.

[lewis]

I'm wrong in my assumption in the last comment because I have multiple VIPs.

[kiokoman]

@lewis
i'm not sure on this but maybe destination could be your vip address or the alias with the vip addresses, i never had the need to have 2 different configuration so i'm always putting "this firewall" as destination
also i'm always configuring my backend on port 80, without self signed cert and stuff, i trust my own network so i don't need this kind of complexity

[lewis]

I don't see any options for adding a VIP in destination.

[stephenw10]

Yes the destination on the pass rule should be the VIP. It would work fine using 'This Firewall' but best practice there is to make a rule as precise as possible.

[lewis]

Can someone share the specifics of this rule?

I'll test the single server first, and if that works, I'll then tackle the three web servers I've talked about.

[stephenw10]

It should looks something like:

Where 172.21.18.1 is the VIP.

[lewis]

There is nothing in the rule creation page that offers any way of selecting a VIP.
Do I have to create another alias, just so I can use 'other'?

[stephenw10]

Just select 'Address or Alias' and enter the IP.

[lewis]

This post is deleted!

[lewis]

So here's my WAN rule;

I seem to be reaching the firewall now but not the site. I say that because testing the SSL cert is now showing the days left of the cert on pfsense instead of the one on the web server.

I'm getting a
"Browser detected a potential security threat and did not continue to www.aaa.com because this website requires a secure connection"
No traffic is getting to the web server. I'm guessing that's the mess in the haproxy setup now.

[stephenw10]

It's because by default the pfSense GUI listens on those ports. If you have not already you will need to change the port that runs on:
https://docs.netgate.com/pfsense/en/latest/config/advanced-admin.html#tcp-port

And disable the webgui redirect rule there if you want to be able to connect on port 80 to HAProxy too.

[lewis]

I don't use standard ports for the GUI so that's not a problem.
I do however, have those standard ports in use on the primary IP which is why I'm using VIPs for web servers.

Maybe I'm missing something in what you're saying?

[lewis]

BTW, I do now see Byes in and out in the haproxy status page for that web server.
Which also seems to confirm that the load balancing has never actually worked, even if some part of it definitely does.

[lewis]

I might be close now.

I'm using a remote browser and I'm getting this;

has a security policy called HTTP Strict Transport Security (HSTS), which means that Tor Browser can only connect to it securely.
You can’t add an exception to visit this site.
The issue is most likely with the website, and there is nothing you can do to resolve it.

I've disabled HSTS on the web server but still getting the warning.
Maybe now I have to look at the server but no one has answered some of the questions I've asked about the server.

I've asked about how SSL should be set up on the server and especially if it's one that has virtualhosts.

Some talk about a self signed cert and some don't.

[stephenw10]

Can you connect using https? You are connecting to HAProxy so policy changes would have to be there. If you wanted to allow http connections which you probably don't. More likely you would add a redircet rule in HAProxy to redirect http requests to https.
But I would get it working using https directly first.