problem configuring HAproxy with subdomains

[bensz]

Hello,
I have a new netgate 3100 up to date.
I want to configure subdomains with HAproxy.
I use this guide https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki
So I try with one server for the moment.
On http port 80, no problem.
But I have a problem with https.
I change the webgui port to 4433. But the problem is pfsense redirect query from wan on port 443 to himself on port 4433.
I create a NAT to transfer port 443 to IP 127.0.0.1, like for port 80. But I wanted that queries goes on port 443 come to HAproxy.
I don't know what else for informations.
Thanks for help
Benoit

[PiBa]

@bensz
Might be a browser cache that remembers the redirect? Have you tried a different browser? Also have you tried disabling the 'webgui-redirect' from advanced settings?

No need to portforward when using haproxy, it can listen on your wan-ip directly, do need regular firewall rules to allow the traffic though.. And are you sure its listening properly according to diagnostics/sockets?

[bensz]

Hello and thanks for reply
to yours questions:
It's not a browser cache, I try with a test website like webpagetest.org
I tried with disabling redirect, but no changes
You're right for NAT, I disabled the NAT rule and keep the firewall rule
How can I be sure it's listening properly?

Thanks
Benoit

[PiBa]

@bensz
In pfSense Diagnostics/Sockets page you can see which processes are listening on :443 (and :80 perhaps as well?), that should only be haproxy. The webgui nginx server should only be listening on your alternately configured :4433 port.

[bensz]

Hi,
I saw on sockets, everything seems fine, haproxy is earing wan_ip:80 and wan_ip:443.
The problem is this website says it doesn't see http page and I want to install let's encrypt certificate and it says no connection to http.
It's strange, but from the inside, that works correctly.
Nginx wait on port 4433.
thanks
Benoit

[PiBa]

@bensz said in problem configuring HAproxy with subdomains:

this website says it doesn't see http page

Umm?

Where do you want to run the let's-encrypt client? On pfSense for haproxy's use? Do you forward the acme-challenge request to the correct place for the LE servers to see the file? How do you have haproxy configured? Can you share the haproxy.cfg ? (obfuscate IP, domainname, stuff if desired)

[bensz]

I want to run let's encrypt on the server in the subdomain.
The master domain is zelec.homelinux.net, the subdomain is dolizelec.zelec.homelinux.net. I use cerbot to install LE certificate.
it reply:
root@dolizelec:~# certbot --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache

Which names would you like to activate HTTPS for?

---

1: dolizelec.zelec.homelinux.net

---

Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1
Requesting a certificate for dolizelec.zelec.homelinux.net
Performing the following challenges:
http-01 challenge for dolizelec.zelec.homelinux.net
Enabled Apache rewrite module
Waiting for verification...
Challenge failed for domain dolizelec.zelec.homelinux.net
http-01 challenge for dolizelec.zelec.homelinux.net
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

• The following errors were reported by the server:

  Domain: dolizelec.zelec.homelinux.net
  Type: connection
  Detail: Fetching
  http://dolizelec.zelec.homelinux.net/.well-known/acme-challenge/MIrPxgDyeG5K6h4UqLLbdHMz-YCnzJYlih5dCFHnsfU:
  Timeout during connect (likely firewall problem)

  To fix these errors, please make sure that your domain name was
  entered correctly and the DNS A/AAAA record(s) for that domain
  contain(s) the right IP address. Additionally, please check that
  your computer has a publicly routable IP address and that no
  firewalls are preventing the server from communicating with the
  client. If you're using the webroot plugin, you should also verify
  that you are serving files from the webroot path you provided.

[0_1610661081568_haproxy.cfg](Uploading 0%) haproxy.txt

[PiBa]

@bensz
have you made a firewall rule on the wan to allow tcp access from any to wanip:80 ?

[bensz]

Is that correct as I've done

[PiBa]

@bensz
Yes

Dolizelec Default Page
It works!

[bensz]

Thank you,
I'll continue to search with certbot why it doesn't want to work.
Thanks again
Benoit

[PiBa]

@bensz
So now you should be able to use acme to get a certificate, after that the https site should be configured with it..

[PiBa]

@bensz said in problem configuring HAproxy with subdomains:

search with certbot why it doesn't want to work.

Previously you got the message "Timeout during connect (likely firewall problem)" .. I think that should be solved now.. So what is Certbot / LEservers saying now for error.?

[bensz]

certbot reply

• The following errors were reported by the server:

  Domain: dolizelec.zelec.homelinux.net
  Type: unauthorized
  Detail: Invalid response from
  http://dolizelec.zelec.homelinux.net/.well-known/acme-challenge/_bWOWkqCKIrs7bOzwJbxGLfmetEfGCG6phCO8q4CqdY
  [92.188.16.194]: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML
  2.0//EN">\n<html><head>\n<title>404 Not
  Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

[bensz]

Hello,
I solved things with certbot, but now my problem is port 443 goes to pfsense webgui instead of my haproxy backend.
I don't understand why
Thanks

[PiBa]

@bensz
Webgui is still configured on :4433?
Is haproxy still the only service listening on :443 ?
Is haproxy configured to forward traffic to pfSense webgui with some backend, maybe a acl doesnt match properly and sends the traffic to that backend then?
Do you have a nat rule that points 443 traffic to the webgui port?

[bensz]

Hi,
Webgui is on port 4433
As I know haproxy is the only service listening 443
there's no backend to webgui. For the moment, I have 2 frontend 1 for port 80 and 1 for port 443. and 2 backend for rhe same reason.
I use a NAT rule for 443 but il's for my old server, and I disable it for test.
My new server (in a subdomain) and my old server (with NAT) use apache. And when I try to certificate with LE, the reply is a nginx server reply in 443. As I don't have any nginx server, I suppose it's the webgui which is served by Nginx.

Thanks

Here my haprowy.cfg if that can help
haproxy.txt

[PiBa]

@bensz said in problem configuring HAproxy with subdomains:

the reply is a nginx server reply in 443

That might be, but the pfSense webgui does not seem the send the <!DOCTYPE HTML PUBLIC which the certbot error shows, so if it is nginx then its a different nginx server than that running on pfSense.. i doubt its finding its way to the webgui.. your Apache server however does send that doctype when requested for a page that does not exist.. I think the certbot>letsencrypt request does properly land on the Apache server but maybe the token file was just not placed in the correct subfolder?

[bensz]

You're right, I delete certbot and install a selfsignated certificate. That works correctly. The problem come from LE. As the server is empty, I will reinstall it.
Thank you for your help
Benoit