HAProxy / Lets Encrypt / Postfix - Dovecot

RHLinux

Is this configuration possible?

pfSense / HAProxy will offload the SSL (w/ ACME cert) and forward on to the postfix dovecot server with a self signed certificate. The idea is that ACME will renew the certificates with HAProxy decrypting (using LetsEncrypt Cert) and re-encrypting with the self signed certificate, which will not expire (in a reasonable amount of time) and the data will be encrypted to the back end.

My assumption is that it's totally possible, but I don't want to go down a rabbit hole to find out its not. If anyone has done this type of configuration, any pointers or things to watch out for?

Mats

To make this easier to understand.

Is it for securing the mail protocols (ie IMAPS) or for a webfront on the mailserver?
I have done the later

jimp

I don't think that's going to work properly. Especially for clients that want to do STARTTLS. HAProxy doesn't know enough about SMTP/POP3/IMAP protocols to actually proxy the protocols, just the TCP/TLS portion.

So it might work for some cases but I don't think you or your clients would be happy with the limitations.

Setup your acme client/acme.sh/certbot/whatever on the mail server directly and let it have its own certificate directly. Or setup ACME on pfSense to write the certs out and then have the mail server periodically fetch them from there and reload. Or have a script push them from pfSense to the mail server.

Gertjan

I'm using postfix myself on a dedicated server.
No firewall what so ever that protects the mail ports 25, 465 and 587 - 465 and 587 being used by my mail clients, as 993 SIMAP and 995 SPOP, 110 and 143 are abandoned these days, and not postfix related.
Even STARTTLS (587) starts to fade out, it's all"465" = SMTPS these days.

postfix uses the same certs from LetsEnscrypt / acme.sh as the web servers on that server.
Most mails leave and enter saying " .... Trusted TLS connection established from/to .... " on port 25.

What I want to say : postfix, IMHO, seems rock solid to me, and can be exposed to the net directly.

Note : I do have fail23ban scanning my main postfix mail log so it can block the mail-port hammers, and other mail servers that do not support my "minimum mail protocol requirements".

@jimp said in HAProxy / Lets Encrypt / Postfix - Dovecot:

Setup your acme client/acme.sh/certbot/whatever on the mail server directly and let it have its own certificate directly. Or setup ACME on pfSense to write the certs out and then have the mail server periodically fetch them from there and reload. Or have a script push them from pfSense to the mail server.

My acme.sh "deploy.sh" hook script :

#!/bin/sh
set -e

  check_path="/root/.acme.sh/${Le_Domain}/${Le_Domain}.conf"
  destination="/etc/ssl/"
  destinationdir=${destination}${Le_Domain}
	if [ -f $check_path ]; then
		if [ ! -d $destinationdir ]; then
			mkdir $destinationdir
		fi
		cat $CERT_KEY_PATH $CERT_FULLCHAIN_PATH ${destination}dh/RSA4096.pem > ${destinationdir}/${Le_Domain}.pem
		cp $CERT_KEY_PATH ${destinationdir}/${Le_Domain}.key
		chmod 400 ${destinationdir}/${Le_Domain}.pem
		chmod 400 ${destinationdir}/${Le_Domain}.key
		service apache2 reload >/dev/null
		service postfix reload >/dev/null

	# courier will also use these certs.
		service courier-pop-ssl force-reload >/dev/null
		service courier-imap-ssl force-reload >/dev/null

	# exception - extra treatment :
		if [ "$Le_Domain" == "yyyy.xxxx-bbbb.org" ]; then
			service monit reload >/dev/null
			service webmin restart >/dev/null
		fi


	ACCOUNT_EMAIL=gw.kroeb@gmail.com

	cat <<-EOF | mail -r acme@aaaa-vvvvv.tld -s "Certificates renewed" $ACCOUNT_EMAIL

	Renewed the following certificate(s):
	Host: $Le_Domain

	$(/root/.acme.sh/acme.sh  --version 2>&1)

	EOF
	fi

used by acme.sh :

   --deploy-hook                     The hook file to deploy cert

where the hook file is this "deploy.sh"
For every cert on my server, the using processes are restarted / reloaded.

edit : note : the acme.sh usage above is not to be confonded with the"acme.sh" version used by the LetEnsrypt package written by Jimp.

RHLinux

Thanks for the information. I think I will setup a more conventional method to have the certs on the mail server. Just wanted to see if it was possible and not to go down the rabbit hole and waist lots of hours and head scratching trying to implement something that is not doable.

RHLinux

VioletDragon

@Gertjan That's not true. SMTP 465 is deprecated. Should use 587 Submission.

Proxying SMTP & IMAP using Haproxy will not work, You can set smtpd_upstream_proxy_protocol=haproxy in main.cf for Postfix but the problem is that IPs for SPAM filtering will not proxy pass.

I suggest not bothering, this will waste your life away, Haproxy is a HTTP Load Balancer not a SMTP/IMAP Load Balancer, better to Open Port 25, 587 & 993 NAT -> Port Fowarding then use Port 443 for Webmail through Haproxy.

Clients that use STARTTLS will not work and will not Initialize through Haproxy.

Old Post I know but this is more for people that comes across this in the future.

Regards.

Gertjan

@VioletDragon said in HAProxy / Lets Encrypt / Postfix - Dovecot:

That's not true. SMTP 465 is deprecated. Should use 587 Submission.

If I read this : What SMTP port should be used? Port 25 or 587? and I compare that with the very first phrase from RFC 8314 stating the very obvious ( is it ? ) "Cleartext Considered Obsolete: Use of Transport Layer Security (TLS) for Email Submission and Access" I see a conflict.
Port 587 is initiated using clear text.
Port 465 is TLS (before known as SSL) from byte 0.

So : in 2024, we have to go back to a partial 'clear' message exchange ? I'm not sure about that ^^

Example : I hope to shut down in the nearby future all "port 80" ports for all my web server. It will become "port 443" only. Non pure https (TLS) traffic for web server is dead.
(Ok, I agree, maybe not this decade yet ^^ )

But, this isn't really an issue, as mails dropped from our mail clients (people I know), on our mail server is not really "Public Internet thing". I as an admin can decide how mails get dropped by the MUA as I see fit.
This port can't be used by the 'public', only by the known mail clients, the ones that have an account == a mail addresses + password with this mail server.

My own opinion is : everything goes TLS these days, "no more 'in the clear' text streams'.
( the major exception is : DNS resolving, as our 13 main DNS root servers still "don't do TLS" [for understandable $$$$ reasons] )

Btw Submission 587 dates from somewhere in the last century, and 465, using SSL and now TLS, is far more recent. IMHO : more logic to use.

I still expose port 587 as an 'mail drop' entry point on on mail server, but the STARTTLS is mandatory, is has to be asserted by the client, if not : no access. So why even initiating this a clear text initial handshake ? I went 100 % TLS == port 465 - all the way right from the start.

The only things that needs to 100 % compliant RFC is port 25 : I still (have to) accept clear (non TLS) mails, as some mail server don't do TLS, or have their setup wrong. Working with certificates is still not main stream knowledge these days.
I also send, ones in a while, clear mails, as the destination doesn't accept 'TLS' - also strange.

Btw : my mail server is hosted on a dedicated bare bone iron case, somewhere in a data center in Paris. It has it own pool of IPv4 and IPv6 (for all my individual domain names) and it's not behind a firewall worth mentioning.
That said : ip(4/6)tables is used by fail2ban to keep the mail/web/etc abusers out of the door for a while.
I started this server using Debian Lenny, way, way in the past. Debian 11.9 Bullseye today (I know, I should upgrade).
I'm not a proxy user neither, as I don't have to hide several hosts (domains) behind one IP over a single upstream connection.

VioletDragon

@Gertjan Reason why 465 is deprecated due to its encryption over SSL which isweak hence why it’s been replaced for 587 over TLS. Just because cloud providers and ISPs supports 465 still doesn’t mean you should.

Port 25 is used for Mail Servers to Exchange Mails and should not be used for Client to Server Roles.

zewl23

Hi folks,

I know this thread is kinda old, but thanks to everyone for replying as it was interesting reading for me.

I am now trying to resolve similiar thing. Maybe a bit different and would like to know the opinion on how to do that correctly from the start.

Currently there is a working mail server that is publicly available with lets encrypt certs...

But as I have only one Public IP and there are plans to have more subdomains with the same ports which will present different services, I belive that HA proxy is the only option for me.

However, if I have lets encrypt on the servers already, do I need to setup the acme for the HA proxy?

If I would like to filter the encrypted traffic then I believe that the only chance is to go with ACME setup and push certs to servers from pfsense, right?

EDIT: there is SSL passthrough I had no idea about, but then I wouldnt be able to unencrypt? Or could I sent the certs to the pfsense for filtering the SSL traffic from the servers using lets encrypt directly on them?

To achieve what you are describing—using HAProxy on pfSense to distribute HTTPS (port 443) traffic to different backend servers based on the subdomain (SNI-based routing) while having only one public IP—you can set up SSL passthrough or SSL termination on the HAProxy. Let me explain both approaches and how to implement them:

Key Concepts:
SSL Termination: HAProxy handles the SSL decryption. It decrypts the traffic and forwards plain HTTP to your backend servers. In this setup, HAProxy manages the SSL certificates, and the backend servers don't need to handle encryption.
SSL Passthrough: HAProxy forwards the encrypted traffic as-is to the backend servers, and each backend server handles its own SSL certificates.
Since you're already using Let's Encrypt directly on the backend hosts, I recommend setting up SSL passthrough, which forwards HTTPS traffic to the correct backend server based on the SNI (Server Name Indication), preserving the encryption on the backends.

Steps for SSL Passthrough on HAProxy in pfSense:

Install HAProxy on pfSense
Navigate to System > Package Manager > Available Packages.
Search for HAProxy and install the package.
Once installed, you'll see HAProxy under Services > HAProxy.
Configure HAProxy for SSL Passthrough
Create Frontend in HAProxy
Go to Services > HAProxy > Frontend and click on +Add.
Configure the frontend for SSL passthrough:
Name: Give it a descriptive name (e.g., https_frontend).
Bind Address: Set this to your WAN/Public IP on port 443 (WAN_IP:443).

SSL Offloading: Uncheck this option, as we're doing SSL passthrough, meaning no SSL decryption on HAProxy.
Type: Set to tcp because we are forwarding encrypted traffic.
Default Backend: Leave this as none for now, as we'll route based on SNI.

Enable SNI-based Routing in Frontend
Scroll down to the Actions section and click on Advanced pass thru.
Add the following line to enable SNI-based routing:

acl host_example1 hdr(host) -i subdomain1.example.com
acl host_example2 hdr(host) -i subdomain2.example.com
use-server backend1 if host_example1
use-server backend2 if host_example2

This example routes traffic based on the Host header (SNI), so requests to subdomain1.example.com will go to backend1, and requests to subdomain2.example.com will go to backend2.

Configure Backend Servers
Go to Services > HAProxy > Backend and click on +Add.
For each backend server:
Name: Give it a name (e.g., backend1 for subdomain1.example.com).

Mode: Set this to tcp because SSL passthrough is used.
Server List: Add your backend server IP and port, typically port 443 for HTTPS traffic. For example, if subdomain1.example.com points to 192.168.1.10, the server line would look like:

192.168.1.10:443

Repeat this for all your backend servers (one per subdomain).

Configure Firewall Rules
Ensure you have a firewall rule on your WAN interface to allow incoming traffic on port 443:

Go to Firewall > Rules > WAN and create a rule:
Protocol: TCP
Destination Port Range: HTTPS (443)
Destination: Your public IP address (WAN interface).

(Optional) Setting up Health Checks
HAProxy can check if the backend servers are online. This helps in load balancing and failover situations:

In each backend server configuration, enable Health Checking.

Set up a health check using HTTP or TCP ping checks to ensure the backend is reachable.

Test the Configuration
Ensure that your DNS is configured correctly, with the subdomains (e.g., subdomain1.example.com, subdomain2.example.com) pointing to the same public IP address.

When you visit https://subdomain1.example.com, HAProxy should forward the traffic to the appropriate backend (192.168.1.10:443 in this case).

Repeat for subdomain2.example.com.

Example Configuration Summary

Frontend (HTTPS):
Name: https_frontend
Bind Address: WAN_IP:443
SSL Offloading: Disabled (because we are using SSL passthrough)
Type: TCP
SNI-based Routing:
acl host_example1 hdr(host) -i subdomain1.example.com
acl host_example2 hdr(host) -i subdomain2.example.com
use-server backend1 if host_example1
use-server backend2 if host_example2

Backend Servers:
Backend1:
Name: backend1
Mode: TCP
Server: 192.168.1.10:443 (backend server handling subdomain1.example.com)
Backend2:
Name: backend2
Mode: TCP
Server: 192.168.1.11:443 (backend server handling subdomain2.example.com)

Considerations:
DNS: Make sure your DNS entries for each subdomain (subdomain1.example.com, subdomain2.example.com) are pointed to your public IP.

Certificates: Since SSL passthrough is used, the SSL certificates must be correctly configured and active on each backend server. Each backend server will serve its own Let's Encrypt certificate for the respective subdomain.
This setup should route traffic to the correct backend server based on the subdomain, while keeping SSL termination at the backend servers. Let me know if you need any further clarification or help with troubleshooting!

zewl23

@zewl23

If your Mail server uses protocols other than HTTP/HTTPS (such as IMAP, SMTP, POP3, etc.) that also require SSL/TLS certificates, combining everything behind a single public IP while managing traffic through pfSense + HAProxy can be a bit more involved. These protocols also need to be handled securely using SSL/TLS certificates.

Here's how you can approach this:

List the Protocols Mail Uses:
HTTP/HTTPS (Webmail, Admin Interface): Typically served over port 80 (HTTP) and 443 (HTTPS).
SMTP (Mail Transfer): Typically served over port 25 (unencrypted), port 465 (SMTPS), and port 587 (Submission).
IMAP (Mail Retrieval): Typically served over port 143 (unencrypted) and 993 (IMAPS).
POP3 (Mail Retrieval): Typically served over port 110 (unencrypted) and 995 (POP3S).
All of these protocols need to work with SSL/TLS, and they may use the same Let's Encrypt certificates or other SSL certificates.
Challenges of SSL Termination for Non-HTTP Protocols:
While HAProxy can handle TCP and HTTP(S) traffic, it doesn’t natively support protocols like IMAP, SMTP, POP3 with their specific TLS mechanisms. However, you can set up SSL passthrough or use a combination of HAProxy and stunnel to handle those protocols securely behind pfSense.
Approach:
There are two main ways to handle SSL termination or passthrough for multiple services using different protocols:

Use SSL Termination for HTTP/HTTPS with HAProxy and SSL Passthrough for IMAP/SMTP/POP3.
Combine HAProxy with stunnel to manage TLS termination for all protocols.
Option 1: HAProxy (SSL Termination for HTTP/HTTPS) + SSL Passthrough for Other Protocols
For this approach:

HTTP/HTTPS traffic will be handled by HAProxy, where SSL termination will happen.
IMAP/SMTP/POP3 will be configured for SSL passthrough, allowing the traffic to flow encrypted directly to the mail server, which will handle the decryption.
Step-by-Step Setup

Install and Configure HAProxy for HTTP/HTTPS Traffic: For HTTP/HTTPS, follow the same steps as mentioned earlier to perform SSL termination for your webmail:

Terminate SSL for HTTPS traffic at pfSense using HAProxy.
Forward decrypted traffic to the Mail server over HTTP.
2. Configure SSL Passthrough for Other Protocols: To handle protocols like IMAP, SMTP, and POP3 (SSL Passthrough):

SMTP/IMAP/POP3 SSL Ports should be configured as TCP mode in HAProxy, and the traffic will be forwarded as-is to the backend server.
Here’s how to do it:

Configure a TCP Frontend for IMAP/POP3/SMTP:

Go to Services > HAProxy > Frontend and click +Add.
Name: Set it to something descriptive like smtp_frontend or imap_frontend.
Bind Address: Use your WAN IP and the respective ports for IMAP, SMTP, and POP3:
SMTP: WAN_IP:465 (SMTPS)
IMAP: WAN_IP:993 (IMAPS)
POP3: WAN_IP:995 (POP3S)
SSL Offloading: Disable SSL offloading (since we're using passthrough).
Mode: Set to TCP.
Backend Server: Configure backends to forward traffic to your Mail server.
Configure Backend Servers for Each Protocol:

Go to Services > HAProxy > Backend and click +Add.
Name: Set it to smtp_backend, imap_backend, or pop3_backend based on the protocol.
Mode: Set this to TCP (SSL passthrough).
Server List: Add your Mail server with the respective port for each service (e.g., 192.168.1.10:465 for SMTP, 192.168.1.10:993 for IMAPS).
Repeat this for all the services (IMAP, SMTP, POP3).
This approach forwards the SSL/TLS encrypted traffic for non-HTTP protocols directly to your Mail server, which then handles the encryption and decryption.

Example Configuration:
Frontend for SMTP (port 465):

Bind Address: WAN_IP:465
Mode: TCP
Backend: smtp_backend
Backend for SMTP:

Server: 192.168.1.10:465 (Mail server on internal network)
Frontend for IMAPS (port 993):

Bind Address: WAN_IP:993
Mode: TCP
Backend: imap_backend
Backend for IMAPS:

Server: 192.168.1.10:993
Option 2: Combine HAProxy with stunnel (SSL Termination for All Protocols)
If you want to terminate SSL/TLS for all protocols (HTTP, IMAP, SMTP, POP3) at pfSense and filter the decrypted traffic, you can use stunnel along with HAProxy.

stunnel is a proxy designed to add SSL encryption to existing clients and servers without changes to the application code.
You can use stunnel to decrypt IMAP/POP3/SMTP traffic and forward it to HAProxy or directly to your backend.
Steps:
Install stunnel on pfSense:

Install the stunnel package from System > Package Manager > Available Packages.
Configure stunnel for Mail Protocols:

Set up stunnel to listen on the respective ports (465, 993, 995) and decrypt traffic using your Let's Encrypt certificate.
stunnel will forward the decrypted traffic to your iRedMail server over plain IMAP/POP3/SMTP.
HAProxy for HTTP/HTTPS:

Use HAProxy to handle SSL termination for HTTP/HTTPS as explained earlier.
Firewall Rules:

Configure firewall rules to allow traffic on the mail ports (25, 465, 993, 995, etc.).
Example Configuration Summary (Using SSL Passthrough for Mail Protocols)
Frontend (HTTPS, SMTP, IMAP, POP3):
HTTPS (port 443):

Bind Address: WAN_IP:443
SSL Offloading: Enabled (HAProxy manages SSL for HTTPS).
Backend: Webmail backend on HTTP.
IMAPS (port 993):

Bind Address: WAN_IP:993
SSL Offloading: Disabled (SSL passthrough).
Backend: iRedMail server on port 993.
SMTP (port 465):

Bind Address: WAN_IP:465
SSL Offloading: Disabled (SSL passthrough).
Backend: iRedMail server on port 465.
Backend Servers (Decrypted HTTP, Encrypted Mail):
Backend1 (HTTP for Webmail): 192.168.1.10:80
Backend2 (IMAPS for Mail): 192.168.1.10:993
Backend3 (SMTPS for Mail): 192.168.1.10:465
Conclusion:
To handle a setup where Mail uses multiple protocols over SSL/TLS:

For HTTP/HTTPS (web interfaces), you can use SSL termination at HAProxy to decrypt and filter the traffic at pfSense.
For SMTP, IMAP, and POP3, it’s better to use SSL passthrough to forward encrypted traffic directly to the Mail server, which will handle SSL/TLS for these protocols. Alternatively, you can use stunnel to decrypt these protocols if you want to inspect/filter the traffic.
This combined approach gives you flexibility and control over traffic while maintaining security for all protocols. Let me know if you need further assistance with any specific configuration!