i have the exact set up but not able to sync

@louis2

I think separate back-ends for IPV4 and IPV6 are not necessary not even a good idea since the proxy has two completely isolated connections. One to the front end, one to the server.

So I decided to handle all server related connections via IPV4 since local IPV4 can not be reached from the internet.

That action already reduced the number of backends by a factor two 😊

@louis2

I worked around the problem by defining the mail-server addresses in my local DNS and using those names in the GUI. Never the less it is definitively not OK

Also note that I had the problem back when switching the health check on (to basic). Even more obscure switching the problem did persist when switching the health check off again.

No idea how the check should be done since there is no proper field to define the health check port number.

Up

from wan2, i can ping common website when the both wan are connected.
But when wan1 is disconnected, everything is very slow and i can't ping anything :
Screenshot 2024-10-17 140902.png

@accidentallyadmin
pfSense can only translate the source address to the stated IP. Maybe your ISP does an additional translation, but this is unusual if you have a public IP already.

You can verify the function of your outbound NAT rule by sniffing the traffic on WAN (Diagnostic > Packet Capture).

The outbound NAT rule is the last in the pipe, before packets leave the firewall.
See Firewall/NAT Processing Order Example for details.

@mcury
Thanks a lot, i am working with Gateway Groups and VIP addresses to check that everything is working as intented

Warm Regards

correction - it worked for a while... (until it was cached?)

@zewl23

If your Mail server uses protocols other than HTTP/HTTPS (such as IMAP, SMTP, POP3, etc.) that also require SSL/TLS certificates, combining everything behind a single public IP while managing traffic through pfSense + HAProxy can be a bit more involved. These protocols also need to be handled securely using SSL/TLS certificates.

Here's how you can approach this:

List the Protocols Mail Uses:
HTTP/HTTPS (Webmail, Admin Interface): Typically served over port 80 (HTTP) and 443 (HTTPS).
SMTP (Mail Transfer): Typically served over port 25 (unencrypted), port 465 (SMTPS), and port 587 (Submission).
IMAP (Mail Retrieval): Typically served over port 143 (unencrypted) and 993 (IMAPS).
POP3 (Mail Retrieval): Typically served over port 110 (unencrypted) and 995 (POP3S).
All of these protocols need to work with SSL/TLS, and they may use the same Let's Encrypt certificates or other SSL certificates.

Challenges of SSL Termination for Non-HTTP Protocols:
While HAProxy can handle TCP and HTTP(S) traffic, it doesn’t natively support protocols like IMAP, SMTP, POP3 with their specific TLS mechanisms. However, you can set up SSL passthrough or use a combination of HAProxy and stunnel to handle those protocols securely behind pfSense.

Approach:
There are two main ways to handle SSL termination or passthrough for multiple services using different protocols:

Use SSL Termination for HTTP/HTTPS with HAProxy and SSL Passthrough for IMAP/SMTP/POP3.
Combine HAProxy with stunnel to manage TLS termination for all protocols.
Option 1: HAProxy (SSL Termination for HTTP/HTTPS) + SSL Passthrough for Other Protocols
For this approach:

HTTP/HTTPS traffic will be handled by HAProxy, where SSL termination will happen.
IMAP/SMTP/POP3 will be configured for SSL passthrough, allowing the traffic to flow encrypted directly to the mail server, which will handle the decryption.
Step-by-Step Setup

Install and Configure HAProxy for HTTP/HTTPS Traffic: For HTTP/HTTPS, follow the same steps as mentioned earlier to perform SSL termination for your webmail:

Terminate SSL for HTTPS traffic at pfSense using HAProxy.
Forward decrypted traffic to the Mail server over HTTP.
2. Configure SSL Passthrough for Other Protocols: To handle protocols like IMAP, SMTP, and POP3 (SSL Passthrough):

SMTP/IMAP/POP3 SSL Ports should be configured as TCP mode in HAProxy, and the traffic will be forwarded as-is to the backend server.
Here’s how to do it:

Configure a TCP Frontend for IMAP/POP3/SMTP:

Go to Services > HAProxy > Frontend and click +Add.
Name: Set it to something descriptive like smtp_frontend or imap_frontend.
Bind Address: Use your WAN IP and the respective ports for IMAP, SMTP, and POP3:
SMTP: WAN_IP:465 (SMTPS)
IMAP: WAN_IP:993 (IMAPS)
POP3: WAN_IP:995 (POP3S)
SSL Offloading: Disable SSL offloading (since we're using passthrough).
Mode: Set to TCP.
Backend Server: Configure backends to forward traffic to your Mail server.
Configure Backend Servers for Each Protocol:

Go to Services > HAProxy > Backend and click +Add.
Name: Set it to smtp_backend, imap_backend, or pop3_backend based on the protocol.
Mode: Set this to TCP (SSL passthrough).
Server List: Add your Mail server with the respective port for each service (e.g., 192.168.1.10:465 for SMTP, 192.168.1.10:993 for IMAPS).
Repeat this for all the services (IMAP, SMTP, POP3).
This approach forwards the SSL/TLS encrypted traffic for non-HTTP protocols directly to your Mail server, which then handles the encryption and decryption.

Example Configuration:
Frontend for SMTP (port 465):

Bind Address: WAN_IP:465
Mode: TCP
Backend: smtp_backend
Backend for SMTP:

Server: 192.168.1.10:465 (Mail server on internal network)
Frontend for IMAPS (port 993):

Bind Address: WAN_IP:993
Mode: TCP
Backend: imap_backend
Backend for IMAPS:

Server: 192.168.1.10:993
Option 2: Combine HAProxy with stunnel (SSL Termination for All Protocols)
If you want to terminate SSL/TLS for all protocols (HTTP, IMAP, SMTP, POP3) at pfSense and filter the decrypted traffic, you can use stunnel along with HAProxy.

stunnel is a proxy designed to add SSL encryption to existing clients and servers without changes to the application code.
You can use stunnel to decrypt IMAP/POP3/SMTP traffic and forward it to HAProxy or directly to your backend.
Steps:
Install stunnel on pfSense:

Install the stunnel package from System > Package Manager > Available Packages.
Configure stunnel for Mail Protocols:

Set up stunnel to listen on the respective ports (465, 993, 995) and decrypt traffic using your Let's Encrypt certificate.
stunnel will forward the decrypted traffic to your iRedMail server over plain IMAP/POP3/SMTP.
HAProxy for HTTP/HTTPS:

Use HAProxy to handle SSL termination for HTTP/HTTPS as explained earlier.
Firewall Rules:

Configure firewall rules to allow traffic on the mail ports (25, 465, 993, 995, etc.).
Example Configuration Summary (Using SSL Passthrough for Mail Protocols)
Frontend (HTTPS, SMTP, IMAP, POP3):
HTTPS (port 443):

Bind Address: WAN_IP:443
SSL Offloading: Enabled (HAProxy manages SSL for HTTPS).
Backend: Webmail backend on HTTP.
IMAPS (port 993):

Bind Address: WAN_IP:993
SSL Offloading: Disabled (SSL passthrough).
Backend: iRedMail server on port 993.
SMTP (port 465):

Bind Address: WAN_IP:465
SSL Offloading: Disabled (SSL passthrough).
Backend: iRedMail server on port 465.
Backend Servers (Decrypted HTTP, Encrypted Mail):
Backend1 (HTTP for Webmail): 192.168.1.10:80
Backend2 (IMAPS for Mail): 192.168.1.10:993
Backend3 (SMTPS for Mail): 192.168.1.10:465
Conclusion:
To handle a setup where Mail uses multiple protocols over SSL/TLS:

For HTTP/HTTPS (web interfaces), you can use SSL termination at HAProxy to decrypt and filter the traffic at pfSense.
For SMTP, IMAP, and POP3, it’s better to use SSL passthrough to forward encrypted traffic directly to the Mail server, which will handle SSL/TLS for these protocols. Alternatively, you can use stunnel to decrypt these protocols if you want to inspect/filter the traffic.
This combined approach gives you flexibility and control over traffic while maintaining security for all protocols. Let me know if you need further assistance with any specific configuration!

Set the the OpenVPN server to listen on localhost.

Then setup port forwards on both WANs to localhost for the port the OpenVPN traffic is arriving on.

Clients will be able to connect to either WAN and replies will go back correctly.

Additional information

On my cluster where dpinger is failing, I added a floating rule attached to the WAN interface and set it to match. I then set the rule to apply to basically any traffic outbound from the WAN interface and I set it to log connections in the system log. Then I restarted dpinger. Sure enough, dpinger on the secondary is trying to use the CARP address as its source (which the secondary doesn't have right now). So that explains that.

I performed the same test on my cluster where dpinger is working correctly. On that cluster, the rule recorded that the secondary used the interface IP address as its source, NOT the CARP VIP like my misbehaving cluster did. The oubtound NAT rules on both clusters are indeed THE SAME.

As an experiment, I tried changing the NAT rules on the secondary that used localhost / loopback as its source so they natted to the WAN interface IP and not the CARP VIP. No change, secondary still trying to use the CARP VIP that it doesn't have.

When I added the aforementioned NAT rule to the secondary that referenced This Firewall (self) as the source and the destination as any non-private IP address, then dpinger on the secondary starts using the WAN address instead of the WAN CARP VIP. However, this behavior doesn't occur on the primary when we manually failover to make the secondary the master.

The only thing I can tell that's different between the configurations, is that in the cluster that is working, we are running Netgate appliances with pfsense+ 23.09.1. The cluster that is working incorrectly is running pfsense CE 2.7.2 on third party hardware. Possibly a bug in CE that's not present in pfsense+? Regardless of the cause, I can only conclude that dpinger is behaving differently in one of my clusters than it is another.