Multi-WAN, asymmetric routing and policy routing for local traffic

[kukoarmas]

Hi there!

I have a configuration with 2 WAN links to different providers (different public IP) and haproxy as a reverse proxy for internal services (in the pfSense host)
I would like to be able to access the haproxy services from the internet using both WANs, so I can (kind of) load balance incoming traffic

All incoming connections through the secondary WAN (the one that is NOT the default gateway) doesn't work because all responses go out through the default gateway

To make it more "visual". In the following setup, My default gateway is WAN1

–--- WAN1 ----- IPWAN1 |
Internet                                              |  pfSense
              ----- WAN2 ----- IPWAN2 |

If I try to open a TCP connection to the external IP of WAN2 (IPWAN2), the response packet gets routed through WAN1, and of course the TCP connection is not established because the SYN+ACK comes from a different IP (IPWAN1)

I've experienced this "asymmetric routing" also in Linux (I recently migrated from Linux firewalls to pfSense). In Linux I used to create different routing tables for each WAN and create policy routing rules to use each table depending on the source IP

I've read that with FreeBSD I can also use multiple routing tables but it's not enabled by default.
I've tried to create policy routing rules to force the traffic to be routed through the correct WAN but it doesn't work because (as I read) locally generated traffic does not pass through PF, and so no rule is applied to this traffic

Is there an easier way to do this? If not, is there any plan to include this multi routing table functionality in pfSense?

[kukoarmas]

Ok, I've found the solution (thanks @pruiz)

I've read in many messages that it was not possible, and following all the guides I found it didn't work. Maybe all I found only applied to old versions.

This is how it worked:

• Define the gateway for each wan interface in the interface configuration
• Define the rules on each WAN interface. It can also be a floating rule, but in that case it has to be defined as a IN direction rule and applied ONLY ON ONE WAN INTERFACE. So, you will have to define a rule for each WAN, what makes useless using floating rules…
• DO NOT go to advanced options and define the gateway, the reply-to will be correctly defined based on that WAN's gateway

[GTA_doum]

@kukoarmas Thanks a bundle for this post! Putting a gateway on the WAN static IP interface fixed the issue I was having with asymmetrical traffic! The other WAN interface is DHCP, so no need to put a gateway, it gets one by itself.
I searched for days for a resolution and this post finally helped me understand what was happening and how to fix it.