Load Balance http and picking the correct gateway

[goodi]

I have two external interfaces WAN and DMZ and one internal LAN (172.16.1.1).
WAN is a router port, has the IP 1.1.1.1 and is the default gateway.
DMZ is connected to a switch and has the IP 2.2.2.10.
The DMZ interface also has the virtual IP 2.2.2.15.

I want to load balance incoming traffic on 2.2.2.15 to other servers in the LAN to 172.16.1.5, 172.16.1.6. The traffic flows kindly from 2.2.2.15 to 172.16.1.5 and .6 - but the repsonse packets never reaches their destinies.

I think I understand why: the default LAN GW is 172.16.1.1 - so the traffic from 172.16.1.5 and .6 reaches pfSense, but since the default gateway for the pfSense is 1.1.1.1, this 1.1.1.1 will be used to send the response, but it should use 2.2.2.15 - the incoming DMZ interface. I'm unsure how to solve this problem, anyone got a hint/tip?