Multi WAN, Multi lan local routing problem

mrvanity

Hello.
I have a 2.3.2 installation with 4 WAN interfaces and 3 LAN
The interfaces are
WAN1= PPPoE VDSL (ISP1)
WAN2= PPPoE VDSL (ISP2 Static Public IP)
WAN3= PPPoE VDSL (ISP3 Static Public IP)
WAN4= Static IPv4 (Its another VDSL from ISP1 but i have the problem mentioned here https://forum.pfsense.org/index.php?topic=85623.0 so i switched to static IPv4)

I have created a balancer with tier1 for all lines, and a couple of failover groups for https and VPN

The lan interfaces are
LAN1= 192.168.16.0/21
LAN2= 192.168.30.0/23
LAN3= 10.10.10.0/24

Through firewall rules for every LAN interface i have configured that all traffic from those interfaces should have the balancer as gateway. (except https and vpn of course in which the traffic travels through the according failover groups)

This setup works really well.

The problem arised when i had to route traffic between my lan interfaces. it didnt work.
Ex. from LAN1= 192.168.16.0/21 to LAN2= 192.168.30.0/23

I have read that the is no need for additional static routes for "local" subnets in pfsense.
Also i have read that the firewall rules overcome the static or the default routes in the routing table.

Testing this further i tracerouted a packet from a host in LAN1= 192.168.16.0/21 to a host in LAN2= 192.168.30.0/23
The first hop was public addresses that was i proof that pfsense tried to forward traffic using the balancer.
Disabling the firewall rules in the lan interfaces about * Ipv4 -> Balancer, it worked like a charm.

Reading that the firewall rules are applied from up to bottom i created a rule in ex LAN1 interface BEFORE the balancer rule

Protocol 	Source 		Port 	Destination 	Port 	Gateway 	Queue 	Schedule 	Description 	Actions
IPv4 TCP 	LAN1 net 	* 	LAN2 net 		* 		* 			none 	  	
IPv4 * 		LAN1 net 	* 		* 	* 	Balancer 	none

accordingly to LAN2 interface BEFORE the balancer rule.
Cleared the states and tried again.
No luck.
I even tried adding the rules after the balancer rule, but again without result.
It only works if i disable the balancer rule.

What should i do, to keep the balancer rule, but also have traffic between my interfaces?
My regards.

dotdash

If you are testing by trying to ping, that's not going to work if your allow rule between the lans only allows TCP. Try switching the LAN1 to LAN2 rule to 'protocol any' instead of TCP.

mrvanity

Hello.
Thank you for your answer.
I am pretty sure i have tested * instead of TCP only.
I will test again tomorrow.
The rule for intra traffic should be before the balancer rule.Correct?

P.S. Do you have any idea for https://forum.pfsense.org/index.php?topic=85623.0?

dotdash

The order is correct. Make sure you do the same thing on LAN2 so the LAN2 traffic can pass to LAN1.