Impossible to use Multi-WAN in the same net?

irontec

Hello!

I have the next scenario in a PfSense 2.1.3:

WAN1: 192.168.10.11
WAN2: 192.168.10.12
WAN3: 192.168.10.21
WAN4: 192.168.10.31

I have made some WAN-groups with different priorities with Trigger Level "member down":

G1 (wan1,wan4,wan3)
G2 (wan3, wan2,wan4)
G3 (wan4, wan2,wan3)

My PfSense has

WAN: 192.168.10.101 (default gw 192.168.10.11)
LAN1: 172.16.0.0/24
LAN2: 172.17.0.0/24
LAN3: 172.18.0.0/24

And the Rules for the LANs are:

LAN1 -> all traffic through gateway G1
LAN2 -> all traffic through gateway G2
LAN3 -> all traffic through gateway G3

The Outbound NAT is in automatic mode, and I have seen that all traffic goes via WAN1, always!

If I make few modifications, creating separated NETs for every WAN, and it works perfect:

WAN1: 192.168.10.1
WAN2: 192.168.11.1
WAN3: 192.168.12.1
WAN4: 192.168.13.1

In the pfsense:

Wlan1: 192.168.10.100 (default gw 192.168.10.1)
Wlan2: 192.168.11.100 (default gw 192.168.11.1)
Wlan3: 192.168.12.100 (default gw 192.168.12.1)
Wlan4: 192.168.13.100 (default gw 192.168.13.1)
LAN1: 172.16.0.0/24
LAN2: 172.17.0.0/24
LAN3: 172.18.0.0/24

And the rules the same as in the top. As I said, this works perfect, but I don't know why don't works with the WANs in the same NET, any ideas?

I have created another testing scenario with the next "output" (maybe is a little confusing :P:

| WAN | GW1 (default) | WAN2 | GW2 | LAN1 | LAN2 | NAT | LAN1 – GW | LAN2 – GW | NOTAS |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | default | default | All traffic through GW1 |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | default | GW2 | All traffic from LAN1 through GW1, and from LAN2 through GW2 |
| 10.10.0.247 | 10.10.0.164 DOWN | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | default | GW2 | All traffic from LAN1 timeout, and from LAN2 through GW2 |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 DOWN | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | default | GW2 | All traffic from LAN1 through GW1, and from LAN2 timeout |
| |
| |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | tier12 | tier21 | All traffic from LAN1 through GW1, and from LAN2 through GW2 |
| 10.10.0.247 | 10.10.0.164 DOWN | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | tier12 | tier21 | Ping from LAN1 through GW1 but not HTTP ¿?, traffic from LAN2 through GW2 |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 DOWN | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | tier12 | tier21 | All traffic from LAN1 through GW1, from LAN2 through GW1 |
| |

DOWN -> This GW is down
tier12 -> A GW group Trigger Level "member down", GW1,GW2
tier21 -> A GW group Trigger Level "member down", GW2,GW1

If I change the gateway to be the second one:

| WAN | GW1 | WAN2 | GW2(default) | LAN1 | LAN2 | NAT | LAN1 – GW | LAN2 – GW | NOTAS |
| 10.10.0.247 | 10.10.0.164 DOWN | 10.10.11.2 | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | tier12 | tier21 | All from LAN1 through GW2, traffic from LAN2 through GW2 |
| 10.10.0.247 | 10.10.0.164 | 10.10.11.2 DOWN | 10.10.11.173 | 10.10.3.2 | 10.10.5.2 | automatic | tier12 | tier21 | no traffic!! ?!?!? |

eSpezi

Hi irontec,

you almost explained it yourself:
"don't work with the WANs in the same NET"

pfSense, as any router, uses the first gateway to a network it has in its routing table. Therefor all WAN traffic is put out on WAN1 as it is presumbaly the first gateway. Since all other gateways are in the same subnet, pfS also puts out their traffic on WAN1.
I'm not quite sure if it is achiveable at all to have all gateways on the same subnet, but anyway it would be a lot of senseless effort…

cmb

Each WAN must have a unique subnet and gateway IP, with the exception of PPPoE. You can put a NAT router in between the additional WANs that are on the same network, so your multi-WAN system sees them all as different networks. Less than ideal, but that's the only way that scenario is going to work.