Distributed multiwan with multiple pfsenses

GruensFroeschli

A few evenings ago some friens and i where discussing about networking our homes together with directional WLAN antennas. The idea is to combine our internet bandwith, but eveyone still has his own network.
Since then i've thought about it a lot and came up with some questions i hope someone can answer for me.

1: Everyone of us has a WRAP at home.
Lets assume the
network will look like that.

pf1:
WAN: dynamic Internet
LAN: 192.168.1.1
1to2: 10.10.10.1/30
1to3: 10.10.10.5/30

pf2:
WAN: dynamic Internet
LAN: 192.168.2.1
2to1: 10.10.10.2/30
2to3: 10.10.10.9/30
2to4: 10.10.10.13/30

pf3:
WAN: dynamic Internet
LAN: 192.168.3.1
3to2: 10.10.10.10/30
3to1: 10.10.10.6/30

pf4:
WAN: dynamic Internet
LAN: 192.168.4.1
4to2: 10.10.10.14/30

pf1
|
|
pf2–--- pf3
|
|
pf4

since we have on every pf static link-IP's we can setup multiwan.
now here comes the problem i see:
when pf1 sends 1/3 of it's data directly on it's own WAN, 1/3 to pf2 and 1/3 to pf3, wont they do the same thing with the incomming data?
that pf2 will send:
1/4 to pf3.
1/4 to pf4
1/4 on own WAN
1/4 back to pf1
and pf3 will do the same thing again. send 1/3 of traffic from pf2 to pf1, thus creating a loop.
also that 1/3 of the incomming traffic will be sent back to the origin.

will that happen like i think it will and if yes. can someone think of a way to avoid it?

thanks in advance :)

hoba

This will work if you set it up correctly. You just have to setup correct firewallrules for loadbalancing. The trick is to only use the gatewaypool in rules at interfaces LAN and not at any of the connections coming in from one of the other pfSense's.

The mainproblem that I see is that you have dynamic WANs at all the involved boxes which won't work as you manually have to enter a static gateway for the pool. You can fake this by using simply modemrouters.

*** Typo corrected, Thanks Bill! ***

billm

@hoba:

This will work if you set it up correctly. You just have to setup correct firewallrules for loadbalancing. The trick is to only use the gatewaypool in rules at interfaces wan and not at any of the connections coming in from one of the other pfSense's.

Hoba meant lan here naturally, not wan :) And yes, this could work, good luck debugging it! :)

–Bill

GruensFroeschli

ok i did some testing today with a few old computers i've set PfSense up on.
Right now i'm able to get to the internet with loadbalancing from every "private" lan.

i'v set up firewall rules:

pub. == "not private address-range"
pri. == "private address-range"
1to2 == link-interface-name for connection between pfSense1 and Pfsense2

LAN-Interface
destination: pub. –> gatewaypool WAN & 1to2
destination: pri. --> gatewaypool 1to2

If i find the time i'll setup a test-system in the network-labor at school which will look about like that:

6
/ |
/ |
5----1-----2
\ / \ /
\ / \ /
4------3

If i'm not mistaken i will have to make a lot of different pools.
i'll post them here to have them written down when i'll do them ^^"

PfSense1:
from LAN
dest. = pub. --> pool WAN, 2, 3, 4, 5, 6
dest. = priv. --> pool 2, 3, 4, 5, 6

from 1to2
source = 2, dest. = pub. --> pool WAN, 3, 4, 5, 6
source = 3, dest. = pub. --> pool WAN, 4, 5, 6
source = 4, dest. = pub. --> pool WAN, 3, 5, 6
source = 5, dest. = pub. --> pool WAN, 3, 4, 6
source = 6, dest. = pub. --> pool WAN, 3, 4, 5

source = 2, dest. = priv. --> pool 3, 4, 5, 6
source = 3, dest. = priv. --> pool 4, 5, 6
source = 4, dest. = priv. --> pool 3, 5, 6
source = 5, dest. = priv. --> pool 3, 4, 6
source = 6, dest. = priv. --> pool 3, 4, 5

and so on for every interface on every PfSense.

this would be a lot of work whenever a new PfSense gets added to the system and i think it's still not quite good with how the traffic can take long ways.

edit: moving questions regarding OLSR/RIP