Pfsense as a router and default gateway with multiwan

bravo_prochu

No, i did not;

Still searching…
I guess it's impossible on pfsense;

Right now i'm considering changing my whole networ into bridge and vlans ...or drop pfsense.

Reiner030

No, i did not;

Still searching…
I guess it's impossible on pfsense;

Right now i'm considering changing my whole networ into bridge and vlans ...or drop pfsense.

mmh… do you both used the search function of this board with the right keywords? ;) just ready some posts "before"

@Nachtfalke:

So if you put all your 4 WAN connections into a group and all gateways have the same Tier then these do loadbalancing and if one GW hoes down the other 3 WAN connectiongs do loadbalancing.

If you use "Sticky connections" on SYSTEM –> ADVANCED then this one group will be enough.
If you do not use sticky connections you should create another group which will handly traffic for https and other secure protocols because https and other secure connections do not like loadbalancing because the IP switches.

I guess that your routing problems comes also from non-sticky loadbalancing connections…

BTW: if you want monitor your sub-networks there was found a "fix" for it:
http://forum.pfsense.org/index.php/topic,69443.0.html

Bests

basitkhan

I have something similar configuration but i have layer3 switch instead of your routers and pfsense is working as gateway with 5 wans,
each and everything is going fine.

In interfaces LAN, do not select any gateway just leave it blank.
hope this will fix your problem!

heper

~~why do you have 2 pfsense machines in the first place ? unless i'm missing something (i generally do), i don't see any reason for that pfsense between the ppoe-pfsense and the lans ?~~

After closer inspection i concluded i have no clue what you are trying to accomplish by putting 2 routers behind each other, and then using 4 additional routers lateral of each other, pointing to the X.X.10.X gateway (router2).

So i'm fairly confident it is possible to make it work somehow … we probably need more info to sort out what goes wrong.
If it works for 1 WAN, but does not for 2 WANs then you should probably look at a NAT issue. => disable automatic NAT (AON)
and create your own nat rules as needed.

bravo_prochu

Hello

That is how my network works. I use routers (with disabled nat function) and pfSense works as a 'NAT serwer' (in a future dhcp serwer too)
Everything works great when I use only one (default gateway);

To make it multiWAN works I need to specify default gateway in firewall rules and thats the problem, because when I point 'mulitWAN' gateway - there are no traffic (out) from static routes. [with static routes - there are mulit LAN gateways]

Disabling AON, to do what ?

johnpoz

"At pfSense I added a gateway (192.168.0.2/24) and traffic between networks and pfSense works great when I use only one WAN int."

Why would you set a gateway, that is not a WAN interface its a LAN interface - you would not set a gateway, you would only setup routes.

bravo_prochu

Did You ever try to set up static routes on pfsense ?

johnpoz

Have I? Yes quite simple. You don't set gateways on LAN interfaces.

If you would use a non 192.168 range between pfsense and your first router pfsense route table would be quite simple with one entry for 192.168.0.0/16

Its seems you have a cluster of a setup there that seems way more complicated than it needs to be.

LAN rules looks like this for every single IP of my whole networks ie: (i use aliases)

| 192.168.32.115/24 | port: * | dest: * | gateway: DEFAULT [use in/out for that IP]

| 192.168.40.235/24 | port: * | dest: * | gateway: DEFAULT [use in/out for that IP]

| 192.168.50.25/24 | port: * | dest: * | gateway: DEFAULT [use in/out for that IP]

| 192.168.10.95/24 | port: * | dest: * | gateway: DEFAULT [use in/out for that IP]

What??? Every single IP has its own entry - WTF were you thinking?

bravo_prochu

Every single IP has its own rule entry, because I want to control bandwidth of every single alias-ip;
I use limiters - for every IP - as many limiters as aliases (ip)

How can I set static routes to routers without seting up geteways ?
(i need a remote connection to devices behind those routers)

I'm newbie so if You can, please, share an idea how to do it different way..

johnpoz

"How can I set static routes to routers without seting up geteways ?
(i need a remote connection to devices behind those routers)"

Yeah you set the routes here - see attached. And yes in the routes you pick a gateway.. But you do not apply that gateway to your lan interface. If you apply the gateway to the interface directly then pfsense thinks thats a wan interface.

routespfsense.png_thumb

bravo_prochu

I still don't understand: 'But you do not apply that gateway to your lan interface' ?
First i needed to setup lan gateway (see attached - staticRoutes02) to pick it up to the destinated network (see attached - staticRoutes01)

staticRoutes01.jpg_thumb

staticRoutes02.jpg_thumb

johnpoz

But is that applied to your interface?

Look - here is dummy gateway I created for network 10.0.0.0/24 – it uses my DMZ interface.. But notice on the actual dmz interface there is NO gateway set!!

routestatic.png_thumb
$interfracedmz.png$
$interfracedmz.png_thumb$

bravo_prochu

Gateway is on my LAN interface… so there is no 'gateway' for LAN int..

What does it change ? Because I still don't get it..

I would like to use multiwan tutorial, so when I specify default gateway in rules ('multiWAN' = PPPoE1 [tier1] and PPPoE2 [tier1]) there is no traffic to routers..
(on attachement i don't have created multiWan gateway yet but i already tried it)

staticRoutes03.jpg_thumb

johnpoz

"What does it change ? Because I still don't get it.."

If there is a gateway on the interface - pfsense thinks its a WAN, and will auto nat it for starters.

bravo_prochu

There is no gateway on LAN interface, but still when I change default gateway on FIREWALL-RULES-LAN…alias..- gateway to multiWAN instead of default - there are no traffic

staticRoutes04.jpg_thumb

phil.davis

Not sure where you are up to, but here are the general principles when you have some internal networks and multi-WAN with gateway groups…

Define a gateway for each other internal router (gateway) that leads to an internal network that is NOT directly connected to pfSense.
Define a static route to each of these internal networks pointing to the correct internal gateway.
Add gateway group(s) that group together your real public WANs in whatever tiers you wish.
First add rules on LAN to pass traffic to the the internal networks - without specifying any gateway in the rules - the packets will be passed to the ordinary routing table and the static route/s you defined will get them to their destination.
If you have VPN site-to-site links to other offices, these also need to use the ordinary routing table - put pass rules for traffic to subnets that are across the VPN, and let the ordinary routing table and VPN software deal with it.
Further down in the LAN rule list, put rules that send traffic to particular gateway groups (e.g. near or at the end you might commonly have a rule that passes all protocols source LANnet destination any gateway LoadBalanceGWG - to load balance everything that did not match any previous special rule)
Some screen shots attached of one of my setups, with a test Firebox internally that goes to various test subnets in 10.99.0.0/16. The LAN rule passing INF_Subnets to INF_Subnets matches this Firebox traffic (among other stuff). INF_Subnets is an alias that contains all my internal subnets, local to pfSense, at the same office and across VPNs at other offices. This makes it easy to write 1 pass rule that lets all this internal private traffic pass through to the ordinary routing table, before any rules that pump traffic into a gateway or gateway group.

Gateways.png_thumb

Static-Route.png_thumb

GWGroups.png_thumb

LAN-Rules.png_thumb

johnpoz

^exactly – needs to be turned into a doc.. If I find time tmrw at work (its been slow normally as we get closer to holidays) I will do just that ;)

bravo_prochu

Thanks for Your answers Guys !

Still have some questions
I use limitters for every aliases (clients) (att. staticRoutes06) on my network; My LAN rules looks like (att. staticRoutes05)
Instead of 'Private_devices' (block private devices to outside DNSserver) can I use all local subnets ??
Do 'Inf_Subnets' have LAN subnet included ? (pfsense box IP) ?

Can You help me with this ?

How to drop any other stations to the Internet (but aliases..) and still have local connections to the lan routers/Access points using multiWAN..

Can You point me a basic firewall isolation for that kind of policy ?

staticRoutes05.jpg_thumb

staticRoutes06.jpg_thumb

phil.davis

Instead of 'Private_devices' (block private devices to outside DNSserver) can I use all local subnets ??
Do 'Inf_Subnets' have LAN subnet included ? (pfsense box IP) ?

I have lots of different LANs, so I made some aliases like:
'Private_Devices' = IP address ranges that I give out to private devices on the LAN (like people's smart phones…)
'Inf_Subnets' = all the private subnets in my private extended network (includes LAN subnet and others)
Then I can make easy rules to block or pass traffic to or from these groups of IP address subnets.
If you just have 1 LAN subnet, then it is easy to just make rules like:
Block UDP+TCP source LAN subnet, destination !LAN IP port DNS (53)
That stops LAN clients getting DNS from anywhere else outside pfSense LAN.

bravo_prochu

Thanks for an explanation;
So, i should use this rule at the top of my others ?
then, the 'inf_subnets' and then - my 'aliases rules' ?

As I wrote before I use one rule for one alias - to use bandwidth per IP tutorials; Right now my rule is - alias to 'any' [any port], but when first set to exclude 53 port to any devices on LAN subnet then i should use only [alias to any - DNS port (with multiWAN gateway)] rule ?

staticRoutes07.jpg_thumb