Help with Multiple WAN setup.

Pootle · Feb 20, 2007, 2:12 PM

torrent isn't really a good way to test this because torrents get terminally confused if 1 client tries to use 2 different IP addresses - at least that's my experience with uTorrent.

I have set up forwarding on 1 of my WAN connections to forward to pfsense - see below - to run torrents. To test balance, run a couple of speed tests or something similar.

leimrod · Feb 20, 2007, 2:52 PM

see this is my main problem, I can't think of a way of testing the load balancing is actually working because i'd need 3 seperate downloads that would max out each connection.

the way you have it set up does that allow for torrents to be load balanced, or are you just setting up one line as your torrent line?

EDIT: Also is there any settings I need to change in the firewall section to ensure that NAT and the Firewall is turned off?

Pootle · Feb 20, 2007, 3:05 PM

@leimrod:

see this is my main problem, I can't think of a way of testing the load balancing is actually working because i'd need 3 seperate downloads that would max out each connection.

You don't need to max the connection for load balancing to kick in. I find just doing traceroute a few times usually picks up different WAN connections. Anyway what's so hard about running three browser sessions?

the way you have it set up does that allow for torrents to be load balanced, or are you just setting up one line as your torrent line?

I just use 1 connection for torrenting - it gets confused if I leave it load balanced.

Also is there any settings I need to change in the firewall section to ensure that NAT and the Firewall is turned off?

Wow, if you turn off nat and the firewall, I'm not sure you have anything left worth keeping, can't really help you there

hoba · Feb 20, 2007, 3:06 PM

To test the loadbalancer try tracerouting to different locations from a client behind pfSense like:
tracert google.com
tracert yahoo.com
tracert lycos.com
…

You should see the traffic taking different routes through your WANs. You also can visit a page with lot's of external images and open status>traffic graph for each wan in different windows. You should see traffic going through all WANs while browsing.

If you want to shut down NAT go to firewall>nat, outbound and enable advanced outbound NAT. Then delete all autocreated rules at the bottom, save and apply.

Btw, if you shut down the firewall you won't be able to use policybased routing or loadbalancing as it is handled by pf too. Create pass rules for all the traffic with appropriate pools and gateways instead.

leimrod · Feb 20, 2007, 3:43 PM

@hoba:

To test the loadbalancer try tracerouting to different locations from a client behind pfSense like:
tracert google.com
tracert yahoo.com
tracert lycos.com
…

You should see the traffic taking different routes through your WANs. You also can visit a page with lot's of external images and open status>traffic graph for each wan in different windows. You should see traffic going through all WANs while browsing.

If you want to shut down NAT go to firewall>nat, outbound and enable advanced outbound NAT. Then delete all autocreated rules at the bottom, save and apply.

Btw, if you shut down the firewall you won't be able to use policybased routing or loadbalancing as it is handled by pf too. Create pass rules for all the traffic with appropriate pools and gateways instead.

Thanks, that has given me plenty food for thought. I never thought to do a tracert, but it makes sense. Can I ask why it will use different routes if it doesn't have to? I thought it would only change over to the second connection once the first one was maxed out.

Also, do you have any links to tutorials on how to set up pass rules for the load balancer? What are the benefits of policybased routing and loadbalancing?

hoba · Feb 20, 2007, 4:21 PM

It distributes each new connections roundrobin to all poolmembers in loadbalancer mode. Loadbalancing won't work for some special applications like https for example (as you are hopping between IPs) so you want to use policybasedrouting for this. You should create a failoverpool and use this as gateway in your firewallrules for this kind of traffic.

http://doc.pfsense.org/index.php/Multi-Wan/Load-Balancing has quite some extensive information about multiwan setups.

databeestje · Feb 20, 2007, 5:29 PM

the default any LAN rule matches anything, including icmp/traceroute.
To make it even a better example, a traceroute from a LAN pc to each monitor IP should use the corresponding wan connection.

I balance with a 8/1 Dsl and a 8/8 fiber. I want all email traffic to use the fiber line because it has more bandwidth. So i make sure it uses that one, and fails over to the DSL if that one does not work. (ordering is important)

For all the webtraffic (which is mostly downstream) I match that with a rule that refers to my load balanced pool.

leimrod · Feb 21, 2007, 11:15 AM

@databeestje:

the default any LAN rule matches anything, including icmp/traceroute.
To make it even a better example, a traceroute from a LAN pc to each monitor IP should use the corresponding wan connection.

I balance with a 8/1 Dsl and a 8/8 fiber. I want all email traffic to use the fiber line because it has more bandwidth. So i make sure it uses that one, and fails over to the DSL if that one does not work. (ordering is important)

For all the webtraffic (which is mostly downstream) I match that with a rule that refers to my load balanced pool.

Could you post a screenshot of this setup?

Also databeestje I went through tutorial on the wiki, but it's for a DHCP setup, its hard for me to distinguish what settings are pertaining to DHCP and what ones I need for Static IPs. Some advice on what sections of it can be applied to a static IP setup would be appreciated.

hoba · Feb 21, 2007, 12:47 PM

Have a look at http://doc.pfsense.org/index.php/Multi-Wan/Load-Balancing

leimrod · Feb 21, 2007, 1:14 PM

@hoba:

Have a look at http://doc.pfsense.org/index.php/Multi-Wan/Load-Balancing

Hi hoba, yeha I did have a look at that link but its a tutorial for a DHCP setup which is not what i'm setting up. Would the load balancer and firewall rules be the same for static IPs as they would for DHCP?

hoba · Feb 21, 2007, 1:58 PM

Yes, the only difference is the interface>wan and interface>optwan configuration. You configure them static there instead of DHCP. The rest is exactly the same.

leimrod · Feb 26, 2007, 1:30 PM

Ok thanks everyone for the help. I think i'm nearly there. I'm having one problem though with our wireless broadband connection.

The thing is it connects to an antenna on the roof, then connects from their to the bas station. There is no gateway interface to change any of the connection settings.

We connect using (examples):
IP: 89.16.71.84
GW: 89.16.71.1
DNS1: 213.168.233.7
DNS2: 213.168.233.6

Now in my Opt1 settings, i'm setting it as static and inputting IP and GW shown above, but there is no option to add the DNS IP addresses? I can't connect to the internet on this connection? In my status/interfaces settings (i've attached a screenshot) it shows that is isn't picking up the DNS servers which i'm guessing is the reason I have no internet connection. Is there a way to tell pfSense what the DNS servers are for a specific connection?

If not, is there a possible solution to my problem?

hoba · Feb 26, 2007, 6:31 PM

At system>general uncheck the "allow to override DNSservers by DHCP/PPPOE" box and add your DNS Servers there. For Redundancy you should use one from WAN and one from OPTWAN. You'll need a static route for the DNS-server at OPTWAN to be routed out through OPTWAN.

leimrod · Feb 27, 2007, 10:38 AM

what do I do if I have 3 WANs? I'm going to be load balancing 2 WAN connections from the same ISP and a 3rd wireless connection. How can I set up the DNS for this?

One thing also, the 2 ISP connections share the same DNS addresses, is this going to be an issue when load balancing them?

hoba · Feb 27, 2007, 12:42 PM

We are working on making the dns server fields more multiwanfriendly atm (adding more fields and adding a dropdown next to it to dtermine which connection they belong to to automatically create the needed routes nbehind the scenes). However in the meantime setup the dns like I said earlier. You can edit the config.xml to hold more DNS items if you really expect 2 links to fail simultaneously.

leimrod · Feb 27, 2007, 12:43 PM

Its not that I expect them to fail, more will leaving out any of the DNS addresses affect the correct load balancing of the 3 lines?

leimrod · Feb 28, 2007, 3:58 PM

Ok well i've gotten both the wireless line and one of the router lines set up and load balanced. I now need to setup failover. Has the option to set up failover been removed from the load balancing section? Is it located somewhere else?

Attached is a screenshot of the load balancing configuration screen in my version of pfSense, and below is the screenshot from the tutorial I was following to set up load balancing.

http://doc.pfsense.org/index.php/Image:EditPool.jpg

You can see there is a section called "behaviour" that has been removed or is not available in the screenshot i've attached.

hoba · Feb 28, 2007, 10:43 PM

The failovercode can only be found in the latest snapshots. Please upgrade. You'll find them at http://snapshots.pfsense.com/FreeBSD6/RELENG_1/

leimrod · Mar 1, 2007, 11:02 AM

Ok, i'll update to the latest snapshot. I'm having some other weird problem now. When I set the wireless internet line up as the main connection (WAN) I have internet access. When I set the main routered connection as the WAN connection I have internet access, but when I set them up together and load balance them my internet connection is intermittant, sometimes it connects sometimes it doesn't. I'll have pages where they'll begin to load then half the pictures won't display.

I've attached the connection logs below, the ICE connection is my wireless connection. Maybe you can explain whats going on with my LAN connection, it seems very odd.

leimrod · Mar 1, 2007, 11:59 AM

Ok I upgraded to the latest release. I'm not getting this error in my status>interfaces section for the wireless ICE line, it says "Status: no carrier". Do you have any information on why it would be giving this error?