Hi Valhalla:

You can setup static routes en general config, but don't forget to permit that routes in firewall. Also you can force the route to out through the wan interface (put these routes before the loadbalance route which shoul be the last one).

I hope some developer could tell us why the interface is not backing up when the link is recover.

I have time to troubleshoot these issues.

Regards,

Olaf

you  create your firewall rules on the interface that the phones are. each PASS rule should have OPT1 as the gateway. this is policy based routing

A diagram would look like this:

WAN 1 (WAN on pfSense)STATIC IP Phisical NIC 1 _____
                                                                                            ___ pfSense ___ Phisical NIC 0 LAN (LAN on pfSense)         
WAN 2 (Opt 1 on pfSense)STATIC IP Phisical NIC 2___/
                                                                        |
                                                                 Virtual NIC 3
WAN 3 (Opt 2 on pfSense)PPPoE________|

Hope it clears it up

I hope this is possible to acomplish in some way

Static is usually better for business applications where you would have multiple IP addresses assigned to the pfSense to setup multiple internal services. There's no easy way to do multiple external DHCP addresses with one interface.
Also, if you use DNS for remote access, you don't have to worry about dynamic updates.

@coloured:

I use ubuntu for my desktop and have used slackware and gentoo in the past. Is BSD and good as a desktop environment?

Totally off topic, but check out http://www.pcbsd.org/ or http://www.desktopbsd.net/ if you are interested in a more desktop-friendly BSD.

@GruensFroeschli:

Read the whole thread.
You dont need to modify any code.

You just have to edit the config.xml

i have edit the config.xml. But it keep using the WAN gateway, that i set on : interface/WAN/gateway

Thanks very much, you have just clarified for me that in fact what I have done is correct.
I didn't think it would be so easy (what comes in from one wan gateway goes out via the same gateway).

for anyone else who is in the same situation; just setup your multiwaned pfsense box, configure the manual outbound nat rules for both the wan and opt1 (wan2) interfaces to allow any destinations, and source to be the LAN network.
configure firewall rules for your DMZ interface (make sure you tick the not box and specify the LAN subnet in destinations (to disallow any communication between the lan and dmz).
thats pretty much it apart from the port forwarding rules (set under nat inbound) for your services ie: www, smtp etc.

pfsense is definitely powerful and easy to use.

ok possibly not a bug..

things are good on my end now…

and sorry not 1.3, 1.2 http://doc.pfsense.org/index.php/MultiWanVersion1.2

and the only visual needed is:

MultWanRouterBridge1.png_thumb
MultWanRouterBridge1.png

cmb has connected to look at it last time and he ran out of time…...

pretty much * = default = rl1, it registers the ip at rl1 but in/out are rl2.

i'm not running load balancing and there are nothing in the pools

pfsense.jpg
pfsense.jpg_thumb

Well i "think" it should just be possible with the current loadbalancer/failover pools.

If you read this thrad:
http://forum.pfsense.org/index.php/topic,9422.0.html
I describe a way to add gateways that are not in the dropdown list.
The primary entry would be the other side of the fiber-connection.
The secondary entry would be the other side of the VPN connection.
As monitor IP you just set the other end of the respective connection.

Maybe you could just test it first with 2 pfSenses?

Also i'm not sure if there are some problems.
I've read a few threads about problems with failover, that it doesnt fall back after the primary connections comes back.
Although i suspect the people reporting the problem expected that existing states over the backup tunnel get redirected to the main-connection after the mainconnection-comes back up.

Thanks for fast reply! However there is one big problem. I will have a fileserver in LAN and sometimes I need to share files between hosts..
In this article (http://doc.m0n0.ch/handbook/examples-filtered-bridge.html) I have found: "Remember you cannot access hosts on a bridged interface from a NAT'ed interface, so if you do have a LAN interface set up, you won't be able to access the hosts on the bridged interface from the LAN. "
Does it mean that there isn't any other way to have public IP's on hosts and enter LAN hosts?! Some how create rule for ex. 192.168.1.23 will have always 213...23, 192.168.1.24 will have 213...24 public IP and etc… and at the same time I can share files between 192.168.1.23 and 192.168.1.24. I am a little bit confused :)

Let me explain a little bit more:
For example 192.168.1.11-192.168.1.77 range of LAn IP's will have public IPs 213...14 - 213...80 and 192.168.1.101 - 192.168.1.200 will have only one 213...100
and all of them must me under firewall :)
thats all
Any advice?!

@Perry:

I tried a winscp transfer test and get around 30Mbps(300KB)
1.3GHz AMD Athlon (pfsense)–-Gb lan nic------100Mbps switch------Gb nic----3GHz P4(winscp)

Things i would try
Try with a crossover cable to eliminate the switch

I also tested on second pfSense box and get same result… Is this a bug?

If hardware spec is the same you could.
The ifconfig -m xl3 will show the supported media types for the nic. it might be wrong
Replace nic to intel.

@http://www.freebsd.org/cgi/man.cgi?query=dc&sektion=4:

TX underrun – increasing TX threshold  The device generated a
    transmit underrun error while attempting to DMA and transmit a packet.
    This happens if the host is not able to DMA the packet data into the
    NIC's FIFO fast enough.  The driver will dynamically increase the trans-
    mit start threshold so that more data must be DMAed into the FIFO before
    the NIC will start transmitting it onto the wire.

My tests with Windows and IPCop on the same hardware shows that problem is not in NIC. I think that problem is in pfSense NIC driver or SSH daemon. It is not cheap to buy Intel Pro NICs when firewall costs $150 US and is used for 5 workstations. It will be good if more people tests pfSense to know what hardware/software make this problem (or this happens only on my 2 pfSense boxes  :D)

ifconfig on xl3:

# ifconfig -m xl3 xl3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500         options=9 <rxcsum,vlan_mtu>capabilities=49 <rxcsum,vlan_mtu,polling>inet 192.168.155.254 netmask 0xffffff00 broadcast 192.168.155.255         inet6 fe80::250:4ff:fe0b:e464%xl3 prefixlen 64 scopeid 0x4         ether 00:50:04:0b:e4:64         media: Ethernet autoselect (100baseTX <full-duplex>)         status: active         supported media:                 media autoselect                 media 100baseTX mediaopt full-duplex                 media 100baseTX                 media 10baseT/UTP mediaopt full-duplex                 media 10baseT/UTP                 media 100baseTX mediaopt hw-loopback #</full-duplex></rxcsum,vlan_mtu,polling></rxcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast>

@jonnytabpni:

ok that's great!

how do u "unequally disribute" http (port 80) traffic?

when you setup a pool you would normally have one monitor ip from each isp. this will equally distribute connections.

if you want more traffic sent to one isp then you just add more monitor ips for that isp. if you have 2 monitors for isp1 and 1 monitor for isp2 then 2/3 of connections will be for isp1, 1/3 connections for isp2

ok i've been testing some more and after I disconnect one of the WANs it gets marked as offline almost imediately on the load balancer status, but when I do some traceroutes, sometimes it still tries to go through the link that is down.
i'm using the dns servers as monitor ips.

i have a perl script started in background to change the default route to the current available gateway, so when my WAN connection go down it replace the default gateway by the OPT1's gateway, and it seems this cause my problem of monitoring…

it is strange, i dont understand why this modification influence the monitoring system...

Ill give it a try and see what happens

Thanks for the pointer~

This is just in reference to a thread that forum user DotDash assisted somebody sometime back. (http://forum.pfsense.org/index.php/topic,5253.msg31668.html#msg31668)

I just had a few questions following the post.

When creating the NAT 1:1 relationship(at the NAT 1:1 creation screen), would the external IP be the proxy arp ip that was specified in the Virtual IP's? If i wanted to translate this IP to internal address 192.168.1.10, would it look like this:

External Subnet: 123.456.453.789  (do i use /29 or /32?) Originally its /29, but since it was specified as /32 in VIP, im not sure.
Now if the internal subnet needs to use the same subnet as the external, then what should i put in as the internal subnet? Because if i put in 192.168.1.10, wouldnt that be a /24 subnet?

Also i wanted to ask, if this setup translates a VIP into a private /24 address in a LAN, how does this differ from directly assigning an internal computer with a /29 address?.

I hope somebody could provide me with some light in my search for enlightenment :)

Edit: More information added.

You've set the wrong gateway in your LAN rule.
You have to set as gateway the balancing pool. Not a Gateway directly.

Also you've set as monitoring IP's the gateway itself. Dont do that!
Set it to an IP past your next hop or so. As it is right now pfSense cannot detect if a line goes down.

That's what the system's default gateway is for. That's whatever is assigned as the gateway of your WAN interface.

:D :D :D

But you just got me wondering what happens if you set the WAN to static and not as DHCP client, unplug the WAN and reboot.
It might be that then the default gateway stays on WAN.
I have to try when i get home.