@GeorgeM: @zerokool: yea I cant get the stupid "default gateway" box to uncheck. I've been exploring 2.1 on a spare machine (2.0.3 is what's on the live one) and I noticed this problem myself when I got into setting up my dual WANs. In my case I want certain kinds of traffic to go over specific WAN links and that not working would be a real headache. (e.g., cloud backup needs to go over the link with the biggest upstream bandwidth.) Ordinary client traffic (like a big backup to cloud) is easily directed to the WAN (or group of WANs) that you want it to use, by specifying the gateway or gateway group in the rules. It is only traffic originated from pfSense itself that is tricky to direct.

In pfSense/FreeBSD (like other OSes I can think of), a gateway is the IP address of another box to which the computer can send data packets destined for some group of other IP subnets (or for all). The interfaces that have gateways are dealt with as WAN-type, the gateway is supposed to be on that WAN subnet on a different machine (ISPs router, some other box on the way to the internet. Define a gateway on each WAN pointing to the upstream IP address of the path to the internet. Remove any gateways from LAN. Set whichever WAN gateway you prefer to be the default route. Add policy-routing rules (rules that pass traffic and send it into a gateway). On LAN1: Pass protocol any source LAN1net destination any gateway WAN1GW On LAN2: Pass protocol any source LAN2net destination any gateway WAN2GW If you want failover, then make gateway groups with the required order of preference and feed the traffic into the gateway groups.

It was a lease line VPN.. managed by a provider..

There wouldn't be anything exposed over SNMP for the gateway status, at least not currently. The closest you could get would be the ifOperStatus of the WANs but there are plenty of situations where the interfaces are up but the gateways are down/unreachable.

I was get a solution, please follow the forum link below. https://forum.pfsense.org/index.php/topic,66822.0.html or https://forum.pfsense.org/index.php/topic,60977.0.html

Any suggestion guys?  :(

Did you create a vlan1 on the PfSense Side? You will need to do this that way your vlan 1 traffic will be tagged with Vlan 1 as it goes across the trunk port. If you set the switchport on your 2900 switch to trunk make sure that you are using dot1q encapsulation (switchport trunk encapsulation dot1q). The native VLAN should be 1 but you can run the switchport trunk native vlan 1 just to be sure. This should work, if not some screen shots of your pfsense assignment configs might help.

That should work fine. Now that 2.1-RELEASE has been out a while people (like me) won't remember what little bugs/tweaks there might have been in 2.0.3, so I would recommend "going fpr broke" and upgrade to 2.1-RELEASE. Then post your gateways, gateway groups, any special NAT rules you think you needed and firewall rules and we can help sort it out.

The gwlb.inc code change I made at https://github.com/phil-davis/pfsense/commit/4a5bce90cee8c865dd4e3d0b440bfc14e1d64086 should let you do all the setup even while none of the WAN gateways have actually been connected or got an IP address by DHCP. The pull request has been sitting for a while - I guess with other things in development the devs have not had time to be sure there are no side-effects on other things, before committing the change.

You are doing right. In that place in firewall rules you put a netmask (CIDR bit count) to indicate the range of IP addresses you mean. It is not the netmask that the client would have on the LAN. To specify a single IP address it is 192.168.0.x/32 You can also specify "networks" that can cover parts of your LAN, OPT1 etc e.g.: 192.168.0.192/26 matches 192-255 (64 addresses in /26) 192.168.0.160/28 matches 160-175 (16 addresses in /28)

WAN2 was the one that went down.  I know it's a little weird, but I have WAN as my default gateway, yet WAN2 is the one where most traffic goes out, due to the weight & tier settings in the Gateway group below.  That aspect of it is working the way I want/expect it to. Here's my settings: WAN connection: Triple bonded T1s - 4.5Mbps up & down WAN2 connection: Business class Cable modem - 50Mbps down/5Mbps up WAN Gateway Settings: ------------------------------- Address Family IPv4 Gateway IP address 208.x.x.169 Default Gateway Checked Disable Gateway Monitoring Not Checked Monitor IP Weight 1 WAN2 Gateway Settings: ------------------------------- Address Family IPv4 Gateway IP address 71.x.x.17 Default Gateway Not Checked Disable Gateway Monitoring Not Checked Monitor IP 8.8.8.8 Weight 5 Gateway Group MultiWANFailover Settings: ------------------------------------------------------ Gateway Priority WANGW Tier 2 wan_vip3 - 208.x.x.170 WAN2GW Tier 1 opt1_vip4 - 71.x.x.18 Trigger Level Member Down As far as floating WAN rules, I'm not sure what you mean, all that's in my floating firewall rules is autogenerated rules for my VOIP & P2P queues…  I haven't made any changes to my firewall rules in some time, and it used to work fine.  Do you want a complete posting of my firewall rules?

Not sure about the up/down stuff, but for the destination you need to make an alias of your internal networks, make a rule before your gateway failover one with the destination being that alias, and pass all traffic. Trying to pass traffic to your internal network through your failover group will usually not work out well.

Solved. After getting up to level 4 support at Time Warner Cable (Road Runner), they agreed that it was a very strange problem.  During the last of several tests, I connected my computer again directly to the cable modem.  The computer could not connect to pfsense.org.  The traceroute stopped at the first network point. They said they would call back.  They have not.  But I just found that the problem is fixed.  pfsense.org now comes up.  The traceroute now stretches beyond their first network router.  I hope I'll hear an explanation.

You can find the port channel (LAG) configuration under interfaces. Just go to assign and create your LAG there.

creating LAGG groups to your switches in both Office and home might give you better throughput when routing from Office to Home networks but other than that I'm not sure there is much more that you can do. Routing accross interfaces is probably CPU limited. What kind of computer are using for your PfSense firewall? Do you get a public or private IP from your PPPoE modem?