yeah boss, got that!

Everything is good now. Not too hard at all.

Downtime can be seen with the quality graph under "RRD graphs).
If you want it more specifically you will need to set up a network monitoring solution (like nagios) on a different system

UP/DOWN restriction depending on user should afaik be possible with some hacking with FreeRADIUS and the Captive Portal.
Not sure what the status is. Read more in the Captive Portal / packages subforum.

squid is not multiWAN capable.

Look at the available packages if you want to monitor who's using how much bandwidth.

…. I find it quite daring for you to request a "complete tutorial".
Write one yourself....

Just for the semantics:
The rules you posted are NOT the routing rules.
These are firewall rules.
It just so happens that you force in this firewall rules a gateway other than the default gateway.
This is called "policy routing" and has nothing to do with the routing table.

Well, how would you get load balancing when you've got one of the WANs disconnected? I don't understand what you mean…

Can you please post screenshots of your load balancer configuration, LAN firewall rules and outgoing NAT rules along with a description of exactly what isn't working?

PPTP VPN. You need to send PPTP (port 1723) and GRE out over the same interface or the connection cannot be established and if it is, will eventually time out and drop when GRE and PPTP are sent out over different WAN interfaces.

Actually the reason the other rules, HTTPS, POP3, and SSH are first as they are the protocols that don't support load balancing. Load Balancing rule is a catch all rule with buit in failover in itself.

you perhap can try traffic shaper in conjuction with Load Balancing.

Any thoughts?

no internal way in pfSense to just force traffic from one interface to another?

in the load balancing pool, add more instance of the WAN1 or WAN2 to equal the ratio you want.

The pool should be like this:

WAN1
WAN2

Now that should be 50/50

To get 75/25 ratio you do this to the pool:

WAN1
WAN1
WAN1
WAN2

That all

The Load Balancing Rule will take care of failover. But there are ports which dont like load balancing such as
POP3 and SMTP when used using an email client gives authentication error. In order to solve this mystery you need to create Failover rules and redirect these ports traffics through failover Rule. So that there should not be any authentication error.

As the Bit Torrent uses a specifc protocol . IF want it to be routed through one WAN connection you can set a rule in the LAN firewall rules and let go all your bi torrent trafffic thorugh one WAN connection. In that case you will not be able to use the FailOver feature.
In case you want to use the failover feature. IF this WAN connection goes down it will not going to shift on the Other WAN.

Whenever you use a MultiWAN Setup You create a New Gateway IN Load Balacing Services.
So all the request from the client side has to go through that new Gateway.
You will set in Firewall Menu–> Rules--> LAN Tab

You're right, this won't work if the two WAN gateways are the same. Your solution of adding a NAT gateway in between one of them works.

To verify that the static route is working, you can run a traceroute(tracert windows) to something on the other network. The first hop should be pfSense and the second hop should be the 172.16.x.1 device.

Yep it was all my fault. I got confused about the order of the rules on each interface, (I have a total of 5 physical interfaces and 3 VLAN interfaces). So I removed all the extra interfaces and did it one WAN connection and one VLAN at a time, using my laptop and spare system to test internet connectivity. I'll post a example of the configuration when I get a spare moment. Also I must say this was actually really simple and provides more functionality than any other router/firewall solution. Thanks for the help.

finally…..
the issue was with BT. when we signed on, they gave us a username and password. they then followed this up with a different username and password, so we had 2 acccounts doh!
when we tried to log on, the orginal account wouldn't log on so we tried the 2nd account and it worked.
we then asked for a static ip which they said was working etc. but we couldn't get it to route etc.
so after hours on the phone and getting up to top level support, turns out that we shouldn't have been able to use the 2nd account. in fact, they couldn't believe we were actually logged on using an account that shouldn't be in use.
so we then logged on with the original account (which we couldn't do originally) and bang.... static ip's are now matched etc.
BT........ !!!!!

Thank you. I have nothing there at the moment so these 2 rules are not needed now. Perhaps when I need to established DMZ or server then I would look back into them.

gotcha. thanks.

I am using an Atom 230 board with a PCI Intel 82543EI-based card (I think it's an IBM branded low-profile server NIC but I can't recall exactly). I'm really not sure how network performance scales with adding cores in FreeBSD (last I heard, it doesn't make much difference), but PCIe will get you an incremental gain at most. I still doubt you'll be able to do much better than 1Gbps with your hardware, if it can even reach that. There are many factors though, it really depends on your traffic profile, I just think it's unlikely.