yes that makes more sense now the speed tests indicate that both connections are being used however file downloads only max out one connection i assume this is because the file server can't have two connection from two different IPs & gateways to download one file thanks for your help

If you want to simulate having servers available on what looks like the outside "public" internet, then just port forward from the WAN IP on each VM into the server/s. Then add some host overrides in DNS to specify names that point to the WAN IP where the server sits behind. Then you can access the "remote" server by name, which goes to the WAN IP and is forwarded in to the test server. If you want to test a private site-to-site VPN, then simply setup an OpenVPN site-to-site link with OpenVPN server listening on WAN1 192.168.1.9 and OpenVPN client coming from WAN2 192.168.1.12 - pick some private IP tunnel network for the VPN hop. Put a pass rule on WAN1 to let the client in to the port the OpenVPN server is listening on… Then traffic can flow back and forth as if between offices.

Head office does not have a route back to Satellite Office. In the Head Office Routing Table box, there is only a route back to 172.20.20.0/24 listed. Add another route for 172.20.21.0/24. And you may need a pass rule at Head Office allowing traffic in from 172.20.21.0/24.

It was an error/misconfiguration to allow it before. The input validation is correct now. You do not need nor do you want that route manually added. By having an IP in the subnet the OS gets a route to the subnet automatically for that link. A static route can/will break that.

Excellent!  Care to share the resolution?

Your outbound NAT is either manually configured to do that, or you wrongly have a gateway specified under that VLAN interface (Interface>VLANname) in which case the automatic outbound NAT will do that.

@Derelict: Something is not as it is being presented then. Set the rule for packets with a 10.0.0.101 source address to log and check the firewall logs. If nothing shows up, do the same with the default any any rule. Check the state table and state summary to see what's being created. After logging it and checking the current traffic graphs, I have confirmed that it is working properly! Thanks!

i followed this walkthrough… http://pfsensesetup.com/pfsense-load-balancing-part-two/ and this https://doc.pfsense.org/index.php/Multi-WAN_2.0

Hi, sounds mostly like our setup - we only added DMZ network (we have one internal VLAN and DMZ VLAN over WLAN Bridge)       ISP-line1                    ISP-line2         |    |                      |    |        (transfer-networks IPv4/IPv6 fixed)   gw1-jws1  gw2-jws1          gw1-zws1  gw2-zws2         |    |                      |    |       [DMZ ----------------------------- DMZ]      (public static IPv4 / IPv6 networks - here BGP announced)         |    |                      |    |   fw1-jws1  fw2-jws1          fw1-zws1  fw2-zws2         |    |                      |    |        (public NAT for IPv4 servers / public IPv6 networks wanted)       [LANs JWS1]                  [LANs ZWS8] on the gw side we use OpenBGPd … on fw side we use Quagga OSPF. I tried also setup Quagga OSPFd on gw side to get full automatic default routing setting on firewalls but OpenGBPd and Quagga on same servers would conflict :( In http://forum.pfsense.org/index.php/topic,62277.msg336528.html#msg336528 I helped already for an "easy" 2 peer setup. BGPd needs a full mesh setup of all peers... So if you have one firewall each for your office each firewall must communicate to both ISP BGPs and your opposite firewall and the config file need additionel iBGP peer. If you have a setup like ours then you need also a full mesh between your 4 firewalls  and the config file gets "much more prettier" ...  ;) If you have no /24 (single / on both sides each) then you can't publically announce your networks and it would be much easier for you to use OSPF to set the outgoing routes... But I didn't know if your ISP likes offer you OSPF. CISCO/Juniper router supports it but I didn't know if your ISP has a change to filter incoming OSPF packets like he can with BGP. I think he won't accept from you routing offer of Google / Youtube network range for instance ;)

@victorhooi: Is there some way in pfSense to easily setup some kind of routing rules so that the VoIP is locked specifically to one WAN connection, and other traffic goes through a load-balanced gateway group? Can you do this by automatically tagging the VoIP traffic somehow, or do you need to setup VLANs, or specific IP ranges? Cheers, Victor that should be one of the easy tasks… you can setup 2 different "modes" use gw group for loadbalancing of normal traffic, use only gateway x for VoIP traffic create 2 gateway groups, one for loadbalancing (gw x/y same tier1) one for VoIP failover (gw x as tier1, gw y as tier2) Then you need to setup LAN firewall rules which fits your VoIP traffic and your other traffic… You can detect your VoIP traffic in different ways. a) all traffic which goes to IP a.b.c.d / network a.b.c.d/x b) all traffic which is UPD, Port 5060 for SIP and Port xx - yy for RTP media  (Asterisk based PBX uses often 10.000-20.000 for it, 4.000-4.999 for T.38) c) all traffic which comes from local IPs (phone1, phon2, ... phoneN) but nicer and a little more "secure" would be to setup an own VLAN for your phone network and then route it with one rule ;)

@Reiner030: Hi, normally the monitor ip could be gateway IP of your WAN interface… If both WAN interfaces have same gateway then you should "randomly" use external IPs to get a monitoring up. As written in other threads the pfSense would set special routes for this monitoring IP to use only WAN1, WAN2,...WANn to this special IP. So you need to use an "always up" IP to guarantee monitoring is working. GoogleDNS could be offer IPs... 8.8.8.8, 8.8.4.4 OpenDNS could also help: 208.67.222.222, 208.67.220.220 or you use 2 different IPs of your provider...

Hi, normally the monitor ip could be gateway IP of your WAN interface… If both WAN interfaces have same gateway then you should "randomly" use external IPs to get a monitoring up. As written in other threads the pfSense would set special routes for this monitoring IP to use only WAN1, WAN2,...WANn to this special IP. So you need to use an "always up" IP to guarantee monitoring is working. GoogleDNS could be offer IPs... 8.8.8.8, 8.8.4.4 OpenDNS could also help: 208.67.222.222, 208.67.220.220 or you use 2 different IPs of your provider...

Hi, yes, we have done it. In my mesage history are several posts / you can search common for BGP here in forum - there are many good question/answer threads. You should use OpenBGP which works nice. But BGP uses also static IPs… only routing is done in different way. And BGP normally makes sense only for /24 or bigger networks because public announced networks must be minimal /24. I guess your ISP want you to discard default gw and offer you 2 or more gateways to let you automatically failover between backbones. Perhaps he can offer you also OSPF which is a little easier to setup because its done "automatically" per broadcast (but with pfSense package only IPv4 actual possible)? Bests

You use the "Firewall Rules" for that….  (also look at the advanced options of the FW rules)

you could try to enable default gateway switching (System: Advanced: Miscellaneous: Loadbalancing). it's possible that you'd get a faster responds this way, but it might have consequences elsewhere - be ware ;)