so you basically need failover ? you could do that with tiered gateways &policy routing the wiki title is for a multi-wan setup, but the same might be of use in your situation: https://doc.pfsense.org/index.php/Multi-WAN#Failover you might have to watch out for asymetric routing issues ( send by fiber, receive by vpn = not what you would want) the other option is to run a dynamic routing protocol (like ospf or bgp) to handle the re-routing when one link goes down

I ended up moving the CA and server certs to the PFsense and setup the OpenVPN server on it. Works OK now. My main point was to spread the load. The server that it used to run has a much better CPU than the router. It looks like the PFsense can saturate our 50MBIT connection, so thats fine. Thanks a lot for the insight!

you can edit the raw file itself. Thats what we did when we setup our BGP.

Depending on what you want to do.. You would have to create a monitor for your default gateway that goes somewhere outbound and not just your gateway address which is the default.  You then have another gateway setup that uses your other gateway 2 address. You shouldn't have to jump through these hoops.. The company you paid that put in the fortinet needs to do their job!!

Huh?  If you want redundancy then you would setup a carp..  Which is kind of pointless on the same vm host. You don't need 2 pfsense to have different devices use a proxy or not use a proxy be it http or https..

Just noticed the same problem occurs with any nic when the router it is connected to is rebooted: pfsense's DHCP client does not get periodically called to obtain an IP.

The above is correct to my knowledge as well. We run an HA setup and use CARP VIPs for everything WAN. We have a directly allocated /27 to our WAN interface as well as a routed /25. The ISP routes the /25 traffic to our primary IP on the /27 and everything works like magic. We only have one upstream gateway so there was no additional work required on our side.

ok, there is something else wrong….... reduce down to once vpn. pfsense vpn      10.0.10.22 vpn host          10.0.10.21 vpn gateway    10.0.10.1 dpinger from box: dpinger -f -B 127.0.0.1 8.8.4.4    - no packet loss dpinger -f -B 10.0.10.22 8.8.4.4  - packet loss after 3rd ping. what am i missing?

fingured it out found a nice blog somebody has done https://blog.monstermuffin.org/create-an-ipsec-site-to-site-tunnel-between-two-pfsense-firewalls/ with a draytek router you can add phase2 [image: drayek.jpg] [image: drayek.jpg_thumb]  

Thanks for the advice Johnpoz.  I at the very least have a pretty clear understanding why its broken.  Hopefully I can convince some people to make a change.

dude run your own scan, go to canyouseeme.org..  What IP comes up in the box?  Is that your IP your domains are pointing too?  Again I scanned that IP and port 80 is not listening.. Here I just did it from another online scanner.. those 3 ports your firewall shows open 80,443,8080 all come back as filtered!!!  Ie nothing listening.. Notice no packets came back.. Starting Nmap 6.00 ( http://nmap.org ) at 2016-12-13 13:48 EET Initiating SYN Stealth Scan at 13:48 Scanning cradley.heathfield.sandwell.sch.uk (81.145.129.116) [3 ports] Completed SYN Stealth Scan at 13:48, 2.83s elapsed (3 total ports) [+] Nmap scan report for cradley.heathfield.sandwell.sch.uk (81.145.129.116) Host is up. PORT    STATE    SERVICE 80/tcp  filtered http 443/tcp  filtered https 8080/tcp filtered http-proxy Nmap done: 1 IP address (1 host up) scanned in 5.44 seconds           Raw packets sent: 6 (264B) | Rcvd: 0 (0B) I would validate that is your actual IP..  Maybe your IP changed!!  Is your reverse proxy running and listening on those ports?  Because get nothing back from that IP on those ports

Update! As for the questions: as far as i know, any service running on pfsense will bypass policy routing. So loadbalancing is not going to work as intended.

@bjaffe: You have a firewall rule above your load balancing rule (Default LAN to Any ipv4) that's taking precedence on all of your LAN net generated traffic and using the default gateway. PfSense will process the rule set from top down. Move the LAN net to any rule below the one you have configured with the specified GW group. That did it! Thank you! Completely overlooked the firewall priority law.  I changed the "default Lan to any rule" to the gateway LB and killed my own created rule. The two WANs now appear to be load-balancing but not as effectively or efficiently as I would like them to. Each WAN on its own could give me 25-27Mb/s bandwidth (speedtest.com), combined I don't get anything above 15-17Mb/s. I have been tweaking around with the weight ratio (though both have the same speed and are from the same ISP). Aside from using speedtest, I thought downloading a large file via IDM could be a better venue for testing the actual bandwidth speed, but IDM appears to be using only the default gateway (for instance I set the IDM to use 8 connections, and set the weight ratio to 4-4 on pfsense, but IDM is only talking through the default gateway, while the second gateway is idle with no traffic activity). On some youtube videos I have seen people easily aggregating the two bandwidth (illustrated as before and after on speedtest), but so far my attempts have been semi-fruitful (if that's even a word). I will try to research more on this matter on my own, but as always any help that could save me time, frustration, and energy, would be greatly appreciated! Also, you can't use the ping or traceroute tools inside of pfSense to test your load balance configuration because it's considered firewall generated traffic. The rule you configured for specifying your load balancing GW group won't apply when the traffic is generated using those tools on pfSense. It will only apply to "inbound" traffic to that specified interface (LAN). Also, multi-WAN load balancing entails individual connections being balanced in a round-robin fashion, so traceroute wouldn't be the best test here. Try running a speed-test and then checking the traffic graph in pfSense looking at both WANs and making sure activity is taking place on them. I did not know that. Thank you for clarifying the matter for me. UPDATE: So, I tested load balancing only with the  two DSL lines, and now it appears to be I'm getting the aggregated bandwidth of 15 Mb/s (7Mb/s from DSL A + 8Mb/s through DSL B). Another thing that is a bit puzzling with regards to the TD-LTE lines is that when I start downloading a file one of the two connections' RTT begins to hike up very rapidly (from 130ms  to 650ms  where offline state is triggered) while the other one remains pretty stable.  ??? Also at all times the two connections seem to have about 60 to 70ms RTT difference!