2 WAN Failover but with two pfSense gateways?!?!
Today I setup HSRP using Cisco gear in a lab and accomplished exactly what I would like to accomplish at work; but I need to do this with two pfSense boxen since we don't have any Cisco gear at work. Here is the scenario with the Cisco stuff: Two routers each with one WAN link. Router A has IP 192.168.1.1 with a 10mbit fiber link and Router B has IP 192.168.1.2 with a 5mbit Cable link. Both routers connect to one LAN and all hosts on the LAN are configured to use the Virtual IP as their gateway (192.168.1.3). Using HSRP I configured each router to monitor the other and then created a Virutal IP of 192.168.1.3. I then designated one router as Active and the other as Standby. What will cause a failover even in this case is not only a downed router but a downed WAN link. Each router was configured to monitor the state of the WAN interface. I had a host conduct a continuous ping to a loopback interface located outside the network and I took turns physically disconnecting each WAN link (but always keeping one link up to test the failover). It worked very well and I only dropped a few packets.
So how can I accomplish this with two pfsense machines? Here is how my work network is setup: We have a colo with a fiber WAN link (static IP) that connects to a pfSense box. The LAN interface of the pfSense box connects to a small switch in our colo rack that is on our 10.10.0.0/16 LAN. Our actual facility is 45 miles away and connected via a p-to-p fiber link to our core switch that is also part of our 10.10.0.0/16 LAN. Inside our facility we have a Cable Modem WAN link that I would like to use as the backup in the event that either the WAN link at the colocation goes down or our p-to-p fiber link goes down (effectively the same thing). Since I can't get the WAN links within close physical proximity I am stuck using two gateway machines (maybe I could get tricky with some VLANing but that might just add another point of failure). I understand that CARP is for hardware failover and that LB is for link failover but how can I do this with my situation?
I am not too worried about the DNS side of things since I can intervene and manually change the DNS entires in the event of a WAN failure.
btw, what are you experts out there using for network diagrams I see around here? This scenario would be easier to explain with some graphics.
have you considered putting both WAN interfaces on each pfsense box and use carp to failover?
alternatively, you could have pfsense box 2 be tier2 of a failover on pfsense box 1, and vice versa. You could load balance and failover also.