Forcing FW failover, FW + Router configuration
-
Hello,
We have configured succesfully xml-rpc sync, pfsync and CARP, and all works like a charm.
But we have a problem bacause our firewall acts as router too, and we have a complex set up (up to 6 NICs). We have configured with CARP a virtual IP for each NIC, and when there's a issue in a NIC, the VIP moves to the backup firewall as expected. Great!
But obviously the routing it's broken then (ie. the WAN interface is broken, and the VIP moves to the backup server… but the rest of the IPs are still in the master firewall).
So for our setup the only one interesting option is forcing that when a NIC fails, the whole FW turns into fail state and the all the stuff it's managed by the backup firewall.
After some research we haven't found how to group NICs (so when one fails, all the VIPs are moved together) or something like that to mark the whole FW as down when just one NIC fails and migrate all the VIPs to the backup firewall.
Any pointer to achieve this would be very appreciated. Thank you in advance.
Regards,
Juanjo
-
I can't find how to implement this with pfSense.
I'm doing a perl script to monitor the CARP interfaces, and if one of them fails… force the failover of the rest of the CARP interfaces, so we have a failover solution that cover both the fw breakage and the NIC problems.
Regards,
Juanjo
-
Have you tested the configuration? The preemption sysctl should be on (1) on pfSense:
_ net.inet.carp.preempt Allow virtual hosts to preempt each other. It
is also used to failover carp interfaces as a
group. When the option is enabled and one of
the carp enabled physical interfaces goes down,
advskew is changed to 240 on all carp inter-
faces. See also the first example. Disabled
by default._ -
We're using 1.2.2 and It's not working… I don't know if net.inet.carp.preempt it's on or off, but when an interface fails... the carp interfaces doens't work as a froup, so I guess it must be off.
Seems that's what I was looking for. I'll do some test to see what's going on.
-
Anyway, thank you for your answer!
I'm going to check the value of net.inet.carp.preempt ASAP.
-
How is everything connected? Optimally, you want a dedicated failover interface. Also, things will not work as expected unless the systems can communicate with each other over a given interface. E.G. If I lose link on the WAN interface of my primary box, the state may get stuck in INIT, but the secondary box should still go to MASTER on all the CARPs. You do need the matching interfaces of each box to be able to send CARP updates to each other. I usually hardware VLAN a switch for this and peel off three or more ports for each connection.
-
Yep, I think it's working in the way you're saying.
Each NIC can communicate with the other one (WAN with WAN, OPT1 with OPT1, etc). But when just one NIC fails (let's say WAN), we would expect that all the CARPs go MASTER on the backup server, but only the failing NIC is being moved.
That's confusing, because the fw does routing also, and because only the failing NIC gets moved, the fw it's useless (just WAN moved to the backup, but the rest of the NICs remain on the fw with the broken NIC).
I don't know if we're doing something wrong.
-
The situation hasn't come up for me in real deployments (the primary firewall would have to have a NIC/cabling failure or somesuch), but in testing, if I (for example) pull the WAN cable on the primary, the primary will hand control of the LAN over to the secondary (as the two LAN interfaces will still be talking).
-
I've verified net.inet.carp.preempt it's ON.
I'm lost. I don't understand why the CARP interfaces aren't working as a group when there's a NIC failure.
I don't know what wrong, but when one CARP interface fails, it moves to the other fw… but JUST THAT interface.
-
I would check the carp interfaces with ifconfig when one interface was failed. I would also verify I could ping the other node from the non failed interface (that should be going to standby).
-
All interfaces will fail over when one fails. If that's not happening, you have a network problem of some sort, that means the multicast CARP traffic isn't getting between the primary and secondary (your switches may be blocking this).
-
I just experienced something similar (or same) while setting up two new 1.2.3 based embedded routers. pfsync is working fine (over a dedicated interface)
I created 3 vips (LAN,LAN2,WAN) in that order, vhid's: 1,2,3. The interfaces would fail over separately - I unplug WAN, it fails over to router2, but LAN stays on router1, and obviously does not provide upstream connectivity.In desperation, I removed LAN2 and WAN vips, and recreated just WAN vip (LAN vhid:1, WAN vhid:2). Now LAN+WAN seem to fail over together when WAN cable is pulled. In fact, it all seems to work OK (except DHCP which I'll start a separate topic on) except that when we fail-back to router1, WAN VIP shows as "master" on both machines! I have to do some more checking as to whether this is affecting anything, but it seems pretty strange.
