How to make DHCP WANs trigger CARP failover
-
I registered in the forum today, but I have been a steady reader over some years. I have been installing, configuring, maintaining and administered pfSense at work and home the last 4 years or so. Needless to say, I'm very impressed with the software. Anyway, at home I have my Internet connection from an ISP which gives me a signal converter/modem that serves the pfSense routers with up to 4 IP addresses via DHCP. I only use two of these 4 IP addresses, since I have a pfSense CARP cluster master/backup.
My issue was that I wanted the DHCP enabled WAN connections to be able to trigger a CARP failover on the LAN VIPs. To my knowledge this is not possible to setup from within the web interface. I did not find a direct match to my issue on this forum or elsewhere either. I found a related forum post (http://forum.pfsense.org/index.php/topic,4984.msg87793.html#msg87793) which greatly helped me solve my issue. I'm very satisfied with my solution, and wanted to share it with you all. I think many of you home users will think this makes the pfSense Master/backup more usable at home. Please post a comment if this helped you.
I'm assuming that you have the same setup as me:
-
pfSense 2.0.1 CARP cluster (Master/Backup)
-
a single DHCP WAN on each of the two routers (Master/Backup)
-
One or more VIPs that you want to failover to the Backup when either the LAN or the WAN connection goes down of some reason.
OK, let me start explaining what I did:
First you should test to unplug your WAN cable from the WAN port of your Master pfSense router, just to see that your LAN VIPs won't failover, and you will lose internet connectivity in the LAN.
Then attach it again;-) Internet is back in the LAN.
From the referenced forum post in the text above I learned that /etc/devd.conf is a file where you can configure a script to run when the CARP interfaces failover. That is not what I want, but it is close.
While experimenting with /etc/devd.conf I found out that (like in the referenced forum post) I had to change the "carp" to "vip" in this section (shown as after the change):# CARP notify hooks. This will call carpup/carpdown with the # interface (carp0, carp1) as the first parameter. notify 100 { match "system" "IFNET"; match "type" "LINK_UP"; match "subsystem" "vip"; action "/etc/rc.carpmaster $subsystem"; }; notify 100 { match "system" "IFNET"; match "type" "LINK_DOWN"; match "subsystem" "vip"; action "/etc/rc.carpbackup $subsystem"; };I found out this while I solved the issue which hindered me from getting notification mails when a fail-over took place. The result from wrong value in the "match "subsystem"" is that the /etc/rc.carpbackup isn't run during failover.
Back to the main issue;-) Further down in /etc/devd.conf comes this section (which is very similar to the former one):
# Signal upper levels that an event happened on ethernet class interface # notify 0 { match "system" "IFNET"; match "type" "LINK_UP"; media-type "ethernet"; action "/usr/local/sbin/pfSctl -c 'interface linkup start $subsystem'"; }; notify 0 { match "system" "IFNET"; match "type" "LINK_DOWN"; media-type "ethernet"; action "/usr/local/sbin/pfSctl -c 'interface linkup stop $subsystem'"; };This section is more what I could use to solve my issue. If one ethernet link goes down, then "action" should happen. But how can I change it without damaging the logic already in it? After some experiments I found that I could just add one more line just below the action line to run my script. This solution made less text in the devd.conf file, but after considering for a couple of minutes I decided to choose the following addition to /etc/devd.conf just below the section above:
# Trigger VIPs when WAN goes down or up # notify 0 { match "system" "IFNET"; match "type" "LINK_UP"; media-type "vr1"; action "/usr/local/sbin/eth-up.sh"; }; notify 0 { match "system" "IFNET"; match "type" "LINK_DOWN"; media-type "vr1"; action "/usr/local/sbin/eth-down.sh"; };vr1 is the name of the WAN NIC in both my Master and Backup router. You can check the names of all relevant interfaces by running the following command:
ifconfig -aIn the added code section from /etc/devd.conf above you can see that I have called a couple of bash scripts with the action command:
eth-down.sh
eth-up.shThe content of those two scripts is as follows:
/usr/local/sbin/eth-down.sh
#!/bin/sh /sbin/ifconfig vip1 down /sbin/ifconfig vip2 down /sbin/ifconfig vip3 down /sbin/ifconfig vip4 down /sbin/ifconfig vip5 down /sbin/ifconfig vip6 down /sbin/ifconfig vip7 down /sbin/ifconfig vip8 down /sbin/ifconfig vip9 down /sbin/ifconfig vip10 down /sbin/ifconfig vip11 downeth-up.sh
#!/bin/sh /sbin/ifconfig vip1 up /sbin/ifconfig vip2 up /sbin/ifconfig vip3 up /sbin/ifconfig vip4 up /sbin/ifconfig vip5 up /sbin/ifconfig vip6 up /sbin/ifconfig vip7 up /sbin/ifconfig vip8 up /sbin/ifconfig vip9 up /sbin/ifconfig vip10 up /sbin/ifconfig vip11 upThe two scripts should be chmod'ed to 555, to be executable:
chmod 555 /usr/local/sbin/eth-up.sh chmod 555 /usr/local/sbin/eth-down.shAfter restarting devd
killall -9 devd && /sbin/devdyou will see that unplugging the WAN interface of your Master router will trigger a failover on the LAN VIPs. When you plug it back in, the VIPs will fall back to the Master again. It was just what I wanted:-)
-
-
Thanks strandte, this was very helpful in my situation. Basically, I've got two pfSense 2.0.1 boxes, each one with independent WAN circuits, but joined LAN with failover via CARP.
With your write-up, I'm able to get the WAN link failures to properly disable the respective LAN port and thus get traffic properly routed over to the slave box automatically. My only problem is that this only this failover only work is the WAN port link itself fails (ie the link between said WAN port and it's supporting modem/router), not if the whole circuit fails. For example, if something in between the modem and the Internet fails, but the link between the modem and CARP master's WAN does not fail, then traffic continues to be routed to a circuit that's effectively dead.
My question/suggestion is this: I know pfSense is capable of knowing if an entire WAN circuit fails by pinging the gateway of said interface or other user specified address in the Gateway settings. Is it possible to use this mechanic for triggering to down the respective LAN interface and in turn triggering a CARP failover?
In other words, can CARP be made to trigger a failover when a gateway is no longer reachable as opposed to simply when the WAN link goes down?
Thanks in advance.
-EJS -
Hi EJS01,
Glad I was able to help :) What you describe is of cause what everybody actually wants. Recently I have come to the conclusion that load balancing the two pfsense gateways is a better solution than my solution above. So far I have not tried load balancing, but I'm sure it will do what I think it will do from reading about it. The reason this is a better solution is of cause that the load on each of the gateways will be about half instead of the backup just sitting there and doing nothing, while the master takes the whole load.
If one gateway goes down, the other will take the entire load.Regards
strandte
-
Hi again!
I thought that today would be the day for load balancing my two pfsense routers, but it turned out that the load balancing does not work as I thought and hoped it should. It seems to me that the gateway load balancing is only to spread the traffic for eg the master router between two or more WANs connected to the same router (master). See the "Multi-WAN 2.0" HOWTO article on pfsense.org: http://doc.pfsense.org/index.php/Multi-WAN_2.0
There seems to be another form of load balancing also related to eg load balancing incoming requests for internal web servers. See this article: http://doc.pfsense.org/index.php/Setup_Incoming_Load_Balancing
Just now I can't see how to use any of these tools to do the load balancing of incoming and outgoing traffic from LAN to WAN between the master and backup router. I guess we would have to setup another kind of cluster for this to work. The current pfsense CARP cluster is a active/passive cluster type, while to do what I thought might be possible we will need an active/active type cluster.
It is possible to use the backup router to access internet while it is in the backup state and there is another active master, but you would have to setup your network via static IP on the client computer. With this static network settings you would have trouble in some rear cases. If you eg have a IPSec VPN that is active on the master router, and you tried to access the resources on the other side of the tunnel while you had your computer setup with these static network settings, it would not work. You would also not be able to enjoy the benefits of automatic fail over between the routers, without manually changing your network settings.
This means that to get load balancing between the two pfsense routers it should be setup as an active/active cluster and the dhcp server has to be integrated in the scrips for doing this possible. I dont have any experience with load balancers, but to my opinion it would not be a good solution to put a third box to do the load balancing (dedicated load balancer) in front of the two pfsense routers as this would reduce the redundancy and complicate the system.
I guess this might be a next step for pfsense? Or is it already a solution to what we talk about here????Regards
strandte -
You can't do active/active CARP nodes in that way. Only one router node can/should be active at a time for handling any traffic. You can't spread the load between multiple firewall units in an effective way at this time.
Some people have hacked things up manually to make some VIPs master on the primary box and some VIPs master on the secondary box but that's really not something I would suggest.
As you noted, multiple WANs should be connected to all carp nodes to be used properly. Anything else is just ugly and asking for trouble…
