Disabling DHCP on WAN interface when carp in backup mode
-
In my setup, in the shown diagram, I have two pfSense firewalls.
Once is a VM on my lab host the other and APU2 both have to share a single DHCP supplied IP address but there is currently no way that I know of to get the CARP address on the WAN side to be supplied by DHCP.
Having both firewalls up causes all sorts of problem because of this, as both firewalls sharing the same MAC address on their WAN interface see the same DHCP offer and allocate it to the interface.
The switch between the modem and the firewalls sees the IP address move between the two ports and things are not very pretty.
I can see two ways to solve this, firstly adding the ability to use a DHCP address as the CARP address somehow.
The second is to disable the WAN interface on the backup if the LAN interface is not the CARP master, or put another way to pin the WAN interface status to the CARP status of the LAN interface.I have actually got the second method to work with some small hacks and was thinking of turning this into a more robust solution by writing a module to allow you to select a CARP interface to watch and pin the status of the other interface to it.
https://github.com/deasmi/pf_interface_pinBefore I do this I wanted to check if people agreed with my approach, and whether there are any better ideas to solve this, admittedly, edge case problem.
One other thing to note is that I don't really need this anymore, as since I got the APU there is much less need for me to take firewalls offline, so in some ways this is more of an interest item than a critical problem.
Thanks :)
-
[T]here is currently no way that I know of to get the CARP address on the WAN side to be supplied by DHCP.
There isn't one. It isn't a supported HA configuration (with good reason).
Having both firewalls up causes all sorts of problem because of this, as both firewalls sharing the same MAC address on their WAN interface see the same DHCP offer and allocate it to the interface.
You cannot spoof a MAC address in combination with HA. All you have done is create a MAC conflict. That can not be made to work in a supportable fashion.
I can see two ways to solve this, firstly adding the ability to use a DHCP address as the CARP address somehow.
The second is to disable the WAN interface on the backup if the LAN interface is not the CARP master, or put another way to pin the WAN interface status to the CARP status of the LAN interface.Don't do either of those things.
Either allow them both to pull a unique DHCP address on that interface and ignore CARP there, or static assign them unique addresses and do not use DHCP.
If you must have it on a single address on that interface then you'll need another NAT device on that segment between the DHCP server and pfSense, so that device will get DHCP and then you can use a static assignment on the back end. The extra layer of NAT is ugly, but it's still less ugly than trying to hack up DHCP to work in an unintended fashion.
-
[T]here is currently no way that I know of to get the CARP address on the WAN side to be supplied by DHCP.
There isn't one. It isn't a supported HA configuration (with good reason).
Are you saying it's not supported with FreeBSD/CARP for a good reason, or that there is never a good reason to have dhcp failover for HA purposes (on any platform) ?
If the former, yes I can see that CARP just doesn't support this.
If the latter then I'd have to disagree that it's never a good idea for HA.
This kind of setup was supported by, from memory, at least netscreen firewalls.
I think it is a valid requirement to have DHCP failover on WAN interface, for a start I have that requirement :), even if it's not a common one.
I can see two ways to solve this, firstly adding the ability to use a DHCP address as the CARP address somehow.
The second is to disable the WAN interface on the backup if the LAN interface is not the CARP master, or put another way to pin the WAN interface status to the CARP status of the LAN interface.Don't do either of those things.
Either allow them both to pull a unique DHCP address on that interface and ignore CARP there, or static assign them unique addresses and do not use DHCP.
If you must have it on a single address on that interface then you'll need another NAT device on that segment between the DHCP server and pfSense, so that device will get DHCP and then you can use a static assignment on the back end. The extra layer of NAT is ugly, but it's still less ugly than trying to hack up DHCP to work in an unintended fashion.
Sadly multiple IPs are not an option, my ISP provides a single IP via DHCP and I don't want another device or double NAT.
Whilst potentially unusual I do want this as have a physical and virtual firewall and like to be able to failover, I can't get my ISP to provide multiple IPs.
There is a page where someone has already looked at this https://sites.google.com/site/bsdstuff/dhcarp which I'm going to digest, I'm sure there must be a sensible way to provide this functionality, and if there isn't at least I'm learning a lot about the internals of pfSense. :)
-
Ah! Somebody else that solved this COMMON problem/scenario in their own way.
Question: How are you calling your scripts? (Just putting a file in /usr/local/bin won't make it get executed?).
I also had this need (HA for Comcast Residential ISP) and figured out a way too.
My solution involves changes only to the backup machine where I have 2 scripts, one to be executed when carp goes to master and another for when it goes to backup. Well, the master host needs an ip alias (1.1.1.1) on the WAN so the backup host can reach the Internet through it.
Carp backup script sets MAC address of WAN to default/true MAC address and IP to 1.1.1.2.
Carp master script changes MAC address of WAN to the same as the master machine WAN (so cable modem talks to backup machine without rebooting cable modem) and gets DHCP address on WAN.The funny thing here is how to get around pfblocker and the WAN built-in blocking of private networks
(I want the option enabled) so you have to add a couple of exceptions and check the specific order in which rules/exceptions get applied, etc.To put it all together I put a small script in /usr/local/etc/rc.d/ that simply calls a script after boot.
That script simply modifies (sed) /etc/rc.carpmaster and /etc/rc.carpbackup (adds a line to call the corresponding script above) and then calls the script that puts the machine in carp backup mode.So this way my scripts get executed when carp goes to master or backup.
It works well and this way the backup machine can have internet connectivity for pfblocker/suricata updates while in carp backup mode.Cheers.
J. -
I was with a feedback as you like.
