[SOLVED] Firewall rule on CARP interface keeps being deleted after sync
-
Yes, you have to enable the "Synchronize Rules" only on the master.
Make sure also that both server has the same exact NIC's (e.g. 3 NICS on master and 3 NICS on slave for WAN, LAN and SYNC). Don't use 2 NIC's on the SLAVE while 3 NIC's on the master or vice verse because once the master goes down, the LINK between the two server will be drop specially on the LAN side.
On the SLAVE you need to set the IP on what you have done on the MASTER at Virtual IP.
eg. on master: 192.168.100.100/24 (CARP) with vhid 1 = for WAN Interface
192.168.1.100/24 (CARP) with vhid 2 = for LAN Interfacehere's another link for you:
http://www.pfsense.org/mirror.php?section=tutorials/carp/carp-cluster-new.htm
-
bitsync: try to also use http instead of https and see if that makes a difference.
-
Hi.
Thank you both for your reply.
I got the same ammount of interfaces on both servers (3 NICs whereas 1 is used as dedicated link for CARP only). Since the machines are not of the same build (one beeing 6 years old and the other only 3 months) the interfaces are a bit different but they are "named" the same in pfSense.
The failover is working fine though (tested and confirmed ;)) if all rules are synched.And I also tried to switch from HTTPS to HTTP which had no effect. –> Edit: it did have the effect of not having https for the web-gui - which of course was expected - but the problem still exists
Maybe I can reformulate my problem a little bit:
Everything in my configuration with CARP works as I would expect. If the master firewall goes down the backup system jumps in automatically and resumes all tasks until the master comes back.
To make the whole think work though I always have to login on my backup firewall and create the firewall rule for the carp interface "any2any" to get the next sync working.
Once I change something on the master and "apply changes" everything is synched and the backup is up2date. However the one rule that is missing is the one for the CARP interface.
And this causes the next sync to fail. -
Is there a rule in the master node for the CARP interface for the any-to-any?
-
Hi.
Looking for the usual suspects are we? ;D
Yes the rule is there and that's whats bugging me the most:
every rule gets synched only that one does not. if it were the setting on the backup would still be there after the sync. -
Yeah. It is always a good place to start. When you created the CARP interface and renamed them, did they both start out as opt1 before renaming them?
-
Hello.
No. Both interfaces had different names orginially.
On the master that is "em0" and on the slave it is "fxp0" -
That is not a problem. I have different NIc types as well. But if you assigned them differently before renaming them, there might be a problem. Like if one was opt2 and the other was opt1, and you renamed them both to CARP, then potentially, I am just guessing, there might be a problem.
I know clustering works, i have setup it up 5 or more times and they all are still running with no problem.
-
Those interfaces have not been used before in another manor so i could rule out that possibility.
-
Are you using the CARP network to sync settings also? Can you post a sanitized copy of /tmp/rules.debug from the master node?
-
I am not sure what you mean. The settings I use are shown in the screenshots.
-
Forgot this was the second page :). Anyway, looking back at the screen shots it does look like on the master node that CARP was originally opt2 as one of your VPN interface took opt1. I can tell by the ordering of the tabs. This should not make a difference as they are named. Try this, on the master, add a description to the allow all CARP rule (ie CARP Allow All). Sync the settings, and see if that description show up on another interfaces rules.
Are those other interfaces (RV,OVPNS1, OVPNC1,MGMT) VLANs?
-
Yes. OVPNS1, OVPNC1, MGMT and RV are VLANs
I tried adding a description to the CARP-interface rule on the master and started the sync. After that my rule on the backup FW is gone (as always) but the rule from the master does not show up on any other interface.
Edit: I was wrong: the rule does show up (i had it not to replicate via "No XMLRPC Sync" option).
It appears on the "MGMT" interface -
Here are 2 screenshots for my interfaces:
on master i got this:
an on backup i got this:
-
I even went further now and found out that the rules are synced on the wrong interfaces in several occasions:
Master -> Backup
OVPNS1 -> CARP
CARP -> MGMT
OVPNC1 -> RV
-> OVPNS1
-> OVPNC1With all that i am surprised that WAN and LAN aren't synced on the wrong interface as well ;)
Edit: Looking at the screenshots i believe that the sync does not apply to the interface names but to their creation order.
-
That's what I was going to suggest checking.
The number and order of interfaces in carp cluster members must be the same. What you are seeing is the result of the interfaces not being assigned in the correct order on the slave.
-
Does that mean i have to remove and recreate my interfaces on the backup server to get the correct order?
OR can i simply update some config file through the console to get the same result?The rules and settings for those interfaces should be synced automatically, shouldn't they?
-
Yes, unless you want to hand edit the config to swap things around.
-
sometimes hand editing is the easiest. especially if you have to replace a lot of IPs. But in this case, prolly easier to just redo the slave.
-
Do you know which file i need to edit?
