CARP / VLAN Interfaces / Interface specified for the virtual IP address XXX does not exist. Skipping this VIP

steve.scotter

Hi all,

Any help gratefully received!

We have two PowerEdge R420's running pfsense with 11 NICS each. Both firewalls have identical hardware (save for the MAC addresses of the NICs) and are connected to two seperate switches which are linked.

Each server has eight NICs in use, seven of which has CARP setup (without VLAN configuration in the pfsense side, solely using the switches which have the VLANS tied to the port). The one NIC without CARP is a cross linked directly between the two servers for HASYNC. The two FW's are kept in sync using HASYNC, with all configuration being performed on FW1 (primary FW) and configuration automatically being sync across and applied to FW2 (Secondary FW).

This setup has been working flawlessly for approx 18 months, however I needed to spin up another network and couldn't carve out time to go to the DC to connect a new physical NIC to the switch like we have done previously so I thought I should be able to create a VLAN interface and configure as usual, but alias I get a "Interface specified for the virtual IP address 192.168.215.245 does not exist. Skipping this VIP." message and the Virtual IP never gets to a backup state on the secondary firewall.

steve.scotter

In the screenshots below you'll see everything is setup identically. The white backgrounded screenshots and FW1, the black backgrounded screenshots are FW2.

Interface Assignment
FW1
Interface Assignment FW1
FW2
Interface Assignment FW2

VLAN Interfaces
FW1
VLAN Interfaces FW1
FW2
VLAN Interfaces FW2

Virtual IPs
FW1
Virtual IPs FW1
FW2
Virtual IPs FW2
^^ Notice the last row doesn't have a Interface ^^

CARP Status
FW1
CARP Status FW1
FW2

^^ Notice the last row doesn't have a Interface prefixing the VHID ^^

steve.scotter

If I go to edit the Virtual IP on the secondary firewall I can select VLAN_AD interface from the dropdown list and save it. But as soon as I make ANY changes on FW1 (doesn't even have to be a CARP or interface change, simply adding a firewall rule) causes the config to resync and I get another "Interface specified for the virtual IP address 192.168.215.245 does not exist" notification error.

Virtual IP Edit

I've checked for white spaces around the information I've entered but didn't find any. As previously stated any help gratefully received! Feels like a bug in the synchronization routine, does anyone else agree?

We're running v2.4.4-RELEASE-p1

Cheers

Steve

PS. Had to post this across multiple replies due to getting "Post content was flagged as spam by Akismet.com" when trying to post as one message :(

stephenw10

Have some upvotes to stop Akismet tripping. That's the only 'tunable' available to us unfortunately.

There is some interface mismatch between your nodes most likely but it's not showing in the GUI. I imagine at one time one of those nodes had an interface added and removed and the next available interface is probably different.

Check the config file directly in /conf/config.xml. Looks in the <interfaces> section at the top. They must be exactly aligned including the raw interface names, <opt1> etc.

Steve

steve.scotter

Thank you for the upvotes, much appreciated!

You're absolutely right, now I've compared /conf/config.xml on both firewalls I can see the discrepancy.

Primary firewall

            <opt8>
                    <descr><![CDATA[VLANAD]]></descr>
                    <if>igb6.98</if>
                    <enable></enable>
                    <spoofmac></spoofmac>
                    <ipaddr>192.168.215.240</ipaddr>
                    <subnet>24</subnet>
            </opt8>

Secondary firewall

            <opt7>
                    <descr><![CDATA[VLANAD]]></descr>
                    <if>igb6.98</if>
                    <enable></enable>
                    <spoofmac></spoofmac>
                    <ipaddr>192.168.215.242</ipaddr>
                    <subnet>24</subnet>
            </opt7>

Is there a better alternative than deleting and recreating the VLAN interface on the secondary firewall to get the optX numbers back in sync? I don't suppose I can just edit the /conf/config.xml file and reboot!?

Thanks for your assistance!

Steve

Derelict

You would have to be sure to change every place referencing opt7 to opt8. But that should work.

stephenw10

Some interface is missing from the primary then? Or is there an extra interface that's disabled?

If it is you could add that interface and have it not enabled on the secondary so that when you then add igb.98 it appears as opt8 and lines up with the Primary.

Steve

Derelict

Yeah - I thought about it a while and couldn't think of a gui way to do it while preserving all of the interface config, rules, dhcp servers, etc.

Moving opt7 to opt8 in the secondary config seemed the smoothest way.

steve.scotter

Just to let you all know that I performed the following on the primary firewall :-

Backed up my config via the webgui
Backed up /conf/config.xml to /conf/config.xml and /root/config.xml
Installed nano (pkg install nano)
Search /conf/config.xml for opt8 and replaced it with opt7
Rebooted

I'm now able to make changes on the primary one, have them reflect on the secondary one and not receive a "Interface specified for the virtual IP address 192.168.215.245 does not exist." message. The CARP status pages now have VLAN_AD@12 on both the primary and secondary firewall.

Thank you so much! Me and my team have been putting up with this for about four months!

Hope this helps someone else in the future!

Cheers

Steve

stephenw10

Nice!
The ee editor is built in but if familiar with nano no reason not to install it. Also there's vi.....

Steve