After Update from 2.2 to 2.2.1 Carp makes strange things - IPv6

masterd01

Hello? Any Ideas?

cmb

Is there anything related to the affected IPs in the system log?

The only reason they wouldn't show in the CARP status page is if they're not configured on the interface. What does the output of 'ifconfig' look like?

masterd01

Hi,

for a time it is okay (Screenshot), after that the Interface on the Backup there is no State for the IPv6 if (Second Screenshot).
In the Logfile i see some Records. But i don't know if there are depends on that.

Apr 2 10:00:45 kernel: carp: demoted by 0 to 0 (pfsync bulk start)
Apr 2 10:00:45 kernel: ifa_del_loopback_route: deletion failed: 3
Apr 2 10:00:45 kernel: ifa_del_loopback_route: deletion failed: 3
Apr 2 10:00:45 kernel: carp: VHID 6@em3: INIT -> BACKUP
Apr 2 08:00:45 check_reload_status: Carp backup event
Apr 2 10:00:45 kernel: carp: VHID 5@bce1: INIT -> BACKUP

Thanks,

MasterD

carp-ok.JPG_thumb

carp-fehler.JPG_thumb

cmb

Where they're not showing status like that, it's because they're missing or showing some odd status. The output of "ifconfig" is necessary to tell what's happening.

masterd01

Hi,

here is the Output of the ifconfig (as Snap).

Works:

bce1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
        options=c00bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,vlan_hwtso,linkstate>ether 00:1a:64:7a:52:b0
        inet6 fe80::21a:64ff:fe7a:52b0%bce1 prefixlen 64 scopeid 0x6
        inet ***IPv4***.12 netmask 0xfffffff8 broadcast ***IPv4***.15
        inet6 2a00:***IPv6***f2 prefixlen 64
        inet ***IPv4***.10 netmask 0xfffffff8 broadcast ***IPv4***.15 vhid 2
        inet6 2a00:***IPv6***f0 prefixlen 64 vhid 5
        nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        carp: BACKUP vhid 2 advbase 1 advskew 101
        carp: BACKUP vhid 5 advbase 1 advskew 101
pflog0: flags=100 <promisc>metric 0 mtu 33144
pfsync0: flags=41 <up,running>metric 0 mtu 1500
        pfsync: syncdev: bce0 syncpeer: ***PEERIP***.11 maxupd: 128 defer: on
        syncok: 1</up,running></promisc></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,vlan_hwtso,linkstate></up,broadcast,running,promisc,simplex,multicast>

After a Time:

bce1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
        options=c00bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,vlan_hwtso,linkstate>ether 00:1a:64:7a:52:b0
        inet6 fe80::21a:64ff:fe7a:52b0%bce1 prefixlen 64 scopeid 0x6
        inet ***IPv4***.12 netmask 0xfffffff8 broadcast ***IPv4***.15
        inet6 2a00:***IPv6***f2 prefixlen 64
        inet ***IPv4***.10 netmask 0xfffffff8 broadcast ***IPv4***.15 vhid 2
        nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        carp: BACKUP vhid 2 advbase 1 advskew 101
pflog0: flags=100 <promisc>metric 0 mtu 33144
pfsync0: flags=41 <up,running>metric 0 mtu 1500
        pfsync: syncdev: bce0 syncpeer: ***PEERIP***.11 maxupd: 128 defer: on
        syncok: 1</up,running></promisc></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,vlan_hwtso,linkstate></up,broadcast,running,promisc,simplex,multicast>

The virtual IPv6-IF is missing there.

In the Logfile i only see this:

Apr 5 00:05:29 	php-fpm[78361]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (opt2).
Apr 5 00:05:28 	kernel: carp: demoted by 0 to 0 (pfsync bulk done)
Apr 5 00:05:28 	php-fpm[8462]: /xmlrpc.php: Resyncing OpenVPN instances.
Apr 5 00:05:28 	php-fpm[8462]: /xmlrpc.php: ROUTING: setting IPv6 default route to 2A00:***IPv6GW*:1
Apr 5 00:05:28 	php-fpm[8462]: /xmlrpc.php: ROUTING: setting default route to ***IPv4Gw***.9
Apr 5 00:05:28 	check_reload_status: Reloading filter
Apr 5 00:05:28 	php-fpm[78361]: /xmlrpc.php: Configuring CARP settings finalize...
Apr 5 00:05:28 	php-fpm[78361]: /xmlrpc.php: pfsync done in 0 seconds.
Apr 5 00:05:28 	php-fpm[78361]: /xmlrpc.php: waiting for pfsync...
Apr 5 00:05:27 	kernel: carp: demoted by 0 to 0 (pfsync bulk start)
Apr 5 00:05:27 	kernel: ifa_del_loopback_route: deletion failed: 3
Apr 5 00:05:27 	kernel: ifa_del_loopback_route: deletion failed: 3
Apr 5 00:05:27 	kernel: ifa_del_loopback_route: deletion failed: 3

Greeting

masterd01

Update: I found out, that if i do a change on the first firewall the XMLRPC Sync make the two IPv6-IFs invisible. After some tests i found out, that the Problem comes if i activated the VIP-Config-Transfer from the Master to Slave-Config

cmb

I figured it was the config sync doing it, the question is why. It's not a general issue. I'm guessing if you go to Firewall>Virtual IPs on the secondary, edit (no need to change anything), save, and apply changes, it shows up there fine?

There's something about your config that makes it not work, but it isn't clear what. Could you get me a config backup from both systems, or get me remote access to them?

masterd01

Yes that's true. If i chance something on the slave the IFs going up. So i had stop syncing the Virtual-IPs from the master and everything is good.
I send you a backup of the systems via EMail - is more secure :-)

masterd01

Hi,

after Update to the 2.2.2 the Problem is still existing. Any ideas or solution?

Thanks a lot

cmb

Your config is unusual, and technically incorrect probably with the /128 IPs on the interfaces. I believe if you put an IP within the /64 of the CARP VIP on each interface instead of that /128, it will likely work fine.