High Availability fail-over combined with IPv6

[Marijn]

Hi All,

I am facing the following issue, after creating our pfSense HA cluster we are experiencing ipv6 issues.

After a failover from the primary to the secondary unit, I see many packet drops in ipv6 on all directly connected networks (and on on-link subnet – an average of 75%+ packet loss). Meanwhile, Ipv6 to the outside world is working completely fine, and ipv4 is also working fine.

Failing back to the primary will not solve the issue, but also does not make it worse; it just does not have any effect on our issue.

However shutting down the secondary node will resolve the issue, and everything goes back to normal.

I am aware that for DHCPv6 there is not a fancy failover mothed, so I am using one of the recommended blueprint:

• Configured RA to Managed + DHCPv6 independently using separate local pools
• Gateway is handled by router advertisements, on both, bind to CARP VIP, and use Normal router priority. RADVD will start/stop with CARP status

In addition, for hosts/servers configured using a static IPv6 configuration we see the same behavior.

So to illustrate this strange behavior using a sample:

Static host 2001:fff:ffff:10::2 is not able to ping another static host using IP 2001:fff:ffff:10::3 after a failover without a ping loss of 75%+

This occurs after a failover and will not stop until I shut down 1 of the two pfSense boxes.

Therefore, for me, it looks like a duplicate IP or MAC address somewhere, but I am sure all configured interface IP address and CARP address are correct, and the CARP VIPs do not have duplicated VHIDs.

Any suggestions?

Thanks!

[Marijn]

Ok, I found out the following:

The described problem only occurs to VM's hosted our 2 ESX 6.5 hypervisors.

All bare metal servers will work completely fine (on-link and directly connected), only VM's are affected.

On Vmware the vSwitches are configured including the following settings:

Promiscuous mode enabled;
MAC Address changes enabled;
Forged transmits enabled;

However, I don't think this is strictly needed since the firewalls are physical devices.

Is someone aware of a required setting that is required in VMware / pfSense to get this correctly working?

Thanks anyway :)