CARP stuck in "init" in Hyper-V
-
I have a somewhat unconventional setup, but I'm not sure whether this is the cause of the problem or not:
Two servers in separate locations running Hyper-V R2 linked by an OpenVPN bridge on the hosts, thus sharing a local network 10.1.0.0/24.
On both servers there are several VMs and one pfSense VM each working as a firewall. Each pfSense VM has a LAN interface connected to the local network. "Enable MAC spoofing" is enabled for that Hyper-V NIC.Everything except CARP is running fine. Both pfSenses successfully sync their config via XMLRPC.
I'm trying to setup a virtual IP with CARP failover on the LAN interface. The virtual IP is created on the master and synced to the backup - no problem there. BUT: the VIP is stuck in the "INIT" state and cannot be pinged.
Everytime I enable CARP (via the GUI or ifconfig vip1 down/up), I get the error "kernel: ifa_add_loopback_route: insertion failed" in the system log.I know BSD is not officially supported on Hyper-V, but this error seems to be a pfsense/BSD problem, not a Hyper-V one.
Even if CARP could not work with Hyper-V or my bridged setup, shouldn't I be able to ping the VIP locally? Even while it's in the INIT state?I did some googling for the "insertion failed" error, but BSD seems to claim that this was fixed years ago.
Any help would be appreciated, I'm running out of ideas.
-
Carp uses multicast to check other side health. Can you see (via tcpdump) if these packages are going to each other?
-
What exactly am I looking for?
Neither a packet capture on the pfsense box itself nor a network monitor capture on the host showed up anything that looked like it might be CARP-related…
But shouldn't the CARP state go to "master" anyway, even if it couldn't find a live partner due to network/Hyper-V problems?
-
But shouldn't the CARP state go to "master" anyway, even if it couldn't find a live partner due to network/Hyper-V problems?
Both will be master when everything is correct except helth checks.
Are you on latest version of pfsense(2.0.1)?
-
That's what I expected, too - if they can't find each other, they should both switch to master. That's what's making me think it's a pfSense problem…
Yes, both machines are on 2.01.
-
If it were just a communication problem, you'd have both as master. The only time I've seen a CARP IP get stuck in INIT is when either the parent interface of the CARP IP doesn't exist, or there is some other config issue with the IP subnet the CARP IP is on.
What do the logs for the VIPs show?
-
From the results of another similar thread here, now I'm wondering if it's related to the sometimes flaky low grade NIC driver that hyper-v uses. Does it support emulating e1000 NICs yet like basically every other hypervisor on the planet? Or any ability to change NIC type for FreeBSD guests? It used to work fine, even on 2.0 IIRC and there are no OS level differences between 2.0 and 2.0.1. I don't use hyper-v myself though.
-
Yes, it seems to be the "de" driver for the emulated network cards. Is there any way to swap this driver within pfSense? I can't choose the NIC model in Hyper-V, the only choices are "network card" (which needs the VMBUS drivers) and "legacy network card" (which is a DEC 21140).
-
Any solution about "ifa_add_loopback_route: insertion failed" on pfsense with Hyper-V?
-
Is there any chance to get CARP working under Hyper-V?
That became a trouble for me.. I wanted 2 pfSense's (1st in Hyper-V Server 2012 and 2nd in Windows Server 2008 R2) to 'CARP each other', but had no luck. I had no idea of this problem, so now I see the only possible solution for me - switch to ESXi, but I wanted Hyper-V.
As for impossible solutions… :) what about substituting 'de' driver with something else?
In the FreeBSD sources I found that dev/lmc and dev/dc seem to support some DECs.Has anyone tried this?
-
Microsoft finally dropped some code to provide proper FreeBSD support, which we'll integrate when we get to a base version that supports it (2.2). In the mean time, hyper-v isn't a great option.
-
From the thread, I seem to recognize this behavior from when fiddling around with Windows NLB on Hyper-V…
Have you flagged "Enable spoofing of MAC addresses" on the virtual Nics where your CARP is running?
As I remember the pfSense guidelines for ESX states that promiscous mode must be enabled on the vswitch for running CARP, this seems to be the Hyper-V equivalent...Regards,
Anders -
in vmware i had to enable promiscuous mode in the vswitch to get carp working, or else i would run into the same problems as you are describing.
sadly i don't know where the equivalent for this setting is in hyper-v.
