Hyper-V ICS 1.0 (w/Synthethic Network Driver) for pfSense 2.1 & 2.1.1

Guest

Microsoft let me know last night that they've developed a simple (45 line) patch to FreeBSD's hyper-v net driver (https://github.com/freebsd/freebsd/blob/master/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c) that fixes the issue with CARP.

I'm looking for no more than 5 well-qualified volunteers to test a custom-build with this patch included, on Hyper-V.

Email me if you want to test this.

If it works well, we'll get it into pfSense 2.2

Jim

Guest

@lburton:

@gonzopancho:

In any case, I spoke with the Hyper-V and Azure people from Microsoft at the FreeBSD developer summit earlier this week.
They are quite interested in working with us to develop a fully-certified Hyper-V image, including fixes (via Microsoft)
to the obvious multicast issues with CARP/pfSync in the underlying drivers. They're also interested in more extensive testing, including performance-related work, and tuning.

I can definitely confirm it's a Hyper-V NIC driver issue for carp but I don't believe it to multicast related in carp's case – it looks to be related to the NIC state information as a quick super-hacky-terrible patch I did to the carp kernel code has resulted in functional carp on a test setup for me in Hyper-V. (I haven't tried to get pfsync going in the same setup -- it doesn't seem to be syncing state properly according to pftop.) I added it to the existing ip_carp.c.diff in the patch list:
root@freebsd:~ # cat /home/pfsense/tools/patches/releng/10.1/ip_carp.c.diff
diff --git a/sys/netinet/ip_carp.c b/sys/netinet/ip_carp.c
index a170e34..0a3607e 100644
--- a/sys/netinet/ip_carp.c
+++ b/sys/netinet/ip_carp.c
@@ -532,8 +532,8 @@ carp6_input(struct mbuf **mp, int *offp, int proto)
        /* check if received on a valid carp interface */
        if (m->m_pkthdr.rcvif->if_carp == NULL) {
                CARPSTATS_INC(carps_badif);
-               CARP_DEBUG("%s: packet received on non-carp interface: %s\n",
-                   __func__, m->m_pkthdr.rcvif->if_xname);
+               //CARP_DEBUG("%s: packet received on non-carp interface: %s\n",
+               //    __func__, m->m_pkthdr.rcvif->if_xname);
                m_freem(m);
                return (IPPROTO_DONE);
        }
@@ -1195,8 +1195,7 @@

        CARP_LOCK_ASSERT(sc);

-       if ((sc->sc_carpdev->if_flags & IFF_UP) == 0 ||
-           sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
+       if (
            (sc->sc_naddrs == 0 && sc->sc_naddrs6 == 0))
                return;

@@ -2001,27 +2000,11 @@

        CARP_LOCK_ASSERT(sc);

-       if (sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
-           !(sc->sc_carpdev->if_flags & IFF_UP)) {
-               callout_stop(&sc->sc_ad_tmo);
-#ifdef INET
-               callout_stop(&sc->sc_md_tmo);
-#endif
-#ifdef INET6
-               callout_stop(&sc->sc_md6_tmo);
-#endif
-               carp_set_state(sc, INIT);
-               carp_setrun(sc, 0);
-               if (!sc->sc_suppress)
-                       carp_demote_adj(V_carp_ifdown_adj, "interface down");
-               sc->sc_suppress = 1;
-       } else {
                carp_set_state(sc, INIT);
                carp_setrun(sc, 0);
                if (sc->sc_suppress)
                        carp_demote_adj(-V_carp_ifdown_adj, "interface up");
                sc->sc_suppress = 0;
-       }
 }

 static void
I'm not experienced enough with FreeBSD kernel debugging / drivers to really take this all that much further without it being fairly efforty – but it looks like:
-       if (sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
-           !(sc->sc_carpdev->if_flags & IFF_UP)) {
is not behaving correctly under hyper-v's network drivers. As for the cause for pfsync's woes I may try to take a look later.

The patch is actually somewhat simpler than this, and is completely contained in hv_netvsc_drv_freebsd.c.

lburton

@gonzopancho:

The patch is actually somewhat simpler than this, and is completely contained in hv_netvsc_drv_freebsd.c.

Agreed – I'm using the patch from the BSD integration team @ MS as of 3 days ago and it appears to resolve the NIC state issues as expected.

Guest

@lburton:

@gonzopancho:

The patch is actually somewhat simpler than this, and is completely contained in hv_netvsc_drv_freebsd.c.

Agreed – I'm using the patch from the BSD integration team @ MS as of 3 days ago and it appears to resolve the NIC state issues as expected.

patch is now in the next snapshot. (So much for all the key4ce bullshit.)

cmb

I can confirm that CARP is fully functional in Hyper-V with the most recent 2.2 snapshot available! Thanks to the folks at Microsoft for fixing it and getting us the patch. Those who'd like to use CARP in Hyper-V, check out the most recent 2.2 from snapshots.pfsense.org and let us know your experiences.

cmb

@cmb:

@key4ce:

That you think CARP is fully working: sorry we already confirmed this: it wasn't, even this week we found some new bugs with it.

It's pretty clear from your code changes you have no idea what you're doing. Outside of the Hyper-V NIC driver bug, I'm going to call you out on this one - open just ONE actual bug with CARP at redmine.pfsense.org. Just one would suffice. You're spreading FUD and won't be able to do so.

I'll re-issue my call out now that CARP in 2.2 is working in Hyper-V as well, just in case anyone thinks they have any degree of credibility. We haven't gotten a single bug report from them to prove their point (because they're lying). Ours works with IPv6 too! Which key4ce's hacked up mess does not.

nomaded

@gonzopancho:

Microsoft let me know last night that they've developed a simple (45 line) patch to FreeBSD's hyper-v net driver (https://github.com/freebsd/freebsd/blob/master/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c) that fixes the issue with CARP.

I'm looking for no more than 5 well-qualified volunteers to test a custom-build with this patch included, on Hyper-V.

Email me if you want to test this.

If it works well, we'll get it into pfSense 2.2

Jim

I'm trying to get CARP on FreeBSD 10.1 to work on Hyper-V. Is it possible to share the 45 line patch?

nomaded

@nomaded:

I'm trying to get CARP on FreeBSD 10.1 to work on Hyper-V. Is it possible to share the 45 line patch?

Just to close the loop on this, the patch can be found at https://bz-attachments.freebsd.org/attachment.cgi?id=151515. MAC spoofing also needs to be enabled on the virtual NIC.

Guest

@nomaded:

@nomaded:

I'm trying to get CARP on FreeBSD 10.1 to work on Hyper-V. Is it possible to share the 45 line patch?

Just to close the loop on this, the patch can be found at https://bz-attachments.freebsd.org/attachment.cgi?id=151515. MAC spoofing also needs to be enabled on the virtual NIC.

You can look in the pfsense-tools repo for the same patch.

duntuk

Is this patch necessary for the pfsense 2.2 release ? If so, how do you apply it?

cmb

@duntuk:

Is this patch necessary for the pfsense 2.2 release ?

No, it's included.