Hyper-V ICS 1.0 (w/Synthethic Network Driver) for pfSense 2.1 & 2.1.1

[key4ce]

Well in short: We normally publish under GPL
but as this has a BSD license base it's not possible.
Next in line is Creative Commons share-alike license.

quick overview:
http://creativecommons.org/licenses/by-sa/4.0/

it pretty much mandates that all and any codes based up on our codes has to stay opensource (share-alike).
as in the end, what we create for free should stay for free.

As far as finger pointing goes: We both just want/require different things when it comes to source-codes. so no real party to blame to other then both have different requirements and wishes.

There's no quick howto guide on how to make PFSense fully work with Virtualized environments.
PFSense 2.2 beta will have some basic drivers in it as it's based up on FreeBSD 10 which has it build in
however alot of components like CARP (clustering / loadbalancing) won't work on Hyper-v. (it's why we spend well over 8 months on development on getting it to work 100%).

Regards,
Marco

[Guest]

You can't relicense the pfSense base.

To be perfectly clear, you can choose whatever license you like for your code.

but you can't relicense the pfSense base, which you appear to have done here.

[Guest]

And in our experience, pfSense works fine on Hyper-V.

The Key4ce folks are allowed (of course) to make other claims.

Let the market decide.

[key4ce]

Jim, you can hold your accusations for your self.

We can't re-license the base which is exactly why we can't apply GPL but CAN apply Share-alike.

Unlike GPL, Share-alike can be applied for just a part of the code (OUR CODE) in an entire base you provide (the fork).
which is why we opted for share-alike rather then the more common GPL.
But please note whenever you use OUR code anywhere else the ENTIRE code has to stay opensource (share-alike).

As you can see from the license, codes, and even download sections no pfsense licenses, copyrights, etc have been violated.
We are still working on scrubbing the PFSense name from the builder etc (not the copyrights but the name it self) so we can publish the builders without a CLA in place without it affecting PFSense.

so the builder/tools can only be used with VirtualPF, and won't affect your CLA agreements before people get access to PFSense tools for PFSense it self.

As i told you countless of times: we ARE compliant to license agreements, we ARE NOT planning to voilate anything even though you keep trying to tell everyone that we got bad intentions and don't comply.

As far as CARP not working.. even FREEBSD mailing list tells you it won't:
http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038053.html

It won't on Hyper-v. (stays in init)
It will when you extensively re-develop the protocol like we did and make it compliant with any virtual environment.

You might also want to know you need to adjust other codes to use 10+ Gbit speeds in Freebsd.
Even though you got the drivers in place it will by default not allow you to use that much speed (it's capped to less then 1 Gbit).

Regards,
Marco

[Guest]

@key4ce:

Jim, you can hold your accusations for your self.

Since I am the dude that deals with this for pfSense, that's not going to happen.

@key4ce:

We can't re-license the base which is exactly why we can't apply GPL but CAN apply Share-alike.

Nope.  And I'm sure you know better.  You can choose whatever license you like for your changes, but the code is licensed as a whole (via the license agreement, that you "signed".)  That license agreement prohibits you from relicensing the code.

@key4ce:

Unlike GPL, Share-alike can be applied for just a part of the code (OUR CODE) in an entire base you provide (the fork).

Yes, you can apply whatever license you like to YOUR CODE.

You can't relicense what you got FROM US.

@key4ce:

which is why we opted for share-alike rather then the more common GPL.
But please note whenever you use OUR code anywhere else the ENTIRE code has to stay opensource (share-alike).

Yes, YOUR LICENSE applies to YOUR CODE.  Not pfSense.

@key4ce:

As you can see from the license, codes, and even download sections no pfsense licenses, copyrights, etc have been violated.
We are still working on scrubbing the PFSense name from the builder etc (not the copyrights but the name it self) so we can publish the builders without a CLA in place without it affecting PFSense.

Unfortunately, your statement is not true.

For example, where is the required attribution on this page?: https://virtualpf.com

And… I knew you were after freeing the builder code.  Unfortunately, you'll now have to follow the train, or live with your fork.

C'est la vie, such is open source.  You chose to fork.

@key4ce:

so the builder/tools can only be used with VirtualPF, and won't affect your CLA agreements before people get access to PFSense tools for PFSense it self.

As i told you countless of times: we ARE compliant to license agreements, we ARE NOT planning to voilate anything even though you keep trying to tell everyone that we got bad intentions and don't comply.

As far as CARP not working.. even FREEBSD mailing list tells you it won't:
http://lists.freebsd.org/pipermail/freebsd-net/2014-March/038053.html

Many people are having good luck.  https://forum.pfsense.org/index.php?topic=75549.0

@key4ce:

You might also want to know you need to adjust other codes to use 10+ Gbit speeds in Freebsd.
Even though you got the drivers in place it will by default not allow you to use that much speed (it's capped to less then 1 Gbit).

I've allowed many times, in many places that work is underway on 10Gbps.

As for your release..  (let's stop talking about licensing, because it's obvious you don't care, and instead talk technology):

The only functional differences I see in your release (after quickly running through a test install) is things your team broke - they screwed up interfaces in general (assignments screwy, DHCP interfaces don't pull IPs the way they should), the ISO doesn't reboot post-install (spews "vm_fault pager read error" over and over and over until you reset the machine).

Your new theme on the web interface is broken in a variety of ways.  You try to load some sort of VMware tools, but do so in a broken manner that causes VMware Workstation to complain.

You claim, "We have modified CARP extensively to work on any virtual platform" - as if it didn't already.  (It does!)
The GUI-side is the same, if you modified the OS you might have broken something.  I've not had time to even look for
the source code (where is it?) much less check out what you've done.

From what we can tell, your release is effectively 2.2, except on an older 10-STABLE from before we moved to 10.1, with a lot more stuff that doesn't work.

pfSense 2.2 will be based on FreeBSD 10.1-RELEASE, and there is a lot of work that came after your 14 March posting above.

https://virtual-ops.de/?p=600
https://wiki.freebsd.org/HyperV
http://azure.microsoft.com/blog/2014/05/22/running-freebsd-in-azure/

But hey, it's Open Source!

[key4ce]

Jim, again we only licensed our code, your code stays in tact and so does your license.
The code as a whole is then known as dual licensed and it is shown very clearly (altho the legal page is still in the making to clear it out further).
Thats why where in some sections where we changed you will find your License.txt and our License-VirtualPF.txt.

Bit odd you can claim we don't care about licensing when we spend so much time argueing about it tho.
also not something i care to do again. we checked it over with multiple legal shavy people who all said what we did is actually over doing what is required.

Which makes me wonder 1 or 2 things:

1. Your either not fully aware of legal laws and limitations of your own legal aspects
2. Your very aware but pretend not to know and scare people out of it and/or to see what it is they exactly know.

That i don't have to put credits on every website page: that is correct. you can ask your lawyers and they will tell you that claim is not possible under international laws. (and is also not written in your Legal section.. please read your own legal section careful.)

We did however keep it in the license page within the software and everyone can see it in full view before download.

We did not release any git code yet as it's still fully compatible with PFSense (including the builder).
I doubt you want agreements and CLA's to get crossed.

However we expect that part to be 100% available for public within about 3 weeks (after we fully integrated freebsd 10.1 stable)

The agreement Angelo signed does not limit the use or fork of any code so we still comply to that too.

As far a tech:
Ours already works and tested with over 40 Gbit's speeds.

The gui is indeed not finished, if you check the roadmap –> it was only made in 5 days and got quite alot left to redo (it's why we only made a Beta release their not intended on being perfect but atleast function wise you can test)

The Ejecting CD on what Virtual Environment is it tested? (version)
as we confirmed it working on vmware esxi, and vmware desktop (both latest versions).

That our release is effectively 2.2 is total BS and you know it.
CARP was just one, supporting 10 Gbit+, supporting AND automatically installing any virtual platform driver automatically based up on platform detection would be another. gui improvements is one, and translations to Chinese, Dutch, German are on their way.

Much more features have been added and will be carefully detailed before November 30th

It's just the first release, it won't truly define our selfs just yet. that will come with stable 2.

That you think CARP is fully working: sorry we already confirmed this: it wasn't, even this week we found some new bugs with it.
But ofcourse feel free to believe it is, I personally think our testing grounds are a bit more advanced then yours (as below 1 Gbit speeds won't be accepted by any of our customers, not even our own environment can go below 20 Gbit else we pinch our customers off lol).

Regards,
Marco

[hege]

@gonzopancho:

You claim, "We have modified CARP extensively to work on any virtual platform" - as if it didn't already.  (It does!)

Sorry, but thats not true, you can't use CARP with Hyper-V with current 2.2 (stucks in init as key4ce said)

[rainabba]

@gonzopancho:

And in our experience, pfSense works fine on Hyper-V.

The Key4ce folks are allowed (of course) to make other claims.

Let the market decide.

Everywhere I've found a solution, it is to use legacy nics with a disclaimer that it should not be used in production. Is there another solution or is that advice wrong?

Afterthought: …or is this a reference to 2.2? I'm out of the loop and didn't realize it was available yet since I've only been looking at RELEASE. I realize it is considered BETA, but would anyone with the project be willing to call it stable enough for more basic use in production (NAT, Firewall, IPSEC)?

Thanks in advance

[rainabba]

@key4ce:

Jim, again we only licensed our code, your code stays in tact and so does your license.
The code as a whole is then known as dual licensed and it is shown very clearly (altho the legal page is still in the making to clear it out further).
Thats why where in some sections where we changed you will find your License.txt and our License-VirtualPF.txt.

Bit odd you can claim we don't care about licensing when we spend so much time argueing about it tho.
also not something i care to do again. we checked it over with multiple legal shavy people who all said what we did is actually over doing what is required.

Which makes me wonder 1 or 2 things:

1. Your either not fully aware of legal laws and limitations of your own legal aspects
2. Your very aware but pretend not to know and scare people out of it and/or to see what it is they exactly know.

That i don't have to put credits on every website page: that is correct. you can ask your lawyers and they will tell you that claim is not possible under international laws. (and is also not written in your Legal section.. please read your own legal section careful.)

We did however keep it in the license page within the software and everyone can see it in full view before download.

We did not release any git code yet as it's still fully compatible with PFSense (including the builder).
I doubt you want agreements and CLA's to get crossed.

However we expect that part to be 100% available for public within about 3 weeks (after we fully integrated freebsd 10.1 stable)

The agreement Angelo signed does not limit the use or fork of any code so we still comply to that too.

As far a tech:
Ours already works and tested with over 40 Gbit's speeds.

The gui is indeed not finished, if you check the roadmap –> it was only made in 5 days and got quite alot left to redo (it's why we only made a Beta release their not intended on being perfect but atleast function wise you can test)

The Ejecting CD on what Virtual Environment is it tested? (version)
as we confirmed it working on vmware esxi, and vmware desktop (both latest versions).

That our release is effectively 2.2 is total BS and you know it.
CARP was just one, supporting 10 Gbit+, supporting AND automatically installing any virtual platform driver automatically based up on platform detection would be another. gui improvements is one, and translations to Chinese, Dutch, German are on their way.

Much more features have been added and will be carefully detailed before November 30th

It's just the first release, it won't truly define our selfs just yet. that will come with stable 2.

That you think CARP is fully working: sorry we already confirmed this: it wasn't, even this week we found some new bugs with it.
But ofcourse feel free to believe it is, I personally think our testing grounds are a bit more advanced then yours (as below 1 Gbit speeds won't be accepted by any of our customers, not even our own environment can go below 20 Gbit else we pinch our customers off lol).

Regards,
Marco

For someone who wouldn't have a project without everything provided in the pfSense base, you come across very rude, unappreciative and ungrateful. You sound like you're using open-source ideals as a defense, rather than being a good player in the scene. If the claims made by other on the PFSENSE forums aren't true, what do you gain by arguing; especially when they're providing evidence?

Generally, only guilty parties or those in harm of penalty feel the need to continuously justify themselves because the truth speaks for itself.

Furthermore, you're enforcing what I said earlier; "I" am the type of customer you're going to be pursuing if you are actually trying to build a business, and you're doing everything in your power to ensure that I don't go anywhere near testing, let alone production use of your build. If your build is so different and superior, why not do it from scratch and stay away from these forums so that you can have more control, less licensing issues and more respect for the finished product. Of course, I'm just "a customer" so what does my opinion matter? :)

I will step out of the drama on that note and continue my search for how to run pfSense, stable, on Hyper-V.

[Guest]

@hege:

@gonzopancho:

You claim, "We have modified CARP extensively to work on any virtual platform" - as if it didn't already.  (It does!)

Sorry, but thats not true, you can't use CARP with Hyper-V with current 2.2 (stucks in init as key4ce said)

We're not done with CARP in 2.2.

[Guest]

I don't recommend even trying 2.1 on Hyper-V.  That path leads to madness.

2.2 is in pretty good shape.  There are 10 (count them, 10) outstanding bugs in the way of cutting a release candidate.

one of them concerns CARP.

try it out!  (It's a VM environment, what do you have to lose?)

Jim

[Guest]

@key4ce:

Jim, again we only licensed our code, your code stays in tact and so does your license.
The code as a whole is then known as dual licensed and it is shown very clearly (altho the legal page is still in the making to clear it out further).

Your continued assertion is that you can relicense our code.  You. Can.  Not.

@key4ce:

Thats why where in some sections where we changed you will find your License.txt and our License-VirtualPF.txt.

I don't care about your license.  I care about mine.

@key4ce:

Bit odd you can claim we don't care about licensing when we spend so much time argueing about it tho.
also not something i care to do again. we checked it over with multiple legal shavy people who all said what we did is actually over doing what is required.

I don't claim that you "don't care about licensing".  I claim that you have breeched the license.  Perhaps you don't care that you've breeched.

@key4ce:

Which makes me wonder 1 or 2 things:

1. Your either not fully aware of legal laws and limitations of your own legal aspects
2. Your very aware but pretend not to know and scare people out of it and/or to see what it is they exactly know.

Begging the question via assertion of incorrect facts isn't going to work.  I'm very aware of the law, and pay attorneys to provide cover.

@key4ce:

That i don't have to put credits on every website page: that is correct. you can ask your lawyers and they will tell you that claim is not possible under international laws. (and is also not written in your Legal section.. please read your own legal section careful.)

You 'signed' a license that comes with certain obligations.  "International laws" (there is no such thing) certainly allow you to bind yourself to certain acts in exchange for consideration.

@key4ce:

We did not release any git code yet as it's still fully compatible with PFSense (including the builder).

Yes, you've forked a proprietary copy, and it's fully allowed.  have fun with your proprietary clone, and stay off the pfSense forums with any further discussion about same.

@key4ce:

Ours already works and tested with over 40 Gbit's speeds.

You didn't make pf work at 40Gbps, except possibly with very large packets.  Perhaps you mean that you've included support
for some 40Gbps adapter, in which case… BFD!  pfSense 2.2 supports the Chelsio T5.

But again.. stay off the forums until your fork is open sourced.  I'm not hosting people who aren't willing to play on that field, and make no mistake ... you are (or were) a guest here.

[key4ce]

Jim, keep it simple, check with your legal parties and see if i complied to the agreement AND license.
I checked with mine which simply said: we don't even need to give so much credit as we did.

If you want i can publish the entire report of their findings.

please note we aren't making use of any of it besides that we don't have to place your copyrights on every single website page of VirtualPF.

That i am a guest here: i am well aware of that.
I was just answering another guests question without accusation or anything else.
same as we politely still contributed some simple fixes to your repo which was bothering people on your forum.

Where you fly off again and take the opportunity to bash a little bit more.

That you think we did or not do something like 40 Gbit: We work with Datacenters you can trust them to do simple tests like that.
That you again think we aren't opensource: we can release your builders as is, but that means anyone can download them from us and use them on PFSense.

I was being polite by waiting untill we made those only work on VirtualPF so it doesn't affect PFSense.
If you don't care about that, then our source will be out in the open within end today, please remember that includes the builders and without a CLA, we only apply CLA for contributors not to view the source it self (as viewing source and using it has nothing to do with CLA).

Regards,
Marco

[Guest]

@key4ce:

Jim, keep it simple, check with your legal parties and see if i complied to the agreement AND license.
I checked with mine which simply said: we don't even need to give so much credit as we did.

You signed a license agreement, or you did not.  It's simple.

@key4ce:

I was being polite by waiting untill we made those only work on VirtualPF so it doesn't affect PFSense.
If you don't care about that, then our source will be out in the open within end today, please remember that includes the builders and without a CLA, we only apply CLA for contributors not to view the source it self (as viewing source and using it has nothing to do with CLA).

I'm not sure what you're saying here.

In any case, I spoke with the Hyper-V and Azure people from Microsoft at the FreeBSD developer summit earlier this week.
They are quite interested in working with us to develop a fully-certified Hyper-V image, including fixes (via Microsoft)
to the obvious multicast issues with CARP/pfSync in the underlying drivers.  They're also interested in more extensive testing, including performance-related work, and tuning.

This would fix the only real outstanding issues with Hyper-V, and would put both a pfSense and Microsoft "seal of approval" on the result.

As before, this project is scheduled after pfSense version 2.2 is released.

[Rooter]

That's awesome. Full hyper-v support…yeah.
At the moment i have several installations of the snapshots series running smooth like a charm.

Only worry is the lack of the kvp tools ( i guess).
Live backup can be interrupting the service.

[cmb]

Just in case anyone actually thinks these guys know what they're doing, and to debunk their FUD flinging. I installed their iso and spent a few minutes messing around.

@key4ce:

Ours already works and tested with over 40 Gbit's speeds.

There aren't frames big enough to get > 40 Gb through pf at this point. We've done a great deal of analysis internally here, and continue to do so. There is no way you've made any significant performance improvements beyond what we've already done (having multiple FreeBSD committers on the payroll, one on contract who's a FreeBSD core team member and co-author of The Design and Implementation of FreeBSD book).

@key4ce:

That our release is effectively 2.2 is total BS and you know it.

It's not BS at all. It looks like you changed a tiny fraction of 1 percent of the LOC excluding the web interface design changes, and what you did change there is broken. Out of curiosity I looked at what you're actually doing to "make CARP work", which was pretty entertaining (and scary for anyone who actually thought using your mess was a good idea).

It's complete with source code comments on their own changes including:

needs rework!

That's the only really accurate code change I saw.

Then there's this gem:

Very ugly bitch failsafe...

We're clearly dealing with professionals here, folks!

They replaced CARP with ucarp, which isn't a good idea for this type of use case, and did a poor implementation of it.

@key4ce:

That you think CARP is fully working: sorry we already confirmed this: it wasn't, even this week we found some new bugs with it.

It's pretty clear from your code changes you have no idea what you're doing. Outside of the Hyper-V NIC driver bug, I'm going to call you out on this one - open just ONE actual bug with CARP at redmine.pfsense.org. Just one would suffice. You're spreading FUD and won't be able to do so.

@key4ce:

I personally think our testing grounds are a bit more advanced then yours

That's hilarious given how much basic stuff you broke, like you trashed something in the interfaces code that creates an absurd slew of bugs. If you did any worthwhile degree of testing you wouldn't have thrown out such a mess to the world.

We've been doing this project years before your company even existed. Then for years while your company was fixing people's home computers. It's pretty clear who's competent at what they're doing here, the folks who are behind one of the most widely used network firewall distributions in the world and have over a decade of results to show for it (read: us), not the one that has some weird twisted definition of open source that matches nothing that exists but yet won't release the source to their "open source project."

@gonzopancho:

In any case, I spoke with the Hyper-V and Azure people from Microsoft at the FreeBSD developer summit earlier this week.
They are quite interested in working with us to develop a fully-certified Hyper-V image, including fixes (via Microsoft)
to the obvious multicast issues with CARP/pfSync in the underlying drivers.  They're also interested in more extensive testing, including performance-related work, and tuning.

This would fix the only real outstanding issues with Hyper-V, and would put both a pfSense and Microsoft "seal of approval" on the result.

And…game over for key4ce. ;D  I'm sure there are plenty of home computers they can go fix.

[cmb]

@Rooter:

Only worry is the lack of the kvp tools ( i guess).
Live backup can be interrupting the service.

That's something we'll get into with the folks at Microsoft, but generally speaking for this type of use case, you don't do full VM backups at all. Have HA systems in place, and config backups ready to restore, and you're good. Generally just a waste of disk/tape/cloud/whatever-you-backup-to space to get full backups of a system like pfSense where it's really easy and fast to rebuild from scratch and restore (probably faster than restoring a full disk VM backup actually).

[lburton]

@gonzopancho:

In any case, I spoke with the Hyper-V and Azure people from Microsoft at the FreeBSD developer summit earlier this week.
They are quite interested in working with us to develop a fully-certified Hyper-V image, including fixes (via Microsoft)
to the obvious multicast issues with CARP/pfSync in the underlying drivers.  They're also interested in more extensive testing, including performance-related work, and tuning.

I can definitely confirm it's a Hyper-V NIC driver issue for carp but I don't believe it to multicast related in carp's case – it looks to be related to the NIC state information as a quick super-hacky-terrible patch I did to the carp kernel code has resulted in functional carp on a test setup for me in Hyper-V. (I haven't tried to get pfsync going in the same setup -- it doesn't seem to be syncing state properly according to pftop.) I added it to the existing ip_carp.c.diff in the patch list:

root@freebsd:~ # cat /home/pfsense/tools/patches/releng/10.1/ip_carp.c.diff
diff --git a/sys/netinet/ip_carp.c b/sys/netinet/ip_carp.c
index a170e34..0a3607e 100644
--- a/sys/netinet/ip_carp.c
+++ b/sys/netinet/ip_carp.c
@@ -532,8 +532,8 @@ carp6_input(struct mbuf **mp, int *offp, int proto)
        /* check if received on a valid carp interface */
        if (m->m_pkthdr.rcvif->if_carp == NULL) {
                CARPSTATS_INC(carps_badif);
-               CARP_DEBUG("%s: packet received on non-carp interface: %s\n",
-                   __func__, m->m_pkthdr.rcvif->if_xname);
+               //CARP_DEBUG("%s: packet received on non-carp interface: %s\n",
+               //    __func__, m->m_pkthdr.rcvif->if_xname);
                m_freem(m);
                return (IPPROTO_DONE);
        }
@@ -1195,8 +1195,7 @@

        CARP_LOCK_ASSERT(sc);

-       if ((sc->sc_carpdev->if_flags & IFF_UP) == 0 ||
-           sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
+       if (
            (sc->sc_naddrs == 0 && sc->sc_naddrs6 == 0))
                return;

@@ -2001,27 +2000,11 @@

        CARP_LOCK_ASSERT(sc);

-       if (sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
-           !(sc->sc_carpdev->if_flags & IFF_UP)) {
-               callout_stop(&sc->sc_ad_tmo);
-#ifdef INET
-               callout_stop(&sc->sc_md_tmo);
-#endif
-#ifdef INET6
-               callout_stop(&sc->sc_md6_tmo);
-#endif
-               carp_set_state(sc, INIT);
-               carp_setrun(sc, 0);
-               if (!sc->sc_suppress)
-                       carp_demote_adj(V_carp_ifdown_adj, "interface down");
-               sc->sc_suppress = 1;
-       } else {
                carp_set_state(sc, INIT);
                carp_setrun(sc, 0);
                if (sc->sc_suppress)
                        carp_demote_adj(-V_carp_ifdown_adj, "interface up");
                sc->sc_suppress = 0;
-       }
 }

 static void

I'm not experienced enough with FreeBSD kernel debugging / drivers to really take this all that much further without it being fairly efforty – but it looks like:

-       if (sc->sc_carpdev->if_link_state != LINK_STATE_UP ||
-           !(sc->sc_carpdev->if_flags & IFF_UP)) {

is not behaving correctly under hyper-v's network drivers.  As for the cause for pfsync's woes I may try to take a look later.

[Guest]

That's interesting (thank you!).

I'll look at 'why' a bit later.

[lburton]

Hrm pfsync actually appears to work just fine (I fell into the same trap as https://forum.pfsense.org/index.php?topic=24550.0).  Bad config on my part. I did enable both MAC spoofing http://blogs.technet.com/b/jhoward/archive/2009/05/21/new-in-hyper-v-windows-server-2008-r2-part-2-mac-spoofing.aspx and I disabled "Microsoft Windows Filtering Platform" in the Extensions portion of the vswitch.. although I'm not certain that was necessary – I did it a while before I realized my config issue.. I'll try reverting it/cleaning up the configs later.
I'll also try putting some real connections / clients behind it and doing some testing (I can drop it in place of my existing home router setup with some work/cabling).

I'd love to hear what you find re:the driver/state stuff.. Thanks!