Hyper-V ICS 1.0 (w/Synthethic Network Driver) for pfSense 2.1 & 2.1.1

doktornotor

I believe that is what ESF is trying to accomplish, protecting the trademarks and consumers from potential abuse in the best way possible for everyone involved.

How did this "best way possible for everyone involved" improve any of the 3 example situations mentioned in my previous post. (Just a couple of most obvious/likely ones.) Kinda curious about the answer, since after that, another bunch of semi-random posts appeared, pretty much totally taken out of the context and not following the conversation logic at all.

What you seem to be missing is that these people in general don't give a damn about your trademarks. You are taking away access to required tools from them and are making their life a PITA and are wasting their time, using trademarks as a magic formula due to a largely overblown incident… Frankly, I couldn't care less how's this thing called and what logo it uses, does NOT matter in the least for me. Features, usability, stability, as few bugs as possible, fixing issues fast - that is what users are interested in. Not your trademarks.

Guest

At this point, nothing we say will correct the misinterpretations of "doktornotor".

So, we're done here.

If the thread turns abusive, it will be closed.

peterclark4

@gonzopancho:

At this point, nothing we say will correct the misinterpretations of "doktornotor".

So, we're done here.

If the thread turns abusive, it will be closed.

Agreed, please can we try and keep this on topic!

We now have the opportunity to have a release for the Hyper-V and Xen hypervisors. (yes I realise that the Xen hypervisor is quite off-topic).

I'm looking forward to progress!

Peter

Guest

Me too, Peter.

Let's move pfSense forward. Getting it available on new platforms is one way of making forward progress.

I would really like to see pfSense on Hyper-V and Xen. I'm just looking for some assistance in making it happen.

key4ce

Well, the Xen part is already done and fully tested so it already happened.
thing left to do would be finding a way to get it published ;-)

Our R&D is now working on Hyper-v one and they assume it won't take a long time to finish that one aswell (with CARP working and up to 10Gbit speeds).

Tho due to early stage they are reluctant to give any real time frame.

For any more info and/or finding a way to get it published i assume direct mail will work faster then a off topic forum chat.

Guest

@key4ce:

For any more info and/or finding a way to get it published i assume direct mail will work faster then a off topic forum chat.

jim-at-netgate-dot-com
or
jim-at-pfsense-dot-org

zootie

I've been hammering on it, trying to get it to build a working hyper-v ISO using the tools. They're small changes, it just takes a long time to test them (and it'd help to have help from someone familiar with the build process, specifically on kernel building)

Results have been mixed. Part of the problem has been that the compile options used to build the kernel that is used in the ISO don't seem to work with the hyper-v modules (they compile, just don't work). We'll likely have to use the same AMD64 SMP kernel that gets installed on HD on the ISO (don't know how to specify this in the tools).

When forcing the SMP kernel and modules onto the ISO, I can get the utilities and network driver to load (so the live CD can run a firewall with the synthetic network driver as you'd expect), but I've been unable to get it to work when loading the hyper-v storage modules because it can't find the optical drive when it boots (this is what the hv_ata_pci_disengage.ko module is supposed to prevent, but it doesn't seem to be working), and since it doesn't load the storage driver, you'd have to create the geom labels as described in the first post (if you want improved disk performance).

key4ce

Hmm,

Well it been a while back since i tried your iso (when it was still publicly available).
I do remember there where issues with CARP + FreeBSD at the time (which would be one of the things we do need working).

Seems our R&D is still on initial steps to first get a FreeBSD 8.3 100% working with hyper-V.

Once thats completed they (hopefully) know what to do with PFSense to make it work :-) (or so i was told)

zootie

I think I figured it out. I had to go through some false starts; but, with minor changes to pfsense tools, I finally was able to build an ISO that runs the live CD and installs using the synthetic device drivers. I'll post more details once I run some more tests.

With any luck, this will help the community short term, maybe allow to have an official Hyper-V edition within the pfSense codebase, and give key4ce and team a head start on making a CARP enabled version.

peterclark4

@zootie:

I think I figured it out. I had to go through some false starts; but, with minor changes to pfsense tools, I finally was able to build an ISO that runs the live CD and installs using the synthetic device drivers. I'll post more details once I run some more tests.

With any luck, this will help the community short term, maybe allow to have an official Hyper-V edition within the pfSense codebase, and give key4ce and team a head start on making a CARP enabled version.

That is great news!

If you need any help testing just let me know.

Peter

falumas

Great news for somebody like me who loves Hyper-V and Pfsense.

I can also help with the testing if needed.

tester_02

Now that pfsense is moving onto a newer freebsd, would that mean that the latest hyper-v code is in the new freebsd?

rainnyday

Hi Zootie,

Thanks for this great post. Can you provide the download link to me, the link on your original post did not work for me.

Rainny

zootie

Good news that pfSense 2.1.1 is now released. I'll try and test the modules with it and report back (but they should work using Option B work w/o needing to recompile).

Back to building an ISO with integrated drivers. The key element to be able to get the live CD to work was to set hw.ata.disk_enable so it wouldn't disconnect the optical drive from the live CD during boot (for more info, see "Both FastIDE and CD-Rom work in FreeBSD 10 on Hyper-V (…)" and "svn commit: r252645 (…) stordisengage storvsc vmbus").

As of my last testing (a few weeks ago), I had 2 variations of the ISO (both requiring minor changes to the pfsense builder scripts):

ISO-A - Changing build_iso.sh so it would set hw.ata.disk_enable and loading the ko drivers in the ISO's loader.conf.local
ISO-B - Changing rc.cdrom and detecting if it is running under Hyper-V and then loading the modules on demand and then apply the loader.conf.local changes to the installation

ISO-A is the simplest and most direct, but when I tested it on real HW, it would cause issues (the boot process would get disconnected from the CD), and it wouldn't necessarily setup pfSense on the direct access disk driver (it uses ad0 rather than da0 storage device). ISO-A would likely require having a "Hyper-V only edition ISO" (it doesn't require forking the source, just adding an optional option to make the Hyper-V changes).

ISO-B is more dynamic, and it only loads the drivers when it is running under Hyper-V (which I imagine is that FreeBSD 10's installer does) and then fixes up the new installation by loading the modules, so it wouldn't require a Hyper-V only ISO edition. However, it would allow both the ATA and direct access storage drivers to be visible from pfSense installer (you'd see both ad0 and da0 in the installer - see attached screenshot), so it could be a little confusing (and it might require some editing of loader.conf and/or creating labels afterwards if you want to switch drivers afterwards).

I'm thinking that as part of the fixup in ISO-B, the script could check what disk device driver is being used in the new install's fstab, and then only set hw.ata.disk_enable when the ad0 drivers is being used (or not even load the storage driver), but it seems non-optimal (the installer would still see the 2 disk devices during setup, maybe confusing users). I'll have to try it, and lacking a better alternative, it would work ok (IMO, better than having a Hyper-V only edition)

What would we prefer? A separate ISO only Hyper-V? Maybe there is a way to hide a disk device in the installer? Unsure if there is a way to selectively unload the ATA driver (so it is still working for the virtual CD, but no longer visible for the disk - I imagine this logic is in the FreeBSD 10 as well). Maybe there is another setting similar to hw.ata.disk_enable that only applies to optical drives?

Jim, maybe someone more familiar with the pfSense installer can help?

BTW, I emailed a month ago to try and get access to the tools repo, but I haven't got a reply yet.

pfSense-Installer-da0VSad0.png_thumb

taliesins

Thanks for you hard work Zootie. This will setup kvp so that the integration services will report the ip address. It was essential for me as I was spinning up pfsense in hyper-v with vagrant. Its brilliant for devs to host their own load balancers. Hopefully vagrant will take my patch to handle multiple network card and the ability to specify the switch that they are in, so we can all use it.

I would really love it if someone could create a package for hyper-v so we can simply install it on a base image.

So the approach I took was to try 2.1, 2.1.1 and 2.2.

For 2.1 the network card drivers work but the kvp daemon did not
For 2.1.1 the network card drivers didnt work but the kvp daemon did (hopefully zooties iso will fix the network card problem)
For 2.2 the network card drivers are built in. So I just setup the kvp daemon

https://github.com/FreeBSDonHyper-V/Hyperv-Ports/wiki/_pages

Add required packages

cd /tmp
pkg install curl
exit

Add hv-kvp


cd /tmp
curl -L https://github.com/FreeBSDonHyper-V/Hyperv-Ports/raw/hyperv-ic-master/BIS-1.0/FreeBSD-10.0/bin/hv-kvp-x64.txz -o hv-kvp-x64.txz
pkg add hv-kvp-x64.txz
exit

Start hv_kvpd on boot


cp /boot/kernel/hv_kvp.ko /usr/local/hyperv/
cp /etc/rc.d/hv_kvpd /usr/local/etc/rc.d/hv_kvpd.sh

Need to copy ko module back after an upgrade
/usr/local/etc/rc.d/hv_kvpd.sh


!test -f /boot/kernel/hv_kvp.ko || cp /usr/local/hyperv/hv_kvp.ko  /boot/kernel

Remove stuff from loader.conf as it should live in loader.conf.local
remove from /boot/loader.conf


# Loader labels for Hyper-V drivers -do not modify
hv_kvp_load="YES"

add to /boot/loader.conf.local


# Loader labels for Hyper-V drivers -do not modify
hv_kvp_load="YES"

Move stuff from rc.conf to rc.conf.local
/etc/rc.conf
Get nuked on reboot so we just need to reboot.

add to /etc/rc.conf.local


# Labels for KVP daemon -do not modify
hv_kvp_daemon_enable="YES"

Fix dhcp script
/usr/local/hyperv/scripts/hv_get_dhcp_info
Change:


if_file="/etc/rc.conf"

To:


if_file="/etc/rc.conf.local"

Fix timecounter:


 sysctl kern.timecounter.hardware=TSC

Fix QoS:
/etc/inc/interfaces.inc

search for altq
add "hn" to array of interfaces

duntuk

Thanks @zootie! looking forward to your release.

eri--

I would suggest to use 2.2 for this effort.
Its aim is just to move to FreeBSD 10 with small effort.

That means try to release it ASAP.
Probably your work there is simpler since most of the thing is there or tell us if anything missing, for now.

bryan.paradis

@zootie:

Good news that pfSense 2.1.1 is now released. I'll try and test the modules with it and report back (but they should work using Option B work w/o needing to recompile).

Back to building an ISO with integrated drivers. The key element to be able to get the live CD to work was to set hw.ata.disk_enable so it wouldn't disconnect the optical drive from the live CD during boot (for more info, see "Both FastIDE and CD-Rom work in FreeBSD 10 on Hyper-V (…)" and "svn commit: r252645 (…) stordisengage storvsc vmbus").

As of my last testing (a few weeks ago), I had 2 variations of the ISO (both requiring minor changes to the pfsense builder scripts):

ISO-A - Changing build_iso.sh so it would set hw.ata.disk_enable and loading the ko drivers in the ISO's loader.conf.local

ISO-B - Changing rc.cdrom and detecting if it is running under Hyper-V and then loading the modules on demand and then apply the loader.conf.local changes to the installation

ISO-A is the simplest and most direct, but when I tested it on real HW, it would cause issues (the boot process would get disconnected from the CD), and it wouldn't necessarily setup pfSense on the direct access disk driver (it uses ad0 rather than da0 storage device). ISO-A would likely require having a "Hyper-V only edition ISO" (it doesn't require forking the source, just adding an optional option to make the Hyper-V changes).

ISO-B is more dynamic, and it only loads the drivers when it is running under Hyper-V (which I imagine is that FreeBSD 10's installer does) and then fixes up the new installation by loading the modules, so it wouldn't require a Hyper-V only ISO edition. However, it would allow both the ATA and direct access storage drivers to be visible from pfSense installer (you'd see both ad0 and da0 in the installer - see attached screenshot), so it could be a little confusing (and it might require some editing of loader.conf and/or creating labels afterwards if you want to switch drivers afterwards).

I'm thinking that as part of the fixup in ISO-B, the script could check what disk device driver is being used in the new install's fstab, and then only set hw.ata.disk_enable when the ad0 drivers is being used (or not even load the storage driver), but it seems non-optimal (the installer would still see the 2 disk devices during setup, maybe confusing users). I'll have to try it, and lacking a better alternative, it would work ok (IMO, better than having a Hyper-V only edition)

What would we prefer? A separate ISO only Hyper-V? Maybe there is a way to hide a disk device in the installer? Unsure if there is a way to selectively unload the ATA driver (so it is still working for the virtual CD, but no longer visible for the disk - I imagine this logic is in the FreeBSD 10 as well). Maybe there is another setting similar to hw.ata.disk_enable that only applies to optical drives?

Jim, maybe someone more familiar with the pfSense installer can help?

BTW, I emailed a month ago to try and get access to the tools repo, but I haven't got a reply yet.

Have you been granted access to the repo yet? I would hate to see you become discouraged due to this.

stiggle

What are the options for pfSense on Hyper-V now?

I've got access to an Intel Xeon based failover cluster and a standalone AMD based box to test on.

sirrus

Hi - how about 2.1.2 - OpenSSL HearBleed bug is really serious!

Any news? New Image / Release time?