Running production pfsense on virtual machine



  • how about running pfsense inside virtual machine for real world use?
    will it be more/less secure?
    more/less stabil?
    how host os affect?
    any other issues?

    tia,
    rex



  • You should NEVER EVEN THINK about running a firewall in a virtual machine…

    because in production environment the data shat shhould be saved from internet harm cannot be saved if it shares the same machine with the data, because when running the firewall on a dedicated box the traffic has to pass thru it and in virtual machines it runs thru virtual channels, so you cannot control...

    it will work however, but i would not recommend at all...



  • I don't agree with trendchiller.
    Why can't  you run a firewall/router inside a virtual machine?

    We use "virtual" routers/firewalls to route traffic between networks(production/test environments) inside our LAN/WAN, and it is working great.
    When using VMware ESX servers, on which the Phisical NIC doesn't have a hardware address, I think(but maybe I'm wrong) it's just as safe as an dedicated "hardware" box.



  • @YoMarK:

    I don't agree with trendchiller.
    Why can't  you run a firewall/router inside a virtual machine?

    We use "virtual" routers/firewalls to route traffic between networks(production/test environments) inside our LAN/WAN, and it is working great.
    When using VMware ESX servers, on which the Phisical NIC doesn't have a hardware address, I think(but maybe I'm wrong) it's just as safe as an dedicated "hardware" box.

    You've probably never broken ESX host security.  I have.  Don't run tasks that require utmost security (firewalls) inside a virtual machine.  The host is infinetly less secure than the virtual machine, you will get compromised.  ESX has it's place - anyone that tries to sell you on it being as secure as the backplane on a Sun E15K has been smoking some NBA quality crack and better be including a few ounces in the deal.

    –Bill



  • I've never broken ESX host security, but I can imagine that it could be done for example via Vmware Tools(inside a Vm) which has a direct connection to the ESX host/Vmware Kernel(via heartbeat, memory management).

    But in the case of a firewall, you first have to hack the firewall(VM) itself before you can do something with Vmware Tools on the VM.
    As far as I know, you have no way of connecting to the ESX host because the NIC('s) itself does not have a hardware(MAC) address, only the virtual machines do.
    (The only direct connection is the service console, which is offcouse not on the same network)

    Am I wrong, and if this is the case I would really like to know ;)



  • @YoMarK:

    I've never broken ESX host security, but I can imagine that it could be done for example via Vmware Tools(inside a Vm) which has a direct connection to the ESX host/Vmware Kernel(via heartbeat, memory management).

    But in the case of a firewall, you first have to hack the firewall(VM) itself before you can do something with Vmware Tools on the VM.
    As far as I know, you have no way of connecting to the ESX host because the NIC('s) itself does not have a hardware(MAC) address, only the virtual machines do.
    (The only direct connection is the service console, which is offcouse not on the same network)

    Am I wrong, and if this is the case I would really like to know ;)

    The host still sees the physical hardware before the guest does.  In my case I've broken service console security from the clean side if you will.  The point I'm trying to make is that you can put pfSense in a VM to add to the security of your ESX installation, but it's not as secure as a physical install.  The firewall VM is only as secure as the weakest guest and it's trust relationship to the host (and yes, there are/were tricks you can do with virtual memory).

    –Bill



  • Allright.  ;D
    Not a secure as a physical install, but stil very usable for situations where security is not the most important thing.
    (For example: routing(/firewalling) traffic between different LAN's/WAN's in your internal network)



  • I was wanting to run pfsense on VMware for my home firewall/ router until I read this thread.  Is VMware increasing these security issues with later releases?  Is there anything similar to VMware that would be more secure?  If xen worked with BSD, I would think it would be less secure.  Is there anything you can do to the pfsense install to make it more secure, like disable the shell or something?

    edit 1 more thing:  If a hacker found my network, would he even be able to tell that the firewall is running on VMware?



  • @jinx099:

    I was wanting to run pfsense on VMware for my home firewall/ router until I read this thread.  Is VMware increasing these security issues with later releases?  Is there anything similar to VMware that would be more secure?  If xen worked with BSD, I would think it would be less secure.  Is there anything you can do to the pfsense install to make it more secure, like disable the shell or something?

    edit 1 more thing:  If a hacker found my network, would he even be able to tell that the firewall is running on VMware?

    Not from remote.  The point is that the host is less secure, not the firewall.  The firewall now sits on the same physical hardware as other possibly less secure hosts (say your web server).  A compromise there results in a guest system on the VMWare host being able to attack VMWare (if there are any vulnerabilities…I know of a few, none exploitable via guests at this time).  If they do manage to compromise the vmware host, obviously it's trivial to exploit the firewall at that point as they'll have control over the firewall guests memory (among other things like say...disk).

    Soooo....is it secure enough to run at home?  That's your call.  Is it secure enough to run in my production work environment?  Hell no....but that's my call.

    --Bill



  • I´m not a security wizard, but I think your firewall could be pretty secure in a Virtual Environment, if you don’t allow traffic on layer 3 or up in the host.

    You need to disable all services on WAN NIC in the host and let the WAN interfaces dedicated to your virtual server.

    Take a look on this:
    http://blogs.msdn.com/virtual_pc_guy/archive/2004/11/12/256294.aspx

    We are running pfSense in a production environment with Microsoft Virtual Server 2005.

    I am happy with the result, because in this way we have redundancy of hardware. If my host gets fire, I just put the disk image in another VS box and get the things up quickly

    Just my 2 cents

    Cheers,
    Anderson



  • One of the other issues I see is time keeping. A vmware machine has an emulated hardware clock which is not accurate. My linux vm varies by a few minutes. It took me kernel line agruments, installing ntp, modifying the ntp settings to disable the local backup timesource. Took awhile to figure that out as ntp would not adjust the time as the vm clock varried too much.

    On a firewall you want an accurate timeclock to keep the logs accurate.



  • @rsw686:

    One of the other issues I see is time keeping. A vmware machine has an emulated hardware clock which is not accurate. My linux vm varies by a few minutes. It took me kernel line agruments, installing ntp, modifying the ntp settings to disable the local backup timesource. Took awhile to figure that out as ntp would not adjust the time as the vm clock varried too much.

    On a firewall you want an accurate timeclock to keep the logs accurate.

    Not to mention some aspects of the networking stack demand a sane time as well.  Think traffic shaping.



  • If you're having timing problems when running pfSense as a VM, make sure that /boot/loader.conf contains the following lines:
    kern.hz="100"
    hint.apic.0.disabled=1

    We are running a few pfSense(simple routers) Vm's on Vmware ESX here, and ntpdate offsets are minimum.

    @rsw686:

    One of the other issues I see is time keeping. A vmware machine has an emulated hardware clock which is not accurate. My linux vm varies by a few minutes.

    On Linux you can install Vmware Tools in the Vm. The tools then do direct time synchronization with the hosts hardware clock. This is very accurate.
    Unfortuanatly The Vmware tools for FreeBSD are not as good as the Linux ones.
    (Howver the solution above seems to be working fine)



  • @YoMarK:

    If you're having timing problems when running pfSense as a VM, make sure that /boot/loader.conf contains the following lines:
    kern.hz="100"
    hint.apic.0.disabled=1

    We are running a few pfSense(simple routers) Vm's on Vmware ESX here, and ntpdate offsets are minimum.

    @rsw686:

    One of the other issues I see is time keeping. A vmware machine has an emulated hardware clock which is not accurate. My linux vm varies by a few minutes.

    On Linux you can install Vmware Tools in the Vm. The tools then do direct time synchronization with the hosts hardware clock. This is very accurate.
    Unfortuanatly The Vmware tools for FreeBSD are not as good as the Linux ones.
    (Howver the solution above seems to be working fine)

    The VMware tools for linux work fine with 32bit guests. The problem is when using a 64bit guest. They fail to keep it accurate. Even with the kernel args "clock=pit nosmp noapic nolapic" I still had this issue. The only work around I found was to enable ntpd and disable the backup time clock in the ntpd config and its been working great.


Locked