Bridged networking in VirtualBox not working on pfSense host

[Ektorus]

Hey all,

I have a problem getting "bridged adapter" to work in VirtualBox. I have a 2 NIC setup on my pfSense server. The server has a WAN (re0) and a LAN (em0) interface. I have perfect internet connectivity from my pfSense system and any other computer on the local LAN (which all are connected the em0 interface). I'm now trying to create some virtual machines in VirtualBox, which I need to run in briged mode so I can access them remotely from the local LAN. I'm running VirtualBox from the command line, as I have no X11 or screen on the server. I've configured the briged adapter to use my LAN interface, i.e. the em0 interface on the pfSense server.

The problem is that I don't have any connection between my host and guest at all. The connection only goes from the guest to the host, but not the otherway around. When I try to obtain an IP address from the guest I get the following messeges on the pfSense host:

DHCPOFFER on 172.30.123.13 to 08:00:27:2b:b8:81 via em0
DHCPDISCOVER from 08:00:27:2b:b8:81 via em0
DHCPOFFER on 172.30.123.13 to 08:00:27:2b:b8:81 via em0
DHCPDISCOVER from 08:00:27:2b:b8:81 via em0
DHCPOFFER on 172.30.123.13 to 08:00:27:2b:b8:81 via em0
DHCPDISCOVER from 08:00:27:2b:b8:81 via em0
...

So on the host machine I can seethe DHCP request from the guest. However, on the guest OS i get the following messages:

# cat /var/log/messages
...
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
...
No DHCPOFFERS received.

I use em0 (the LAN interface) as the bridge adapter in virtualbox. The interface is always in promiscuous mode (if that has anything to do with this). The guest OS is CentOS 6.5 and the host OS is pfSense 2.1 (using FreeBSD 8.3). The configuration of the network interface in the guest OS:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=...
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=dhcp

I've also tried with NM_CONTROLLED=no, but that didn't make any difference. I've pretty much followed the guide from http://wiki.sysconfig.org.uk/display/howto/VirtualBox+On+FreeBSD, but it doesn't seem to work with bridging in my setup. I haven't installed any guest additions in the guest OS, but this shouldn't be necessary for networking to work. This is the version of VirtualBox I use on my pfSense host:

# VBoxHeadless --version
Oracle VM VirtualBox Headless Interface 4.2.16_OSE
(C) 2008-2013 Oracle Corporation
All rights reserved.

4.2.16_OSEr86992

I've tried to use all the different nictypes (those listed in the documentation - Am79C970A|Am79C973|82540EM|82543GC|82545EM|virtio), but none of them were able to obtain an IP address. I also tried using a static IP:

# ifconfig eth0 inet 172.30.120.90 netmask 255.255.252.0
# route add default gw 172.30.120.1

And then pinging the host machines, but I didn't get any response.
Does anyone have any ideas on how to solve this issue?

[jimp]

In VirtualBox, on the firewall's NIC settings, make sure "Promiscuous Mode: Allow All" is set. I seem to recall having issues with vbox bridging on one host until I did that. Though it shouldn't normally be required for standard traffic, some host OS quirks may lead to it being necessary. It is required for CARP to work on virtualbox.

[Ektorus]

Setting the NIC interface option to "Promiscuous Mode: Allow All" didn't make any difference.
I would assume (without much evidence, though) that it were some sort of internal routing problem. Like sending a request from one interface to itself. But again I could very well be wrong and the problem lies somewhere else.

[bryan.paradis]

pFsense on baremetal
Running Virtual Box
CentOS is guest
Guest is using pfsense LAN interface in bridged mode

Am I understanding correctly?

Have you tried manually assigning IP addresses?

Hvae you loaded the vboxnet driver?

http://www.freebsd.org/doc/handbook/virtualization-host.html

[Ektorus]

Yes you are absolutely correct.

I have tried to manually set the IP address in the CentOS guest and adding the default gateway, but that didn't help either.

Yes, I have loaded all the required modules:

# kldstat
....
13    3 0xffffffff8195c000 35b44    vboxdrv.ko
16    2 0xffffffff81953000 2a00     vboxnetflt.ko
17    1 0xffffffff81956000 3f90     vboxnetadp.ko

[cdburgess75]

Were you able to get that going?  I am trying something very similar

[Ektorus]

Unfortunately I were not able to get it working. So I'm currently waiting for pfSense 2.2 to be released (it is currently in BETA) and hope to get the hypervisor in FreeBSD-10 (which pfSense 2.2 is built upon) to work.

[johnpoz]

So let me get this right, you installed virtual box on the host pfsense is running bare metal on?  So you installed some freebsd 8.3 virtualbox package on your pfsense box.  And then trying to run another vm in this virtualbox?

If you want to run vms on this hardware, why don't you just run a a type 1, say esxi or xenserver or even hyper-v and then run pfsense as vm, and your centos as another vm, etc.

I run esxi (free) and pfsense is vm, along with lots of other vms.  Pfsense is the gateway/router/firewall for all the vms along with all the physical boxes on my network.. This pfsense vm also routes traffic between physical and vms on multiple segments.

Trying to run pfsense bare on the host and then run some type 2 hypervisor seems like the wrong way to go if you know your going to want to run vms on the host.

[Ektorus]

Yes that is situation.

But you are most likely correct that I should run pfSense as a guest instead of the host system. It just seemed more intuitive to run the main system as the host and just have a few guest systems on top of that.

[johnpoz]

No, if your going to run multiple vms in a production type setup where they are on 24/7 sort of thing.. Then type 1 normally makes more sense.  If your going to need to fire up some play vms now and then then type 2 makes more sense since the host will normally just be doing its main function.

If your going to want pfsense and centos running all the time I would look to a type 1 vm setup.  Esxi is free, easy to use and has lots of support for lots of different hardware, etc.  One of the things I love about running pfsense as vm, its easy to take a snapshot before say you update.  Or if you were going to make lot of changes to firewall rules.. Just take a snapshot before, and if something goes haywire you just reboot to the snap.

If your running pfsense (freebsd) as the main os and something goes wrong with say a update now what you have to restore baremetal, reinstall pfsense, etc. - while also taking down your other vms, etc.

If your pfsense is vm, also gives you freedom to say bring up a 2.2 version.  Bring it up in parallel with your running vm - get it working, and then if you want you can even shut down your original and bring up your 2nd copy with same macs on the interfaces..  So maintain your dhcp from your isp, clients on your lan think its the same box, etc.  Allows you to easy play with a carp setup in pfsense if you want with really only 1 physical box, etc.

No I would not suggest pfsense as your host main OS with some type 2 running in that with other vms..

[Ektorus]

I see, thank you for the explanation. I'll take a look at Esxi.