Bhyve VM not able to access the internet

sdf_iain

This is probably something simple that shows my inexperience with FreeBSD, but here goes.

I recently set up pfSense on a Zotac CI323 and I'd like to replace another machine on my network (which is running dhcpd and bind, if that matters) with a VM using bhyve. I started out with esxi on this box, but it isn't really supported the Zotac does not support passthrough.

I've installed the vm-bhyve and grub2-bhyve packages and run the following

mkdir -p /var/vm
echo "vm_enable=\"YES\"" >> /etc/rc.conf.d/vm
echo "vm_dir=\"/var/vm\"" >> /etc/rc.conf.d/vm
vm init
vm switch create public
vm iso <location of="" a="" debian="" net="" install="" cd=""></location>

… here i copied the debian.conf template from vm-bhyve's git repo to /var/vm/.template/debian.conf...

vm create -t debian -s 8G debian
vm -f install debian <debian iso=""></debian>

Then I watch the firewall logs in the pfsense gui.
If I add tap0 as an interface in the pfsense GUI I can add firewall rules that allow the VM to reach my dhcp server and get an address.

It mostly works… until it needs to reach out to the debian repositories. There's LAN access, but no WAN access. Nothing more appears in the firewall logs. Its as if tap0 doesn't have a route to the gateway. Anybody have any ideas?

Additionally, how does one close out the console when using 'vm console' over ssh?

sdf_iain

I have also tried disabling vm in /etc/rc.conf.d/vm and rebooting. Then creating a bridge in the pfSense GUI and adding a descripition to the bridge so that vm will use THAT bridge for its "switch"

ifconfig bridge0 description vm-public

That get's the bridge into a state where I can manage it in the GUI as well.

sdf_iain

Does anybody have any ideas? Or should I give up until bhyve is configurable from the GUI?

Is there more specific information I could provide? The config is fairly vanilla.

I've added and enabled squid as a transparent proxy. I've also configured an ip6 tunnel via hurricane electric.

sdf_iain

Should I assign the bridge0 to LAN, create another interface (OPTx…. but we can change that to PHYSICAL or ETHERNET, for greater clarity) for the current LAN device (re1), and assign that to the bridge? Will that even make a difference?

I set


net.link.bridge.pfil_member = 0
net.link.bridge.pfil_bridge = 1

and rebooted, but that hasn't seemed to change anything.

sdf_iain

Seems that a functional VM using vm-bhyve is NOT possible at this time. Hopefully a future version of pfSense will rectify this.

sdf_iain

Tenacity prevails!

If you assign the bridge to the LAN interface and add both your previous lan interface and the tap0 interface to it, then you can attain access to the internets for your vms.

1. Create a bridge in Interfaces:(assign):Bridges
2. Go to Interfaces:(assign) and determine which device is assigned to LAN
3. Go to Disgnostics:Command Prompt
4. Enter ifconfig bridge0 addm <device assigned="" to="" lan="">5. Go back to Interfaces:(assign) and set BRIDGE0 as the LAN
6. Add the previous LAN device
7. Enable that device
8. Go to Bridges:BRIDGE0 and assign your new device (OPT1?) to BRIDGE0

When you add the device to the bridge with the command prompt, you make certain that the bridge has access to the wire.
When you add the device to the bridge in the GUI, you make that persist between reboots.

It works! If anybody want to use Netflix with a Hurricane Electric IPv6 Tunnel you can use a VM to provide a barebones BIND install to filter out Netflix's IPv6 addresses.

If someone besides me shows interest in this thread, I could make the instructions more cohesive ;D, but for now I'll leave this as is and hope it's useful to anyone else who wants to do this.</device>