Properly initializing tap interface on boot
Hi, I trying to run a small VM on 6 port alibaba PC. I have installed the bhyve-vm and the UEFI package so the VM is configured and installed and running. The only issues is with the networking. Here is the VM module configuration:
cat /etc/rc.conf.d/vm vm_enable="YES" vm_dir="zfs:zroot/vm"
The VM configuration:
cat /vm/adm-server/adm-server.conf loader="uefi" loader_timeout="5" cpu="2" memory="2G" graphics="no" disk0_type="virtio-blk" disk0_name="disk0" disk0_dev="sparse-zvol" network0_type="virtio-net" network0_device="tap0" debug="yes" uuid="60eee807-b4ac-11eb-8285-0020b7e0213a" network0_mac="58:9c:fc:10:bb:59"
I am trying to modify the underlying system as least as possible. My idea so far is to create a tap interface add it to the already existing LAN bridge and everything should be OK.
The issue is that after startup the tap interface can't connect to the network. There is no ping, DHCP or anything.
If I make some changes to interface via the GUI the traffic starts running with no issue. This makes me think that some initialization of the tap interface is not done at boot up and I still can't figure it out.
ifconfig for the tap0 and the bridge
tap0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> ether 58:9c:fc:10:ff:ea groups: tap media: Ethernet autoselect status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> Opened by PID 53884 bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: LAN ether 0e:4d:3f:1f:ac:20 inet 10.10.0.2 netmask 0xffffff00 broadcast 10.10.0.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: igb4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: igb3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 4 priority 128 path cost 2000000 member: igb5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 11 priority 128 path cost 55 groups: bridge nd6 options=1<PERFORMNUD>
Most of the topics I found were related to bridging OpneVPN tap interface and not much related to running VMs on pfsense :)
Any suggestions how to initialize the tap interface properly? I have tried adding config in /boot/loader.conf and in /etc/rc.conf.d/net but nothing changed.
Disclaimer: I know this unsupported configuration in general but with more and more power HW I think it will be come more useful to deploy small sw controller type of VMs.
After some more digging it seems the tap interface is not set in promiscuous mode at startup as the tap device is missing.
As I mentioned if I do some changes to the bridge, for example start packet capture and enable the promiscuous mode on the interface - the traffic starts.
So the issue seems to be how to enable/create the tap interface early in the init process so it is put in promiscuous mode?
Alternatively how to grammatically restart the bridge once the interface has been created?