Properly initializing tap interface on boot

idimitro

Hi, I trying to run a small VM on 6 port alibaba PC. I have installed the bhyve-vm and the UEFI package so the VM is configured and installed and running. The only issues is with the networking. Here is the VM module configuration:

cat /etc/rc.conf.d/vm
vm_enable="YES"
vm_dir="zfs:zroot/vm"

The VM configuration:

cat /vm/adm-server/adm-server.conf
loader="uefi"
loader_timeout="5"
cpu="2"
memory="2G"
graphics="no"
disk0_type="virtio-blk"
disk0_name="disk0"
disk0_dev="sparse-zvol"
network0_type="virtio-net"
network0_device="tap0"
debug="yes"
uuid="60eee807-b4ac-11eb-8285-0020b7e0213a"
network0_mac="58:9c:fc:10:bb:59"

I am trying to modify the underlying system as least as possible. My idea so far is to create a tap interface add it to the already existing LAN bridge and everything should be OK.
The issue is that after startup the tap interface can't connect to the network. There is no ping, DHCP or anything.
If I make some changes to interface via the GUI the traffic starts running with no issue. This makes me think that some initialization of the tap interface is not done at boot up and I still can't figure it out.
ifconfig for the tap0 and the bridge

tap0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 58:9c:fc:10:ff:ea
        groups: tap
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 53884

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        ether 0e:4d:3f:1f:ac:20
        inet 10.10.0.2 netmask 0xffffff00 broadcast 10.10.0.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: igb4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        member: igb3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000000
        member: igb5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000000
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 55
        groups: bridge
        nd6 options=1<PERFORMNUD>

Most of the topics I found were related to bridging OpneVPN tap interface and not much related to running VMs on pfsense :)
Any suggestions how to initialize the tap interface properly? I have tried adding config in /boot/loader.conf and in /etc/rc.conf.d/net but nothing changed.

Disclaimer: I know this unsupported configuration in general but with more and more power HW I think it will be come more useful to deploy small sw controller type of VMs.

idimitro

After some more digging it seems the tap interface is not set in promiscuous mode at startup as the tap device is missing.
As I mentioned if I do some changes to the bridge, for example start packet capture and enable the promiscuous mode on the interface - the traffic starts.
So the issue seems to be how to enable/create the tap interface early in the init process so it is put in promiscuous mode?
Alternatively how to grammatically restart the bridge once the interface has been created?