Small update: I had to re-create PCIe devices, including disk and network. Not sure why, but they remained "legacy" no matter what, so I copied and tweaked config from another VM that was recognized as "modern". Only bus and slot were different, yet it somehow made a big difference.

So make sure VM is running as UEFI and re-create devices if necessary, then they'll be recognized as "modern".

@nazar-pc said in Chelsio T4 VF is not recognized as NIC:

I mean cxgbev for virtual function, not cxgbe for physical device

Yep, I understood and you are right. I wanted to write "and not available as a module"

@SteveITS thanks appreciate this. That's what I ended up doing. PITA for sure lol. Having some residual weirdness, like packages didn't come back and it won't let me install any. Saying I have an instance of pfsense already running.

Wish there was a way to add the config I edited with my new interface names to my vmdk that had all the packages etc.

@louis2 I do passthru of disks to TrueNAS and I pass thru the NIC's (LAN and WAN) to pfsense, simply because I want the application to be in total control of what they each do best. But I also have a second TrueNAS at our summer home where disks are virtualized and I have been running firewalls with virtualized NIC's, no problem... In fact I do have my failover WAN assigned using VirtIO, and no issues whatsoever...

In the mean time you can still always make backups with it stopped. That has always worked. Though obviously it's far more inconvenient.

We are digging...

@mic-bummer That appears to be a bug in the pfsync code.
Try the 2.8.0 beta. There are a large number of pfsync fixes in there, and it's quite likely that this problem is already fixed there.

@stephenw10 My bad, i'm trying with this.
I will tell you if it solved my issue.

@Ghost-0 Well, I have been using Docker/Portainer to deploy a number of applications and services. But I have no clue about Frigate or the Coral that you mention. The way I have tried to install most or all my Docker containers is as Stacks. And in case I have been unable to find a Docker Compose file to use, I have actually asked ChatGPT or Claude to create one based on a Docker command.
The benefit is that the structure is much more visible and simpler to edit.

Seems to me it's a fairly complex installation that you are trying to do. I found this youtube video of a guy installing what I think you are also attampting? https://www.youtube.com/watch?v=zKk9dnAp8FM

I did reboot pfsensene VM and made sure from ip a in the proxmox host that there was no IP set for the interfaces I was bridging and it magically worked.

Actually, now it's even better. In the proxmox interface I managed to set an IP for the interface that is being bridged, so that when pfsense VM is down I can still reach the proxmox on its fixed ip on the LAN side.

@FInnishFlash Doesn't work even with bridged interfaces, and virtio

@viragomann
I made a mistake in my previous message, sorry about that. Unfortunately, the traffic never reaches the LAN. If it did, the VM would already be accessible.

@adamk11 How many and what type of NICs are you using? Are we to assume that pfsense LAN, and all VLAN's share the same physical port or do you have them separated?
If they are using the same physical port, I'm thinking traffic will go.

From VM1 to switch, back into pfsense VLAN1, out again to the switch on VLAN2, and then return back on the same interface to VM2. That would create a tromboning effect that may limit your throughput, likely to something less than 5, or?

Generally speaking though, my experience from Proxmox using X520 NICs and VirtIO, I don't see any limitations in that regard. Running iperf between two FW's (over WAN interfaces) I get 8+ Gbps. I do have NIC's passed thru (IOMMU) to firewalls, but not to VM's.

Just now tested between two VM's in different VLAN's reaching 9.33GBit/s without any problems whatsoever, both VirtIO.

The problem was the bridges. When creating the bridge for lan it should be tied to the physical network port and have a static ip address. All fixed.

For your information, the two 1:1 NATs (one NAT per IP) are configured on the same interface.

Hard to see how it could be anything else. 😞

@ethanthekiwi said in SCSI error on VM:

For me this issue was caused by thin provisioning on the virtual hard drive. I followed VMware's instructions to "inflate" the disk to thick provisioning and I stopped getting these errors.

Reply

Yeap same solution (https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/vsphere-storage-8-0/working-with-datastores-in-vsphere-storage-environment/using-datastore-browser-in-vsphere-environment.html#GUID-C371B88F-C407-4A69-8F3B-FA877D6955F8-en) worked for myself as well. :)

Just an update for my posted problem.
It turns out that it caused by unstable SDN of Proxmox.

People in Proxmox forum suggest me to use the more robust "Network Bridge".
Now, the connection between the VMs seem to be more stable.
In other words, pfSense has no problem at all.

Thanks to those who've read and replied to my post.