For future reference I made a summarized the all the information possibly required:
My setup:
Proxmox version: 8.1.4
pfSense version: 2.7.2
NIC: i340-t4, i219 (motherboard)
Network configuration:
vmbr0 is assigned to LAN in pfsense and all other VMs in proxmox, it also has slaved physical port (i340-t4) that connects to rest of the lan
vmbr1 is assigned to WAN in pfsense and it has slaved physical port (i340-t4) to ISP1(DHCP)
vmbr2 is assigned to WAN2 in pfsense and it has slaved physical port (i340-t4) to ISP2(PPPoE)
vmbr4 is assigned for proxmox management/cluster only and it has slaved physical port (i219) that connects to same physical switch as vmbr0/rest of the lan
The issue:
Port forwarding works when using NIC passthrough, but not when using virtIO
Specifically, port forwarding doesn't work for the DHCP ISP connection when using virtIO, but does work with PPPoE ISP2
I have tried:
disable hardware offloading in pfsense
ethtool -K XXXX rx off tx off for physical ports as well as vmbr(0-4) on proxmox
manually changing MAC Addresses on vmbr(0-4) in case there would be a conflict, especially vmbr1 having same MAC as the physical interface
This is my /etc/network/interfaces with manual MAC Addresses, to test without that I just comment out the vmbr hwaddress lines:
auto lo
iface lo inet loopback
auto enp1s0f2
iface enp1s0f2 inet manual
iface enp1s0f3 inet manual
iface enp1s0f0 inet manual
hwaddress XXXXXXXXXX
iface enp1s0f1 inet manual
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet manual
bridge-ports enp1s0f3
bridge-stp off
bridge-fd 0
hwaddress 90:e2:ba:37:0d:a0
#LAN
auto vmbr1
iface vmbr1 inet manual
bridge-ports enp1s0f0
bridge-stp off
bridge-fd 0
hwaddress 90:e2:ba:37:0d:a1
#Antik
auto vmbr2
iface vmbr2 inet manual
bridge-ports enp1s0f1
bridge-stp off
bridge-fd 0
hwaddress 90:e2:ba:37:0d:a2
#Telekom
auto vmbr4
iface vmbr4 inet static
address 192.168.0.70/16
gateway 192.168.0.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
hwaddress 50:65:f3:48:34:a4
#PVE