pfSense 2.7.2 + proxmox = low throughput

adamk11

Hi,

I have virtualized pfSense 2.7.2 on proxmox 8.3.3, od SSD, ZFS with deduplication and compression.
My config has many vlans (~100), a lot of IPSec tunnels (~20), a few OpenVPN servers etc.
On proxmox I use VirtIO network card, 10vCPU (Intel(R) Xeon(R) Silver 4114 CPU as a host), 16GiB od RAM, fast SSD drives for system.
IMO performance is not goot, when I did iperf from VM to VM, between two VLANs throughtput was ~1.2Gbps TCP, and 2,5Gbps UDP. Both VMs and firewall was on same proxmox hypervisor.
I did same test, but imported config without proxmox, to baremetal, throughtput with iperf was ~10Gbps. Hardware firewall was connectted by one 10Gbps wire (DAC cable) to core switch.

When I install pfSense, without my config, iperf between vlans on virtualized pfsense was ~3,5gbps, it's still less than 10Gbps.
I have istalled zabbix-agent on the pfSense, load is not high.

Gblenn

@adamk11 How many and what type of NICs are you using? Are we to assume that pfsense LAN, and all VLAN's share the same physical port or do you have them separated?
If they are using the same physical port, I'm thinking traffic will go.

From VM1 to switch, back into pfsense VLAN1, out again to the switch on VLAN2, and then return back on the same interface to VM2. That would create a tromboning effect that may limit your throughput, likely to something less than 5, or?

Generally speaking though, my experience from Proxmox using X520 NICs and VirtIO, I don't see any limitations in that regard. Running iperf between two FW's (over WAN interfaces) I get 8+ Gbps. I do have NIC's passed thru (IOMMU) to firewalls, but not to VM's.

Just now tested between two VM's in different VLAN's reaching 9.33GBit/s without any problems whatsoever, both VirtIO.