Slower than expected iperf3 results under proxmox and virtio

[netnewb2]

Hi,

I'm running pfsense, freenas and a few other linux vms under proxmox.

pfsense to proxmox ~ 2.5 Gbits/s
proxmox (or any vm) to pfsense ~ 1.1 Gbits/s
linux vms to other linux vms or proxmox host ~ 20 Gbits/s

Freenas to proxmox ~ 2.5 Gbits/s

The configurations are pretty standard. Proxmox is using Open vSwitch. Everything is on the same vmbr bridge and using virtio.

Pfsense has 4 cores, 4 GB RAM, CPU type host, numa enabled. All hardware accelerations disabled and so does freenas. I don't have many rules or any IPS.

Since I'm seeing similar results with FreeNAS, I think it's either something with virtio or with proxmox+freebsd.

[netnewb2]

I've setup a new pfsense vm with nothing on it. Results were more or less the same, so the packages weren't interfering.

So, I've setup a new bridge and set MTU to 9000 and now I'm getting about 9 Gbits/s (sometimes 10 with pf disabled). I don't understand networking enough as to why linux VMs work at 20 Gbits/s with MTU 1500.

Other settings such as CPU type, machine type, rx off on the bridge, didn't affect results that much.

Now I'm curious what it would take to get to 20 Gbits/s