Hey BlueKobold,
thank you for your suggestions. We also just recieved an answer from the pfSense-Support.
But i will answer your Questions as good i can :)
@BlueKobold:
We use iperf to test the throughput between the firewall and a virtual machine.
Are they both in a VM? I mean pfSense and the virtual server?
We tried both of them. The virtual firewalls most limited by there amount of cpus and often by the featuresets.
After activating TSO and LRO we also reach 5GBit/s with the virtual pfsense.
@BlueKobold:
If the firewall is the "Server" and the virtual machine is the "client" we only get a throuput about 3GBit/s.
In normal you will be getting something between 2 GBit/s and 3 GBit/s as throughput in real life, from a 10 GBit/s link.
Yes, of course we are talking about a theoretical throughput, but i would expect a similar throughput in both sides of communication, right?
@BlueKobold:
If we send from the firewall to the virtual machine we reach a throughput about 8/9 GBit/s.
Perhaps the virtual machine is able to write the data faster then the pfSense, because there are a RAID in or more
RAM that is acting as buffer for the packets, might this be?
We never send a real amount of data over the cable :) with iperf you send an amount of packets with embedded timestamps and sequence numbers. With this content iperf calculates his statistics.
@BlueKobold:
It does not matter if it is a virtual or a hardware pfSense.
It does for sure! How many cpu cores are given to the pfSense machine?
See my answer above. Of course it matters, because of the amount of cpu - i had to be more specific i think ;)
I mean, it does not matter with the strange behavior of different throughput. But as i said before, when the firewall sends his packets, it expect an ACK after everyone, the vm does not.
So we activate TSO and now the firewall dont expect that anymore - just TSO
@BlueKobold:
We just activate TSO and LRO on the pfsense.
Tunings can be often helping much more then we all would expect from!
high up the mbuf size
shorten down the NIC queues to 4 till 6
and other options or tunings might be helping also, please give them a try out, single or together!
Anyone an idea or some experience with that features on a pfsense?
Tuning and Troubleshooting Network Cards
I checked that article, everything was okay. Tuning the machine is the first i thought about. Troubleshooting the second ;)
BlueKobold, thank you very much for your help.