Performance issue on vmware esxi 7
-
I am running into a weird performance issue with pfsense 2.5.2 on Vmware ESXi 7.0 u2. I have no idea if this performance issue exists on versions before and after for either pfsense or Vmware.
But I have gigabit internet and my ESXi host has multiple NICS, one is dedicated to a pfsense firewall VM as the WAN interface, and another as the LAN interface. When the pfsense is functioning as firewall, the best download speed I can get is 250Mb/s. If I turn off the pfsense firewall and fire up a nethserver firewall, I get 950Mb/s.
I can turn off filtering and run speedtest directly from pfsense with the cli speedtest and I still only get 250-300MB/s.
I have the NICS defined as VMXnet 3 NICS, but I have tried Intel NIC emulation and I have turned off all the acceleration options in pfsense as well as Spectre mitigations and nothing gets the speed where it is supposed to be.
Shut down that VM and load up a VM with the exact same host ports and it works like a champ at the rated speeds.
-
@rtkluttz Since no response, I'm kind of grasping at straws. Can I backup this configuration and install an older version of pfsense and restore the config to that to see if it is version related? I'm on 2.5.2 now.
-
@rtkluttz if you can configure your hardware so the NIC used by pfsense are only used by pfsense I would then use pass through for the NIC to remove the hypervistor overhead and allow hardware off loading
-
What is your esxi host specs?
You should be able to do gigabit with vmx adapters easily if you have a relatively decent machine.
What is your cpu usage look like when you do a speedtest?
Btw, You have to disable hardware large receive offload from advanced settings. Depending on the model of your physical network adapter the vmx adapter may or may not be able to handle hardware checksum offload and tcp segmentation offload.
-
@netnerdy and also answers for others above...
This is a supermicro server that was running an older version of pfsense with no performance issues that had a hard drive crash.
Its an 8 core Xeon server with 32GB of ECC ram. The same virtual NIC runs at full speed with a different OS on it.
For example.. running PFSense 2.5.2 and doing speedtest directly from the PFsense CLI, I get ~250Mb/s. That is with the NIC dedicated to WAN on the Pfsense. No other vms on the WAN vswitch at all.
If I shut down the pfsense vm and fire up a Nethserver VM with the firewall enabled on that and add the RED NIC to the same vswitch, it gets full rated speed on speedtest.Considering that there were no performance issues before my old HD died, It looks like there is a performance issue 2.5.2 or maybe the 2.5 series. But its a ton of work to reinstall and configure back to pfsense 2.4 series.
-
For reference I have esxi 7 running on a passively cooled i5 7200 u and I can get symmetric gigabit using vmx adapters.
Have you tried turning on/off the various options I listed in my previous post? Before concluding there is performance issue you have to make sure you have the right configuration for your hardware. FYI, You will have to restart pfsense after every change.
-
@netnerdy said in Performance issue on vmware esxi 7:
Btw, You have to disable hardware large receive offload from advanced settings. Depending on the model of your physical network adapter the vmx adapter may or may not be able to handle hardware checksum offload and tcp segmentation offload.
I've been chasing random drops in upload speeds with pfsense 2.6 installed on a esxi VM and this fixed it. Thankyou.