pfSense on ESXi | Best Practices

kiokoman

esxcli network nic tso set --enable=0 -n vmnic0
esxcli network nic cso set --enable=0 -n vmnic0
?
i'm pretty sure it's not a pfsense problem
try to use another physical network interface card to connect the virtual switch to the physical switch to eliminate physical problems.

esxi version? ESXi 6.0 is known to have host loses network connectivity randomly
check /var/log/vmkernel.log file, if there is evidence of transmit timeouts

also check if this article can help you
https://kb.vmware.com/s/article/1004109

mohkhalifa

Thanks @kiokoman for you help, really appropriated. But why when I directly connected the WAN without pfSense everything goes fine ? also I tried to change the adapter type from e1000 to VMNET3 and the same problem !!

Rico

Again: Latest VMware Tools installed?

-Rico

mohkhalifa

@Rico yes sure

mohkhalifa

@Rico Open-VM-Tools v10.1.0_2,1

kiokoman

did you set traffic shaping or load balancing on the vswitch ?

mohkhalifa

@kiokoman NO

kiokoman

i'm tring to mess as much as i can with my vm but i'm unable to reproduce it
can you do a test with pfsense 2.5.0-devel ?
also i found this about the vmx driver
https://www.freebsd.org/cgi/man.cgi?query=vmx&sektion=4
ethernet0.virtualDev="vmxnet3" i have it inside the virtual machine configuration and
The hw.pci.honor_msi_blacklist tunable must be disabled to enable MSI-X support.
i have it set on my
/boot/loader.conf.local

hw.pci.honor_msi_blacklist="0"

also i found this but idk if it's still relevant
https://forum.netgate.com/topic/88082/esxi-5-5-packet-loss/12

bbrendon

I have a bunch of pfsense instances running in ESXi. No issues. All 2.4.4-p3 and 2.4.5-RC.

Rico

Yeah I don't think this problem is pfSense related.
Can you spin up two more VMs for testing? One with vanilla FreeBSD 11.2 and another with Linux or Windows. Check if the timeouts happen in one or both test VMs.

-Rico

mohkhalifa

problem SOLVED after "Disabling hardware checksum offload"

kiokoman

i told you 4 days ago to disable cso "esxcli network nic cso set --enable=0 -n vmnic0"
but i forgot to tell you to disable it from the pfsense gui

mohkhalifa

@kiokoman

bbrendon

@mohkhalifa said in pfSense on ESXi | Best Practices:

problem SOLVED after "Disabling hardware checksum offload"

Awesome. I poked around on a few of mine and didn't find any with that enabled. Mostly Dell hardware here. Good find.