Weird performance issue esxi6.5 multi-wan(pppoe) throughput multicore CPU
-
Hello Guys,
I am running several years of pfSense with satisfy in esxi with vmxnet3 interfaces. Few days before 1Gbit FTTH(pppoe) was integrated into the box. I used 4 vCpu. Then I measure the speed (only the new 1Gbit gateway was active) and I was shocked to the results. I've got 50Mbit downstream. After hours diggin in various forums I realised something is wrong with the CPUs. I decrease the number 1 core only, and miracle happens. I've 500Mbit! But the processor is burning when throttling... After this I re-add additional cores, but the throughput decreases as much as more CPUs have been added. I know pppoe is a single threaded thing in pfSense, but I cant belive such bottlenecks....
I tried hw offloading enable/disable, but not noticeable changes...
Can someone please give any advice how to solve this annoying issue?Thx!
Config:
Intel(R) Xeon(R) CPU E5506 @ 2.13GHz
8GB ram
2.4.4-RELEASE-p3 (amd64)
3 WAN -1 DHCP -2 PPPoE -
That is hard to explain. We will need more info.
Try running
top -aSHwhen you are testing to see how the load is distributed.Do you have
hw.pci.honor_msi_blacklist=0set? Without that vmx only uses one queue per NIC.Steve
-
Thanks for the suggestion I've added the line into the tunables section. Might a bit better. Which is the best multi core strategy in pfSense? 1 socket more cores or separate sockets?
-
No significant difference really.
There are some things that are still single threaded, like PPPoE. So fewer but faster cores can be help in some configurations.
Steve
