More than 1 Gbps using VMWare ESXi with VMXNET3?
-
Hello, I'm not sure what forum this should go in, so I figure this is a good start.
I'm utilizing pfSense 2.4.3-RELEASE (amd64) built on Mon Mar 26 18:02:04 CDT 2018 FreeBSD 11.1-RELEASE-p7.
It is running ontop of a VMWare ESXi host running 6.0.0, Build 3620759.It is also running Open-VM-Tools 10.1.0,1.
When the Firewall was set up initially it was configured with E1000 VMNIC's. The problem with this is that all of the network interfaces are limited to 1 Gbps, but the physical server this firewall is running on has 10 Gbps NIC's.
From my understanding, to get a 10 Gbps link, I'd need to reconfigure the interfaces as VMXNET3 VMNIC's.
I'd like to know, what would be the best way to do this? The firewall is only a single firewall to the entire site, but I do have a "backdoor" way to get in if connectivity fails.
Are there any caveats I should know about?
-
You will have to do this onsite. No way you're going to add & remove NICs and reconfigure everything remotely, unless you have another way in to the network. The way that I've seen others do this is:
- make a config.xml backup via Diagostics - Backup & Restore
- power down the vm
- in vmware, remove all the e1000 NICs
- in vmware, add your vmx3 NICs
- edit your config.xml file and replace all instances of em0, em1, em2 etc with vmx0, vmx1, vmx2 etc.
- install pfSense fresh
- restore your config.xml
-
OK, So I've changed over all of the NIC's to VMXNET3, but I'm still not seeing the throughput I want.
The physical NIC's on the server are reporting back 10000 Mbps and they are on a LAG on a virtual distributed portgroup, so in theory, bandwidth should be 20 Gbps.
When I take a peek in the shell of pfSense, an ifconfig | grep media just reports back "Ethernet autoselect" for all interfaces.
I'm honestly not sure how else I can test. The server is running a Xeon E5-2650 @ 2.3 Ghz and it has 40 Logical processors, so that's probably not an issue. The VM itself has 2 vCPU with 1 socket/2 cores. CPU load never goes above 70 percent.
What else can I tweak to get more throughput?
-
And what throughput are you seeing?
-
@johnpoz When I go to speedtest.net and run a test against our ISP (Who is on the same network) I get about 700 Mbps, which is what I would expect for a 1 Gbps connection since there's other traffic going across it.
I think it won't hurt to also have a chat with our Data Centre provider as well to ensure that they're also not throttling us.
I'll update the forums as soon as I find out.
-
right .... you are having too many unknown factors.
- you won't get 10Gbe from speedtest.net (or any other similar that i know)
- 10Gbe over the internet will currently be hard to reach due to latency
- pfsense does not do 10Gbe on small sized packets .... on minimum sized packets its maxes out at around 2.7gbit/s
- you can have a trillion cores ... it doesn't matter. ghz has a bigger impact then huge core numbers.
-
@bbennett That's a poor way to test throughput. Use iperf on each side of your WAN. Testing a 10G connection over the Internet is crazy.