Need help gigabit performance

Pitbully · Mar 10, 2018, 4:21 PM

Hi,
I just moved to gigabit speed at home (2 users); however I can't achieve anything over 300Mb to speedtest.net using my 16gig supermicro c2758 which of course is VM. I have now started from a fresh build :'( I have changed my Mbuf to 1 million I believe I followed all the performance recommendations and my performance still sucks. Any help would be greatly appreciated.

Pitbully · Mar 10, 2018, 3:36 PM

Here are some screenshots

stephenw10 · Mar 10, 2018, 4:09 PM

That should easily pass 1Gbps if it was running bare metal. Are you able to test that?

Have you tested the WAN directly, just a client connected to the modem/ont?

Steve

Pitbully · Mar 10, 2018, 4:31 PM

I had he modem only in gateway and I was able to attain 800Mbps as soon as I put the pfSense inline I cant get anything over 300. I am with Rogers Canada, Hitron CGN3 is the modem currently in bridged mode. When you say baremetal I assume you are talking build out of the box its Supermicro SYS-5018A-FTN4 6 cores and 8 gig dedicated to the box for 2-4 users :(.

The last picture is the box as I was doing testing to speedtest.net and the results.

Pitbully · Mar 10, 2018, 4:36 PM

Here is the box and the second test to Rogers own speed test.

gjaltemba · Mar 10, 2018, 4:54 PM

Supported modem for Rogers gigabit is Hitron CODA-4582. If your CGN3 is a rental you get a free modem swap.

Pitbully · Mar 10, 2018, 5:36 PM

Sorry you are correct t I am running hitron coda 4582.

stephenw10 · Mar 10, 2018, 5:38 PM

@Pitbully:

When you say baremetal I assume you are talking build out of the box…

I mean install and run pfSense on the hardware directly rather than in VMware as a test. You could potentially do that running from a USB stick for example. That would prove the hardware compatibility.

I would expect to get 800Mbps through that box easily. It looks like some VMware issue you're hitting there. Even with the overhead introduced running virtually you should see more than 300Mbps.

Do you have open-vm-tools installed? https://doc.pfsense.org/index.php/Open_VM_Tools_package

Steve

Pitbully · Mar 10, 2018, 6:23 PM

Thanks for taking the time out to help. I just put my modem back into gateway mode and here are the results. I wasnt connected directly to it but through a nortel 4548 1gig switch. I will try to run from usb give me a few to set it up. The only tools I installed was the open-vm-tools.

Pitbully · Mar 10, 2018, 7:03 PM

Well its something within the VM that I am doing wrong. USB pfsense speeds below, should I bother running it in a VM?

SammyWoo · Mar 10, 2018, 7:29 PM

To me VM should only for testing, once you are in production, should run on a dedicated, or "bare metal" as he says.

Pitbully · Mar 10, 2018, 7:37 PM

Thanks, I thought I would be able to run this box in a VM as its for home only not in production.

stephenw10 · Mar 10, 2018, 8:54 PM

I would not expect the hypervisor overhead to make that much difference normally. It looks like you could probably tune that better. You might want to ask in the virtualisation section.

Steve

A Former User · Mar 11, 2018, 10:48 PM

What Ethernet cards are you using in your VM?
Are you using Virtualised ones? vmxnet3 in Vmware or VirtIO in KVM?
Have you turned off all hardware offload?

Tim

Pitbully · Mar 12, 2018, 12:27 PM

I had the offload checked. I was using the e1000 nics and the ethernet cards and its a C2000 SoC I354 Quad GbE Controller. From the supermicro 5014 board from http://www.supermicro.com/products/motherboard/atom/x10/a1sri-2758f.cfm

dmurphynj · Mar 12, 2018, 1:39 PM

I have a similar setup and you should have no problem pushing that kind of traffic ….

WAN provider: Verizon FiOS (Gigabit symmetric)
System: Supermicro SYS-5018A-TN7B (A1SRM-LN7F-2758 system board)
Interfaces:
WAN - ix driver/ Port 1 on an Intel X520 (10gb) PCI3 board with 1gb SFP+ installed
LAN - ix driver/ Port 2 on the same Intel X520 (10gb) PCIe board with 1gb SFP+ installed
OPT1GUEST - igb driver/ gigabit port on the SuperMicro built-on i350-AM2 controller (guest VLAN)
Installation: Bare metal
System drives: Pair of SanDisk SSD Plus SDSSDA-120G drives; using a zpool mirror configuration.

The performance is excellent. Absolutely excellent; zero concerns here.

I did add 3 system tunables for the X520 card:

kern.ipc.nmbclusters=9168192
kern.ipc.nmbjumbop=524288
hw.intr_storm_threshold=10000

But otherwise, things are running very well out of the box. See attached speedtest.

![Screen Shot 2018-03-12 at 9.31.53 AM.png](/public/imported_attachments/1/Screen Shot 2018-03-12 at 9.31.53 AM.png)
![Screen Shot 2018-03-12 at 9.31.53 AM.png_thumb](/public/imported_attachments/1/Screen Shot 2018-03-12 at 9.31.53 AM.png_thumb)

Pitbully · Mar 12, 2018, 10:48 PM

Baremetal
There is just too much overhead on the VM I assume. Thats too bad, but glad I didnt have to buy another box. Now I am going to install some packages on here.

Thanks everyone!

giagl011 · Mar 13, 2018, 10:49 PM

https://forum.pfsense.org/index.php?topic=145162.0

Is the above the same problem? VM overhead?

stephenw10 · Mar 14, 2018, 12:03 AM

No probably not. The overhead from running virtual should not be that large if the hypervisor is setup correctly. And on your hardware you shouldn't be getting even close to any limit at 180Mbps. Assuming you meant bps.

Steve