[Solved] 2.3.2 on ESXi 5.5.0U3 - network performance issue



  • Hey,

    sorry for my English, it's not my native language.

    So far I managed to setup one vm as part of a CARP setup. Most of the stuff is working now(NAT,VIPs,CP+freeradius…). But it seems like I can't get a decent performance out of the box.

    My hardware:

    • vmware host - ESXi 5.5.0U3
    • hp nc364t
    • bridged all four ports through to pfsense as vmxnet3
    • vm with 2CPUs x 1Core, 2G Ram

    Pfsense-settings:

    • 2.3.2 Pfsense
    • CPU Type: Intel(R) Xeon(TM) CPU 2.80GHz/2 CPUs: 2 package(s) x 1 core(s)
    • simple nat on Lan-side, CaptivePortal turned off
    • Wan 10 FW-Rules, Lan 2 Rules
    • 4 Interfaces; 1xWan 1xLan 1xProjectlan(enabled but unused) 1xSync
    • 3 VIPs
    • Packages: AutoConfigBackup/Backup/darkstat/freeradius2/iftop/iperf/OpenVMTools

    I ran iperf against pfsense:
    Wan-side -> 288-504 Mbits/s
    Lan-side  -> 216-277 Mbits/s
    (disabled CP,darkstate)
    While I ran tested the WebUI was not accessable!

    I also tried a debian8 vm on the same card/ports against iperf and I got over 900 Mbits/s, using the e1000 driver.
    It looks like I am doing something wrong  :o

    What is the recommanded vmware driver for better pfsense performance, e1000e or vmxnet3? Is there anything else I could try, maybe more resources?

    Thanks for your time
    otzi

    Edit: forget to mention vmware-tools are installed


  • Rebel Alliance Global Moderator

    did you iperf thru pfsense?  Pfsense is a router/firewall not really optimized for answering traffic to itself.

    What do you mean you bridged thru all 4 interfaces?  In esxi you would have a vmkern, and then other vswitch(es) for your other hardware interfaces.  Can you post your esxi network configuration.  Example below is mine.




  • @johnpoz:

    did you iperf thru pfsense?  Pfsense is a router/firewall not really optimized for answering traffic to itself.

    I used iperf against the pfsense itself.
    Early this week, after I finished setting up this pfsense box, I noticed  that speed maxed out at 300, sometimes at 450MBits/s testing it against a decent nas-box(physical no vm) doing simple ftp. Normally I get 600-700 Mbits/s from this nas, sometimes less depending the network; … While I was moving files from wan2lan I couldn't even load the WebUI. So I requested another cpu and 2GB Ram in total. Performance didn't change after this, so I changed adapter type from e1000 to vmxnet3 on esxi. But it's still isn't moving any faster?! I figured that I should try another vm on the same vswitch, thought it would be better for comparison. As I said debian8 is getting 900+ through, with 1core 1GB ram.

    What do you mean you bridged thru all 4 interfaces?  In esxi you would have a vmkern, and then other vswitch(es) for your other hardware interfaces.  Can you post your esxi network configuration.  Example below is mine.

    You are right. I didn't bridge the adapter through. I meant to say that nothing else is running on those physical ports; only pfsense.

    However I attached the network vswitch-overview. I am not the main administrator of the esxi - I am running another smaller machine(centos-vm-host) which will be the backup-CARP-member. Anyway I think the vswitch-setup isn't the issue, but correct me if I'm wrong.

    I looked, but couldn't find any reports indicating problems regarding pfsense 2.3.2 running on esxi 5.5.0U3!



  • Rebel Alliance Global Moderator

    What are you hiding here, is that your pfsense setup?  Why would its lan/wan be the same vswitch?

    What are those other networks on each vswitch.  I don't see more than 1 vm on those switches - so only pfsense?

    So what is your Iperf THRU pfsense.. ie that is routing/firewalling..  Testing to pfsense IP is not a valid test of the performance of pfsense as a router/firewall its a test of how fast you could move a file to pfsense directly, etc.



  • I changed the vm settings and it seems to be all good now.  :)
    After reading the hardware requirements https://www.pfsense.org/hardware/:

    501+ Mbps -> Multiple cores at > 2.0GHz are required. Server class hardware with PCI-e network adapters.

    I ended up with more cores…

    @johnpoz:

    What are you hiding here, is that your pfsense setup?  Why would its lan/wan be the same vswitch?

    Well, yes… I am hiding the public dns/ip.
    Arguably lan/wan on one vswitch doesn't make much sense and I will change that....

    What are those other networks on each vswitch.  I don't see more than 1 vm on those switches - so only pfsense?

    So what is your Iperf THRU pfsense.. ie that is routing/firewalling..  Testing to pfsense IP is not a valid test of the performance of pfsense as a router/firewall its a test of how fast you could move a file to pfsense directly, etc.

    I did a lot of file transfers and watched the traffic graph max out around 950 or something… I will do iperf through the pfsense as you recommend and report back tomorrow.

    However I can mark the thread [SOLVED].

    Thanks