Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Very poor NAT performance

    Scheduled Pinned Locked Moved Virtualization
    13 Posts 6 Posters 4.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      szabolcs
      last edited by

      Hi all,

      I have a pfsense box running in an ESXi 5.5 with two vmxnet3 adapters. I have IPSec + OpenVPN service on the WAN adapter activated. Accessing a service through IPSec produces around 10MB/s speed whereas accessing the same service via NAT (port forward) produces 200KB/s speed.

      There's nothing exciting about the setup, just a simple port forward. I checked the pf ruleset and indeed nothing extra is generated: (vmx0 being the WAN and vmx1 being the LAN interface)

      no nat proto carp all
      nat-anchor "natearly/" all
      nat-anchor "natrules/      " all
      nat on vmx0 inet from 127.0.0.0/8 to any port = isakmp -> 172.16.0.2 static-port
      nat on vmx0 inet from 172.16.7.0/24 to any port = isakmp -> 172.16.0.2 static-port
      nat on vmx0 inet from 10.0.0.0/24 to any port = isakmp -> 172.16.0.2 static-port
      nat on vmx0 inet from 127.0.0.0/8 to any -> 172.16.0.2 port 1024:65535
      nat on vmx0 inet from 172.16.7.0/24 to any -> 172.16.0.2 port 1024:65535
      nat on vmx0 inet from 10.0.0.0/24 to any -> 172.16.0.2 port 1024:65535
      no rdr proto carp all
      rdr-anchor "relayd/" all
      rdr-anchor "tftp-proxy/      " all
      rdr on vmx0 inet proto tcp from any to 172.16.0.2 port = https -> 172.16.7.1
      rdr-anchor "miniupnpd" all

      This performance is amazingly bad and couldn't figure out why. Tried "fetch" from the pfSense box as well resulting in 92MB/s inwards and 10MB/s outwards of the box, indicating that the network is fine.

      Anyone have any idea why the performance is soon bad?

      VM HW: 4CPUs, 1GB RAM, 2 vmxnet3

      Any help is appreciated.

      1 Reply Last reply Reply Quote 0
      • E
        EMWEE
        last edited by

        Do you have 5.5 U2? FreeBSD10 is only supported on 5.5U2 and 6.

        1 Reply Last reply Reply Quote 0
        • H
          heper
          last edited by

          @EMWEE:

          Do you have 5.5 U2? FreeBSD10 is only supported on 5.5U2 and 6.

          i've got a 2.2.X running on esxi4.1 running the legacy e1000 nics, just fine … hitting 1gbit/s wire speed without too much trouble.

          there is something else going on here.
          did you accidently install the official vmware-tools ? if yes --> reinstall and don't do it again  ;)

          1 Reply Last reply Reply Quote 0
          • S
            szabolcs
            last edited by

            I've got the latest ESXi as the update manager keeps all the hosts updated.
            As for the vmware-tools, I had it installed way back when my pfsense was 2.1 or 2.0 not sure. Back then the FreeBSD kernel did not support the vmxnet3 out of the box. Before I have upgraded the pfsense (using the autoupdated) I have uninstalled properly the vmware-tools.

            However that could be a candidate. One thing which leaves some doubt, if I change the adapters to e1000, the performance still the same. But only for the forwarded ports. When I connect to any VPN provided on the WAN interface and reach the LAN like that, all's good. Only the port forwards are extremely bad. I think I going to have to reinstall maybe, but if I do I'd like to understand why? It just doesn't make much sense to me at the moment.

            1 Reply Last reply Reply Quote 0
            • S
              szabolcs
              last edited by

              Well, the bad news is that I have reinstalled and the issue remains. I used the latest stable 2.2.2 amd64 release.

              1 Reply Last reply Reply Quote 0
              • S
                Supermule Banned
                last edited by

                I can easily get wirespeed on the 2.2.2 release using NAT.

                I use the E1000 NIC's. FreeBSD support VMXnet3 out of the box and it could easily be shitty drivers.

                1 Reply Last reply Reply Quote 0
                • S
                  szabolcs
                  last edited by

                  I mentioned in one of the posts that I tried to change the NIC to e1000 and did not help. But let me try that again.

                  1 Reply Last reply Reply Quote 0
                  • S
                    Supermule Banned
                    last edited by

                    It could be the fact you NAT a very large subnet to a smaller one, but still belonging to the same overall subnet.

                    Pretty weird rules tbh.

                    1 Reply Last reply Reply Quote 0
                    • S
                      szabolcs
                      last edited by

                      What do you mean? It's a simple port forward. Are you looking at the rdr rules (which is the problem) or the nat (which is outgoing NAT). The outgoing NAT couldn't be more standard….

                      1 Reply Last reply Reply Quote 0
                      • H
                        heper
                        last edited by

                        what does the cpu graph show on the vsphere client? (while pushing traffic)

                        how fast can you fetch a file from the pfSense console ? (to find out if its only while forwarding, or a general connection issue)

                        1 Reply Last reply Reply Quote 0
                        • S
                          szabolcs
                          last edited by

                          CPU is around 0-1 percent both on pfSense and ESXi side. Virtually not utilised at all.

                          The fetch is near 100MB/sec with once again near zero CPU utilisation. When I connect to OpenVPN or IPSec on WAN I can reach the LAN with full speed. The network drivers are absolutely fine in my view.

                          1 Reply Last reply Reply Quote 0
                          • P
                            planetinse
                            last edited by

                            Confirming the very same issue

                            1 Reply Last reply Reply Quote 0
                            • A
                              agrant
                              last edited by

                              I'm seeing the same type of behaviour. When the gateway is the CARP Vip my throughput out of WAN is ~3mbps max as soon as I switch to the real router LAN interface I have connection speeds of 50mbps (which is normal). No raise in CPU or memory usage either.

                              ESXi 6.0
                              4gb Ram
                              5 CPUs

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.