Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Slow VPN performance Wireguard & OpenVPN on Netgate 6100

    WireGuard
    3
    9
    475
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      nvs
      last edited by nvs

      Hi,
      I have a remote Netgate 6100 running the latest pfsense 24.03. I have been running OpenVPN (split-tunnel) on this Netgate for a while, and seem to be getting speeds around 10-15MB/s using iperf3 through the VPN to my home.

      The internet speeds at both locations are (checked against an online speedtest):

      • Remote location (running the Netgate and VPN servers): 3Gbit/s download, 2.6Gbit/s upload
      • Home location: 400 Mbit/s download, 50 Mbit/s upload

      I was hoping to get (significantly) higher speeds using wireguard, so I set up a wireguard server (split-tunnel) on the Netgate (following the instructions from this video).

      I've done some iperf3 speedtests (from my home Win10 to a Ubuntu VM behind the remote Netgate) for OpenVPN and Wireguard. The performance remains rather low unfortunately and interestingly is pretty much unchanged when trying different crypto settings (see list below):

      Wireguard speeds typically appx.:

      • 1 iperf connection: 7.8 MB/s
      • 5 iperf connections: 8.2 MB/s total
      • 10 iperf connections: 9.2 MB/s total

      OpenVPN speeds typically appx.:

      • 1 iperf connection: 10.9 MB/s
      • 5 iperf connections: 11.1 MB/s total
      • 10 iperf connections: 13.0 MB/s total

      Tested some different settings, which have not significantly impacted the above speed results:

      • Tried turning off OpenVPN server (only running wireguard)
      • Turned off AES-NI CPU crypto and turned off QAT Crypto. Only IPsec-MB still on.
      • Turned ON QAT crypto again (IPsec-MB also still on, AES-NI crypto turned off)
      • Turned ON AES-NI crypto (IPsec-MB also still on, QAT crypto turned off)
      • Turned OFF IPsec-MB (AES-NI still ON, QAT crypto is OFF)
      • Disable hardware checksum offload
      • Enable hardware checksum offload (and AES-NI and IPsec-MB and QAT turned on)

      I expected significantly higher speeds for VPN (especially using Wireguard). Does anyone have any ideas what the problem could be? Thanks for any feedback you may have!

      Side notes: Before updating to the latest pfsense, I was running 23.09.1-RELEASE and that showed in the OpenVPN log every 30 sec "Note: OpenSSL hardware crypto engine functionality is not available". After updating to the latest pfsense version this does no longer show up though. Also I noticed that in the previous version it said "AES-NI CPU Crypto: Yes (inactive)". I've now changed this to be active though. OpenVPN is using TCP with port-sharing with a webserver on port 443. Sometimes during iperf3 testing I can see occasional short "spikes" of up to 25MB/s for OpenVPN.

      1 Reply Last reply Reply Quote 0
      • N
        nvs
        last edited by nvs

        I installed a standalone wireguard server on a VM behind the Netgate, to see if that would show any difference in performance. Interesting result: I am getting the exact same slow speed. So I guess that means this is not a netgate (crypto) CPU limit but something else on my network is bottlenecking the VPN traffic..

        Any ideas?

        1 Reply Last reply Reply Quote 0
        • AndyRHA
          AndyRH
          last edited by

          I did not see it in your list for WG. Adjust the MTU on the tunnel. 7100 <-> 6100 with an MTU of 1420 I am touching 800Mbps on a 10ms latency link.
          1420 may not be the best, but it is what we settled on when building the tunnel.

          o||||o
          7100-1u

          N 1 Reply Last reply Reply Quote 0
          • N
            nvs @AndyRH
            last edited by nvs

            @AndyRH thanks for the hint, definitely worth trying.

            Stupid question, but following the above linked video instructions from Tom Lauwrence there is no Wireguard interface in the list in pfsense (even though the wireguard server runs). I can manually add this interface though, see screenshot below.

            09a8a60b-58d1-489c-a0de-48054bbe9f66-image.png

            Do these fields look good? What do I need to put under IPv4 address/ in this case? Thanks!

            1 Reply Last reply Reply Quote 0
            • AndyRHA
              AndyRH
              last edited by

              We left MSS blank.
              I am not at the system, but I think the IP address is the gateway on the other side of the tunnel. I could be remembering wrong, it could be the local gateway to the other side.

              o||||o
              7100-1u

              N 1 Reply Last reply Reply Quote 0
              • N
                nvs @AndyRH
                last edited by nvs

                @AndyRH I think I am a bit stuck here then at this point because I am not doing a site to site wireguard tunnel. I just have the wireguard "server" set up on this netgate and connect with clients to it. Following the official netgate docs for setting up wireguard/the video I shared above, makes the wireguard server work but it does not create the wireguard interface.

                So if anyone could point me at what exact settings to tweak where on my pfsense to set the MTU to a lower value that would be really useful (also for OpenVPN if possible). Also wondering if i.e. I need to change the MTU on all other interfaces for WAN/LAN/BRIDGE too?

                N 1 Reply Last reply Reply Quote 0
                • N
                  nvs @nvs
                  last edited by nvs

                  Okay, closing in on the issue.. Maxing out the 400/50 Mbit wire speed with the Wireguard tunnels using a different machine at home. Getting 220 Mbit/50 Mbit on OpenVPN at home on that other machine. Really only remaining points to troubleshoot why I get only ~80Mbit for wireguard/OpenVPN (but 400/50Mbit internet..) on my Win10 machine at home would be something in that OS itself/router.. Didn't quite expect this issue to be on that end!

                  Out of interest I've been playing with changing some of the crypto settings in the first post to see if I could get the OpenVPN speed even above the 220 MBit/s I get now, but it didn't seem to make much of a difference. I guess 220 MBit/t is the max it can do..?

                  ahking19A 1 Reply Last reply Reply Quote 0
                  • ahking19A
                    ahking19 @nvs
                    last edited by

                    @nvs are both computers using wired connections? Wireless throws in a bunch of variables - AP, chipsets, drivers. Such as one computer might support 802.11ac and the other 802.11ax .

                    How far is distance between clients and server? Latency can affect performance. Can try https://speed.cloudflare.com/

                    RE: OpenVPN - OpenVPN is single threaded process. Look to see if you have 1 CPU at 100%

                    N 1 Reply Last reply Reply Quote 0
                    • N
                      nvs @ahking19
                      last edited by nvs

                      Good news. I've managed to figure out my major bottleneck. My main Win10 machine at home has a stu*** Intel I225-V 2.5 Gbit on board ethernet NIC that I was using. It has caused weird issues in the past with random disconnects and is globally known for its issues.. I have plugged in a ASUS USB 2.5 Gbit adapter and sure enough I am able to max out my wire speed now using the wireguard tunnel.

                      OpenVPN also works equally well now as on the other machine, and I figured out that enabling DCO in the OpenVPN client side can bring me close to peak speeds of 400Mbit wire speed. I haven't enabled DCO on the OpenVPN server side yet. I am still using OpenVPN via TCP, and I read for DCO thats not supported/recommended - even though interestingly enough it already gives me a performance boost already now just enabling it on the client side. Switching OpenVPN server to using UDP and enabling DCO on the server side as well might further improve things I imagine - something I might try in the next days.

                      Hope this thread will help somebody down the road troubleshooting the same.

                      @ahking19 Thanks for your reply. Both machines are on wired ethernet on the same router at home. I think I am not near maxing out the Netgate CPU, but I've attached a screenshot below to be sure (not sure how to interpret that plot exactly):
                      e7b8a2a8-aea9-4b8a-a82e-09a91455386a-image.png

                      Latency wise: I have about 9-15 ms ping through the OpenVPN tunnel from client to my netgate.
                      Using wireguard is about 9-11ms.

                      1 Reply Last reply Reply Quote 1
                      • First post
                        Last post