Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Improve OpenVPN throughput

    Scheduled Pinned Locked Moved OpenVPN
    12 Posts 3 Posters 6.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      drkrieger
      last edited by

      Hello Folks,

      I'm trying to find a way to improve the throughput on a single threaded connections across our OpenVPN links. We're running 500Mbps sequential fiber at out data center, and 300/20 on a remote site. The highest I was able to tune my OpenVPN settings was ~85 Mbps on a single thread iperf3 test. With "-P 4" on the iperf, I can hit 300, but that's not my goal. I'd like to try and improve the single thread to get at least 150Mb or better.
      The client router is an old Dell PC with an i5-2400 and 4GB of DDR3 ram, Intel Pro 1000 PT Quad Port NIC.
      The server router is an old Lenovo server, Xeon E3-1230, 16GB Ram, Intel Pro 1000 PT Quad port NIC.
      Both routers running pfSense 2.4.3, checksum and TCP seg offloading enabled.

      Here's my OpenVPN Client config:

      [2.4.3-RELEASE][admin@XXXXXXXXXXXX.com]/var/etc/openvpn: cat client2.conf
dev ovpnc2
verb 1
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_client2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
cipher AES-256-CBC
auth SHA256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local XXX.XXX.XXX.XXX
engine cryptodev
lport 0
management /var/etc/openvpn/client2.sock unix
remote XXX.XXX.XXX.XXX
ifconfig 10.XXX.XXX.X 10.XXX.XXX.X
route 192.XXX.XXX.0 255.XXX.XXX.X
route 10.0.XXX.X 255.XXX.XXX.X
secret /var/etc/openvpn/client2.secret
compress lzo
resolv-retry infinite
fast-io
sndbuf 2097152
rcvbuf 2097152

      Here's my Server config:

      dev ovpns2
verb 1
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_server2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
cipher AES-256-CBC
auth SHA256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local XXX.XXX.XXX.XXX
engine cryptodev
ifconfig XXX.XXX.XXX.XXX XXX.XXX.XXX.XXX
lport 1195
management /var/etc/openvpn/server2.sock unix
push "route XXX.XXX.XXX.XXX XXX.XXX.XXX.XXX"
route XXX.XXX.XXX.XXX XXX.XXX.XXX.XXX
route XXX.XXX.XXX.XXX XXX.XXX.XXX.XXX 
secret /var/etc/openvpn/server2.secret
compress lzo
fast-io
sndbuf 2097152
rcvbuf 2097152

      Here's some 'tunables' that I've tweaked to improve my throughput (started at ~55Mbps, settled at 84 after tuning)
      net.inet.tcp.recvbuf_inc, increased to 131072
      net.inet.tcp.sendbuf_inc, increased to 131072 (these two settings gave the most increase, higher numbers made no change)

      net.inet.tcp.recvspace , 262144 (no change in speed)
      net.inet.tcp.sendspace, 262144 (no change in speed)
      kern.bufcachepercent (Percent of System RAM kernel can use for Cache) 50 (no change in speed)
      kern.ipc.nmbclusters (Network Mbufs available to system) 32786 (no change in speed)
      kern.ipc.maxsockbuf (Maximum socket buffer size) 16777216 (no change in speed)

      I did try 'tun-mtu 9000', but this didn't change the numbers at all.

      Does anyone have any more suggestions I can try to improve the performance?
      Thanks in advance!

      1 Reply Last reply Reply Quote 0
      • RicoR
        Rico LAYER 8 Rebel Alliance
        last edited by

        Have you tried to remove engine cryptodev ?
        Sometimes it make things slower not faster. 😬

        -Rico

        1 Reply Last reply Reply Quote 1
        • D
          drkrieger
          last edited by

          Just tried it, lost about 8-10 Mbps. I also tried removing the AES-256-GCM (only AES-128-GCM now) from the negotiable's, made no change.

          1 Reply Last reply Reply Quote 0
          • RicoR
            Rico LAYER 8 Rebel Alliance
            last edited by

            I'd play around with the compression in the next step, disable it completely or try lz4 to check for any impact.

            -Rico

            1 Reply Last reply Reply Quote 1
            • D
              drkrieger
              last edited by drkrieger

              Didn't seem to make much difference. LZ4 v2 gave ~1-2 Mbps increase over several tests, but nothing quite like what I was hoping for.
              Turning off compression didn't seem to make any difference at all, was still between 78-84 Mbps.

              G 1 Reply Last reply Reply Quote 0
              • G
                gsmornot @drkrieger
                last edited by

                @drkrieger said in Improve OpenVPN throughput:

                Didn't seem to make much difference. LZ4 v2 gave ~1-2 Mbps increase over several tests, but nothing quite like what I was hoping for.
                Turning off compression didn't seem to make any difference at all, was still between 78-84 Mbps.

                Using an SG-3100 here so low power device. I am using 128-GCM, SHA1, DH-2048. If you want to know anymore about my config I can share. I would export your profile again and test. GCM should use multiple cores. On gigabit I have topped out at 110Mbps but that is about it with this setup. IPSec on the other hand would likely provide the speeds you're looking for but I prefer to stay with OpenVPN and the lower overall throughput.

                1 Reply Last reply Reply Quote 0
                • RicoR
                  Rico LAYER 8 Rebel Alliance
                  last edited by

                  Weird stuff, my 11 old Xeon 3000 can easy make around 160 MBit/s OpenVPN AES-256-GCM.
                  Please try this test from the commandline/SSH.

                  openvpn --genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-gcm

                  ( 3200 / execution_time_seconds ) = Projected Maximum OpenVPN Performance in Mbps

                  -Rico

                  D 1 Reply Last reply Reply Quote 0
                  • RicoR
                    Rico LAYER 8 Rebel Alliance
                    last edited by Rico

                    @gsmornot
                    I don't think GCM use multiple cores...
                    With my SG-3100 the max in real world tests is around 83 MBit/s OpenVPN AES-256-GCM.
                    But you can share your config if you want. :-)

                    -Rico

                    G 1 Reply Last reply Reply Quote 0
                    • D
                      drkrieger @Rico
                      last edited by

                      @rico Thanks for that tip, good to know!
                      This was my output:

                      Thu Sep 27 12:33:53 2018 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.288u 0.007s 0:10.30 99.8%    814+178k 0+0io 0pf+0w

                      So with ~10 seconds, I'm guessing my peak would be theoretical 320 Mbps which is the maximum of the pipe on the remote site. I'm wondering how I can tune to get closer to that number.
                      I noticed in the test that MTU's are set to 20k. I'm guessing that I can't do that with OpenVPN's tunnel settings?

                      RicoR 1 Reply Last reply Reply Quote 0
                      • RicoR
                        Rico LAYER 8 Rebel Alliance @drkrieger
                        last edited by

                        @drkrieger said in Improve OpenVPN throughput:

                        I noticed in the test that MTU's are set to 20k. I'm guessing that I can't do that with OpenVPN's tunnel settings?

                        https://forum.netgate.com/post/619858

                        -Rico

                        1 Reply Last reply Reply Quote 0
                        • D
                          drkrieger
                          last edited by

                          I'm finding something interesting about this whole situation: iperf single thread only hits ~85 Mbps, yet a SMB file transfer (Windows Share) is able to hit 38.5 MB/s. Flaw with iperf, or just protocol differences?

                          1 Reply Last reply Reply Quote 0
                          • G
                            gsmornot @Rico
                            last edited by

                            @rico said in Improve OpenVPN throughput:

                            @gsmornot
                            I don't think GCM use multiple cores...
                            With my SG-3100 the max in real world tests is around 83 MBit/s OpenVPN AES-256-GCM.
                            But you can share your config if you want. :-)

                            -Rico

                            Nothing special in the config, just offering to reply with any detail asked for in my current settings.

                            Looking, I thought it was multicore but I think what I want to say is capable of parallel processing. I am a bit more basic in my understanding. I tested the performance versus CBC and found it to be much better, maybe double, so I stuck with it. IPSec was much faster but I kept reading people say they would always pick OpenVPN first so I stuck with it. So far no issues. My main use is access through my firewall while mobile so I can make use of public wifi at times and even bigger for me block ad's with pfBlocker.

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.