Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Poor performace with Openvpn

    Scheduled Pinned Locked Moved OpenVPN
    29 Posts 4 Posters 2.9k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T Offline
      trumee @stephenw10
      last edited by

      @stephenw10

      This is the speed i get with 4 streams in the reverse direction. There is some improvement in speed (14 mbps to 23 mbps).

      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.05  sec  9.12 MBytes  7.61 Mbits/sec   26             sender
      [  5]   0.00-10.00  sec  8.91 MBytes  7.47 Mbits/sec                  receiver
      [  7]   0.00-10.05  sec  6.31 MBytes  5.27 Mbits/sec   39             sender
      [  7]   0.00-10.00  sec  6.20 MBytes  5.20 Mbits/sec                  receiver
      [  9]   0.00-10.05  sec  6.31 MBytes  5.27 Mbits/sec   58             sender
      [  9]   0.00-10.00  sec  6.20 MBytes  5.20 Mbits/sec                  receiver
      [ 11]   0.00-10.05  sec  6.55 MBytes  5.47 Mbits/sec   29             sender
      [ 11]   0.00-10.00  sec  6.38 MBytes  5.35 Mbits/sec                  receiver
      [SUM]   0.00-10.05  sec  28.3 MBytes  23.6 Mbits/sec  152             sender
      [SUM]   0.00-10.00  sec  27.7 MBytes  23.2 Mbits/sec                  receiver
      

      What do you mean by window size and how do i check?

      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        The 'Cwnd' column in iperf. It only shows it at the end sending so you need to check both.

        But you can see it's much larger outside the tunnel. You might need mss-fix to prevent fragmentation.

        Steve

        1 Reply Last reply Reply Quote 0
        • T Offline
          trumee
          last edited by trumee

          I followed this article to get the value for mssfix.

          $ping -M do -s 1470  SiteB
          PING SiteB 1470(1498) bytes of data.
          ping: local error: message too long, mtu=1492
          ping: local error: message too long, mtu=1492
          ping: local error: message too long, mtu=1492
          
          $ ping -M do -s 1464  -c 1 SiteB
          PING SiteB 1464(1492) bytes of data.
          1468 bytes from SiteB: icmp_seq=1 ttl=55 time=51.5 ms
          
          

          According to the article mssfix = mtu-40, so i used mssfix 1424 in the client config of SiteB. Further following this article, i subtracted 28 from the MTU and set link-mtu to 1436.

          So finally the config looks so,

          $cat /var/etc/openvpn/client2.conf
          dev ovpnc2
          verb 1
          dev-type tun
          dev-node /dev/tun2
          writepid /var/run/openvpn_client2.pid
          #user nobody
          #group nobody
          script-security 3
          daemon
          keepalive 10 60
          ping-timer-rem
          persist-tun
          persist-key
          proto udp4
          cipher AES-256-CBC
          auth SHA256
          up /usr/local/sbin/ovpn-linkup
          down /usr/local/sbin/ovpn-linkdown
          local myip
          engine cryptodev
          tls-client
          client
          lport 0
          management /var/etc/openvpn/client2.sock unix
          remote SiteA 1194
          ifconfig 10.8.9.2 10.8.9.1
          auth-user-pass /var/etc/openvpn/client2.up
          auth-retry nointeract
          route 172.16.1.0 255.255.255.0
          route 172.16.9.0 255.255.255.0
          ca /var/etc/openvpn/client2.ca 
          cert /var/etc/openvpn/client2.cert 
          key /var/etc/openvpn/client2.key 
          tls-auth /var/etc/openvpn/client2.tls-auth 1
          ncp-ciphers AES-128-GCM:AES-256-GCM
          compress lz4-v2
          resolv-retry infinite
          topology subnet
          mssfix 1424
          link-mtu 1436
          

          With the above config i get following result in the reverse.

          $iperf3 -s
          Accepted connection from 172.16.9.21, port 35516
          [  5] local 192.168.1.111 port 5201 connected to 172.16.9.21 port 35518
          [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
          [  5]   0.00-1.00   sec   416 KBytes  3.41 Mbits/sec    0   30.9 KBytes       
          [  5]   1.00-2.00   sec  1.09 MBytes  9.16 Mbits/sec    0   78.6 KBytes       
          [  5]   2.00-3.00   sec  2.71 MBytes  22.8 Mbits/sec    0    201 KBytes       
          [  5]   3.00-4.00   sec  2.26 MBytes  19.0 Mbits/sec   20    112 KBytes       
          [  5]   4.00-5.00   sec  2.22 MBytes  18.6 Mbits/sec    0    121 KBytes       
          [  5]   5.00-6.00   sec  2.47 MBytes  20.7 Mbits/sec    0    135 KBytes       
          [  5]   6.00-7.00   sec  2.71 MBytes  22.7 Mbits/sec    0    149 KBytes       
          [  5]   7.00-8.00   sec  2.96 MBytes  24.8 Mbits/sec    0    162 KBytes       
          [  5]   8.00-9.00   sec  3.21 MBytes  26.9 Mbits/sec    0    175 KBytes       
          [  5]   9.00-10.00  sec  3.45 MBytes  29.0 Mbits/sec    0    189 KBytes       
          [  5]  10.00-10.05  sec   252 KBytes  41.2 Mbits/sec    0    189 KBytes       
          - - - - - - - - - - - - - - - - - - - - - - - - -
          [ ID] Interval           Transfer     Bitrate         Retr
          [  5]   0.00-10.05  sec  23.7 MBytes  19.8 Mbits/sec   20             sender
          -----------------------------------------------------------
          Server listening on 5201
          -----------------------------------------------------------
          

          So it hasnt improved the speed.

          1 Reply Last reply Reply Quote 0
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            Hmm the window size is tiny though.

            I would run a packet capture of the iperf traffic over the tunnel and see what's happening there, is it still fragmenting.

            You can test it by setting the window and mss size in the iperf client.

            Steve

            1 Reply Last reply Reply Quote 0
            • P Offline
              pete35
              last edited by

              Hmmm, even the performance isnt symetrical, it is way to low.

              What are the crypto settings of this tunnel ? Is AESNI used? Did you check the tunnel IPv4 settings? What version of Pfsense is that on both sites? Are this the standard Nic of the boards? With the newer OVPN versions, there are some additional buffer Settings , did you use that? Anything other on that connection?

              <a href="https://carsonlam.ca">bintang88</a>
              <a href="https://carsonlam.ca">slot88</a>

              P 1 Reply Last reply Reply Quote 0
              • P Offline
                pete35 @pete35
                last edited by

                What is the value of cryptographic settings in Advanced- Miscellaneous? It should be "aes-ni" on both sites... and inside the tunnel configuration ... "none" ...

                <a href="https://carsonlam.ca">bintang88</a>
                <a href="https://carsonlam.ca">slot88</a>

                T 1 Reply Last reply Reply Quote 0
                • stephenw10S Offline
                  stephenw10 Netgate Administrator
                  last edited by

                  Even without AES-NI it should be faster with that hardware.

                  It is possible to incorrectly use the crypto framework which can actually reduce throughput. OpenSSL will use AES-NI if the CPU has it.

                  But even with that 30Mbps is far lower than expected.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • T Offline
                    trumee @pete35
                    last edited by

                    @pete35 said in Poor performace with Openvpn:

                    What is the value of cryptographic settings in Advanced- Miscellaneous? It should be "aes-ni" on both sites... and inside the tunnel configuration ... "none" ...

                    These are the crypto settings on both sides, https://imgur.com/a/Qzar59q

                    P 1 Reply Last reply Reply Quote 0
                    • P Offline
                      pete35 @trumee
                      last edited by

                      @trumee

                      pls remove all configurations, where "cryptodev" is included and set it to aesni only.

                      <a href="https://carsonlam.ca">bintang88</a>
                      <a href="https://carsonlam.ca">slot88</a>

                      T 1 Reply Last reply Reply Quote 0
                      • T Offline
                        trumee @pete35
                        last edited by trumee

                        @pete35 said in Poor performace with Openvpn:

                        @trumee

                        pls remove all configurations, where "cryptodev" is included and set it to aesni only.

                        I have enabled AESNI in Advanced-Miscellaneous. In the tunnel configuration, should the 'Hardware Crypto' be set to 'No Hardware Crypto Acceleration'?

                        alt text

                        1 Reply Last reply Reply Quote 0
                        • RicoR Offline
                          Rico LAYER 8 Rebel Alliance
                          last edited by

                          Yes.

                          -Rico

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S Offline
                            stephenw10 Netgate Administrator
                            last edited by

                            Set it to no-hardware crypto there.

                            It will be interesting to see if that makes any measurable difference. The speeds you're seeing seem to be less than anything I would expect to be affected by that.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • T Offline
                              trumee
                              last edited by

                              I set it to 'No Hardware Crypto'. It did not make a difference.

                              [  5] local 192.168.1.111 port 5201 connected to 192.16.9.21 port 33160
                              [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
                              [  5]   0.00-1.00   sec  1.74 MBytes  14.6 Mbits/sec    2   92.3 KBytes       
                              [  5]   1.00-2.00   sec  1.87 MBytes  15.7 Mbits/sec    0    109 KBytes       
                              [  5]   2.00-3.00   sec  2.05 MBytes  17.2 Mbits/sec    0    117 KBytes       
                              [  5]   3.00-4.00   sec  2.24 MBytes  18.8 Mbits/sec    0    125 KBytes       
                              [  5]   4.00-5.00   sec  2.43 MBytes  20.3 Mbits/sec    0    138 KBytes       
                              [  5]   5.00-6.00   sec  2.30 MBytes  19.3 Mbits/sec    3    110 KBytes       
                              [  5]   6.00-7.00   sec  2.24 MBytes  18.8 Mbits/sec    0    131 KBytes       
                              [  5]   7.00-8.00   sec  1.99 MBytes  16.7 Mbits/sec   12   71.5 KBytes       
                              [  5]   8.00-9.00   sec  1.49 MBytes  12.5 Mbits/sec    0   81.9 KBytes       
                              [  5]   9.00-10.00  sec  1.49 MBytes  12.5 Mbits/sec    0   94.9 KBytes       
                              [  5]  10.00-10.05  sec   191 KBytes  29.6 Mbits/sec    0   96.2 KBytes       
                              - - - - - - - - - - - - - - - - - - - - - - - - -
                              [ ID] Interval           Transfer     Bitrate         Retr
                              [  5]   0.00-10.05  sec  20.0 MBytes  16.7 Mbits/sec   17             sender
                              -----------------------------------------------------------
                              Server listening on 5201
                              -----------------------------------------------------------
                              
                              1 Reply Last reply Reply Quote 0
                              • RicoR Offline
                                Rico LAYER 8 Rebel Alliance
                                last edited by

                                Please share all your OpenVPN settings.
                                What is your Encryption Algorithm?
                                With GCM I have seen OpenVPN traffic beyond 400 MBit/s
                                My SG-5100 can easy do ~250 MBit/s

                                -Rico

                                T 1 Reply Last reply Reply Quote 0
                                • RicoR Offline
                                  Rico LAYER 8 Rebel Alliance
                                  last edited by

                                  For testing...could you set the Encryption Algorithm to None? Just to rule this out...

                                  -Rico

                                  T 1 Reply Last reply Reply Quote 0
                                  • T Offline
                                    trumee @Rico
                                    last edited by trumee

                                    @Rico I am using 'cipher AES-256-CBC', 'auth SHA256' and ncp-ciphers 'AES-256-GCM:AES-128-GCM'. The server side VPN config is the following and the client side config is posted above.

                                    $less /var/etc/openvpn/server1.conf
                                    dev ovpns1
                                    verb 1
                                    dev-type tun
                                    dev-node /dev/tun1
                                    writepid /var/run/openvpn_server1.pid
                                    #user nobody
                                    #group nobody
                                    script-security 3
                                    daemon
                                    keepalive 10 60
                                    ping-timer-rem
                                    persist-tun
                                    persist-key
                                    proto udp4
                                    cipher AES-256-CBC
                                    auth SHA256
                                    up /usr/local/sbin/ovpn-linkup
                                    down /usr/local/sbin/ovpn-linkdown
                                    local 127.0.0.1
                                    tls-server
                                    server 10.8.9.0 255.255.255.0
                                    client-config-dir /var/etc/openvpn-csc/server1
                                    ifconfig 10.8.9.1 10.8.9.2
                                    tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'VoipVPNServer' 1"
                                    lport 1194
                                    management /var/etc/openvpn/server1.sock unix
                                    route 192.168.0.0 255.255.255.0
                                    route 192.168.1.0 255.255.255.0
                                    route 192.168.2.0 255.255.255.0
                                    route 192.168.5.0 255.255.255.0
                                    route 192.168.6.0 255.255.255.0
                                    route 192.168.10.0 255.255.255.0
                                    route 192.168.18.0 255.255.255.0
                                    route 192.168.40.0 255.255.255.0
                                    route 192.168.50.0 255.255.255.0
                                    ca /var/etc/openvpn/server1.ca 
                                    cert /var/etc/openvpn/server1.cert 
                                    key /var/etc/openvpn/server1.key 
                                    dh /etc/dh-parameters.2048
                                    crl-verify /var/etc/openvpn/server1.crl-verify 
                                    tls-auth /var/etc/openvpn/server1.tls-auth 0
                                    ncp-ciphers AES-256-GCM:AES-128-GCM
                                    compress lz4-v2
                                    persist-remote-ip
                                    float
                                    topology subnet
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • T Offline
                                      trumee @Rico
                                      last edited by

                                      @Rico said in Poor performace with Openvpn:

                                      For testing...could you set the Encryption Algorithm to None? Just to rule this out...

                                      -Rico

                                      There is no change to the result,

                                      $ iperf3 -c 192.168.1.111 -R
                                      Connecting to host 192.168.1.111, port 5201
                                      Reverse mode, remote host 192.168.1.111 is sending
                                      [  5] local 172.16.9.21 port 33962 connected to 192.168.1.111 port 5201
                                      [ ID] Interval           Transfer     Bitrate
                                      [  5]   0.00-1.00   sec  1.21 MBytes  10.2 Mbits/sec                  
                                      [  5]   1.00-2.00   sec  1.61 MBytes  13.5 Mbits/sec                  
                                      [  5]   2.00-3.00   sec   905 KBytes  7.41 Mbits/sec                  
                                      [  5]   3.00-4.00   sec  1.01 MBytes  8.48 Mbits/sec                  
                                      [  5]   4.00-5.00   sec   538 KBytes  4.41 Mbits/sec                  
                                      [  5]   5.00-6.00   sec   753 KBytes  6.17 Mbits/sec                  
                                      [  5]   6.00-7.00   sec   987 KBytes  8.09 Mbits/sec                  
                                      [  5]   7.00-8.00   sec  1.18 MBytes  9.88 Mbits/sec                  
                                      [  5]   8.00-9.00   sec  1.43 MBytes  12.0 Mbits/sec                  
                                      [  5]   9.00-10.00  sec  1.65 MBytes  13.9 Mbits/sec                  
                                      - - - - - - - - - - - - - - - - - - - - - - - - -
                                      [ ID] Interval           Transfer     Bitrate         Retr
                                      [  5]   0.00-10.05  sec  11.6 MBytes  9.67 Mbits/sec  150             sender
                                      [  5]   0.00-10.00  sec  11.2 MBytes  9.40 Mbits/sec                  receiver
                                      
                                      iperf Done.
                                      
                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Did you disable NCP? Otherwise it will still be negotiating those.

                                        But as I said this looks like some sort of mtu/fragmenting issue. Run a packet capture and see what's happening.

                                        Steve

                                        T 1 Reply Last reply Reply Quote 0
                                        • T Offline
                                          trumee @stephenw10
                                          last edited by

                                          @stephenw10 said in Poor performace with Openvpn:

                                          Did you disable NCP? Otherwise it will still be negotiating those.

                                          But as I said this looks like some sort of mtu/fragmenting issue. Run a packet capture and see what's happening.

                                          Steve

                                          I have done packet capture. What should i be looking for in wireshark?

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S Offline
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Fragmented packets is the first thing I would be looking for. Then what size the large fragments are.

                                            Otherwise check the initial TCP transactions for the window size etc. Do you see retranmissions or missing packet errors.

                                            Steve

                                            T 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.