Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Poor performace with Openvpn

    Scheduled Pinned Locked Moved OpenVPN
    29 Posts 4 Posters 2.9k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P Offline
      pete35
      last edited by

      Hmmm, even the performance isnt symetrical, it is way to low.

      What are the crypto settings of this tunnel ? Is AESNI used? Did you check the tunnel IPv4 settings? What version of Pfsense is that on both sites? Are this the standard Nic of the boards? With the newer OVPN versions, there are some additional buffer Settings , did you use that? Anything other on that connection?

      <a href="https://carsonlam.ca">bintang88</a>
      <a href="https://carsonlam.ca">slot88</a>

      P 1 Reply Last reply Reply Quote 0
      • P Offline
        pete35 @pete35
        last edited by

        What is the value of cryptographic settings in Advanced- Miscellaneous? It should be "aes-ni" on both sites... and inside the tunnel configuration ... "none" ...

        <a href="https://carsonlam.ca">bintang88</a>
        <a href="https://carsonlam.ca">slot88</a>

        T 1 Reply Last reply Reply Quote 0
        • stephenw10S Offline
          stephenw10 Netgate Administrator
          last edited by

          Even without AES-NI it should be faster with that hardware.

          It is possible to incorrectly use the crypto framework which can actually reduce throughput. OpenSSL will use AES-NI if the CPU has it.

          But even with that 30Mbps is far lower than expected.

          Steve

          1 Reply Last reply Reply Quote 0
          • T Offline
            trumee @pete35
            last edited by

            @pete35 said in Poor performace with Openvpn:

            What is the value of cryptographic settings in Advanced- Miscellaneous? It should be "aes-ni" on both sites... and inside the tunnel configuration ... "none" ...

            These are the crypto settings on both sides, https://imgur.com/a/Qzar59q

            P 1 Reply Last reply Reply Quote 0
            • P Offline
              pete35 @trumee
              last edited by

              @trumee

              pls remove all configurations, where "cryptodev" is included and set it to aesni only.

              <a href="https://carsonlam.ca">bintang88</a>
              <a href="https://carsonlam.ca">slot88</a>

              T 1 Reply Last reply Reply Quote 0
              • T Offline
                trumee @pete35
                last edited by trumee

                @pete35 said in Poor performace with Openvpn:

                @trumee

                pls remove all configurations, where "cryptodev" is included and set it to aesni only.

                I have enabled AESNI in Advanced-Miscellaneous. In the tunnel configuration, should the 'Hardware Crypto' be set to 'No Hardware Crypto Acceleration'?

                alt text

                1 Reply Last reply Reply Quote 0
                • RicoR Offline
                  Rico LAYER 8 Rebel Alliance
                  last edited by

                  Yes.

                  -Rico

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator
                    last edited by

                    Set it to no-hardware crypto there.

                    It will be interesting to see if that makes any measurable difference. The speeds you're seeing seem to be less than anything I would expect to be affected by that.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • T Offline
                      trumee
                      last edited by

                      I set it to 'No Hardware Crypto'. It did not make a difference.

                      [  5] local 192.168.1.111 port 5201 connected to 192.16.9.21 port 33160
                      [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
                      [  5]   0.00-1.00   sec  1.74 MBytes  14.6 Mbits/sec    2   92.3 KBytes       
                      [  5]   1.00-2.00   sec  1.87 MBytes  15.7 Mbits/sec    0    109 KBytes       
                      [  5]   2.00-3.00   sec  2.05 MBytes  17.2 Mbits/sec    0    117 KBytes       
                      [  5]   3.00-4.00   sec  2.24 MBytes  18.8 Mbits/sec    0    125 KBytes       
                      [  5]   4.00-5.00   sec  2.43 MBytes  20.3 Mbits/sec    0    138 KBytes       
                      [  5]   5.00-6.00   sec  2.30 MBytes  19.3 Mbits/sec    3    110 KBytes       
                      [  5]   6.00-7.00   sec  2.24 MBytes  18.8 Mbits/sec    0    131 KBytes       
                      [  5]   7.00-8.00   sec  1.99 MBytes  16.7 Mbits/sec   12   71.5 KBytes       
                      [  5]   8.00-9.00   sec  1.49 MBytes  12.5 Mbits/sec    0   81.9 KBytes       
                      [  5]   9.00-10.00  sec  1.49 MBytes  12.5 Mbits/sec    0   94.9 KBytes       
                      [  5]  10.00-10.05  sec   191 KBytes  29.6 Mbits/sec    0   96.2 KBytes       
                      - - - - - - - - - - - - - - - - - - - - - - - - -
                      [ ID] Interval           Transfer     Bitrate         Retr
                      [  5]   0.00-10.05  sec  20.0 MBytes  16.7 Mbits/sec   17             sender
                      -----------------------------------------------------------
                      Server listening on 5201
                      -----------------------------------------------------------
                      
                      1 Reply Last reply Reply Quote 0
                      • RicoR Offline
                        Rico LAYER 8 Rebel Alliance
                        last edited by

                        Please share all your OpenVPN settings.
                        What is your Encryption Algorithm?
                        With GCM I have seen OpenVPN traffic beyond 400 MBit/s
                        My SG-5100 can easy do ~250 MBit/s

                        -Rico

                        T 1 Reply Last reply Reply Quote 0
                        • RicoR Offline
                          Rico LAYER 8 Rebel Alliance
                          last edited by

                          For testing...could you set the Encryption Algorithm to None? Just to rule this out...

                          -Rico

                          T 1 Reply Last reply Reply Quote 0
                          • T Offline
                            trumee @Rico
                            last edited by trumee

                            @Rico I am using 'cipher AES-256-CBC', 'auth SHA256' and ncp-ciphers 'AES-256-GCM:AES-128-GCM'. The server side VPN config is the following and the client side config is posted above.

                            $less /var/etc/openvpn/server1.conf
                            dev ovpns1
                            verb 1
                            dev-type tun
                            dev-node /dev/tun1
                            writepid /var/run/openvpn_server1.pid
                            #user nobody
                            #group nobody
                            script-security 3
                            daemon
                            keepalive 10 60
                            ping-timer-rem
                            persist-tun
                            persist-key
                            proto udp4
                            cipher AES-256-CBC
                            auth SHA256
                            up /usr/local/sbin/ovpn-linkup
                            down /usr/local/sbin/ovpn-linkdown
                            local 127.0.0.1
                            tls-server
                            server 10.8.9.0 255.255.255.0
                            client-config-dir /var/etc/openvpn-csc/server1
                            ifconfig 10.8.9.1 10.8.9.2
                            tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'VoipVPNServer' 1"
                            lport 1194
                            management /var/etc/openvpn/server1.sock unix
                            route 192.168.0.0 255.255.255.0
                            route 192.168.1.0 255.255.255.0
                            route 192.168.2.0 255.255.255.0
                            route 192.168.5.0 255.255.255.0
                            route 192.168.6.0 255.255.255.0
                            route 192.168.10.0 255.255.255.0
                            route 192.168.18.0 255.255.255.0
                            route 192.168.40.0 255.255.255.0
                            route 192.168.50.0 255.255.255.0
                            ca /var/etc/openvpn/server1.ca 
                            cert /var/etc/openvpn/server1.cert 
                            key /var/etc/openvpn/server1.key 
                            dh /etc/dh-parameters.2048
                            crl-verify /var/etc/openvpn/server1.crl-verify 
                            tls-auth /var/etc/openvpn/server1.tls-auth 0
                            ncp-ciphers AES-256-GCM:AES-128-GCM
                            compress lz4-v2
                            persist-remote-ip
                            float
                            topology subnet
                            
                            1 Reply Last reply Reply Quote 0
                            • T Offline
                              trumee @Rico
                              last edited by

                              @Rico said in Poor performace with Openvpn:

                              For testing...could you set the Encryption Algorithm to None? Just to rule this out...

                              -Rico

                              There is no change to the result,

                              $ iperf3 -c 192.168.1.111 -R
                              Connecting to host 192.168.1.111, port 5201
                              Reverse mode, remote host 192.168.1.111 is sending
                              [  5] local 172.16.9.21 port 33962 connected to 192.168.1.111 port 5201
                              [ ID] Interval           Transfer     Bitrate
                              [  5]   0.00-1.00   sec  1.21 MBytes  10.2 Mbits/sec                  
                              [  5]   1.00-2.00   sec  1.61 MBytes  13.5 Mbits/sec                  
                              [  5]   2.00-3.00   sec   905 KBytes  7.41 Mbits/sec                  
                              [  5]   3.00-4.00   sec  1.01 MBytes  8.48 Mbits/sec                  
                              [  5]   4.00-5.00   sec   538 KBytes  4.41 Mbits/sec                  
                              [  5]   5.00-6.00   sec   753 KBytes  6.17 Mbits/sec                  
                              [  5]   6.00-7.00   sec   987 KBytes  8.09 Mbits/sec                  
                              [  5]   7.00-8.00   sec  1.18 MBytes  9.88 Mbits/sec                  
                              [  5]   8.00-9.00   sec  1.43 MBytes  12.0 Mbits/sec                  
                              [  5]   9.00-10.00  sec  1.65 MBytes  13.9 Mbits/sec                  
                              - - - - - - - - - - - - - - - - - - - - - - - - -
                              [ ID] Interval           Transfer     Bitrate         Retr
                              [  5]   0.00-10.05  sec  11.6 MBytes  9.67 Mbits/sec  150             sender
                              [  5]   0.00-10.00  sec  11.2 MBytes  9.40 Mbits/sec                  receiver
                              
                              iperf Done.
                              
                              1 Reply Last reply Reply Quote 0
                              • stephenw10S Offline
                                stephenw10 Netgate Administrator
                                last edited by

                                Did you disable NCP? Otherwise it will still be negotiating those.

                                But as I said this looks like some sort of mtu/fragmenting issue. Run a packet capture and see what's happening.

                                Steve

                                T 1 Reply Last reply Reply Quote 0
                                • T Offline
                                  trumee @stephenw10
                                  last edited by

                                  @stephenw10 said in Poor performace with Openvpn:

                                  Did you disable NCP? Otherwise it will still be negotiating those.

                                  But as I said this looks like some sort of mtu/fragmenting issue. Run a packet capture and see what's happening.

                                  Steve

                                  I have done packet capture. What should i be looking for in wireshark?

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S Offline
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Fragmented packets is the first thing I would be looking for. Then what size the large fragments are.

                                    Otherwise check the initial TCP transactions for the window size etc. Do you see retranmissions or missing packet errors.

                                    Steve

                                    T 1 Reply Last reply Reply Quote 0
                                    • T Offline
                                      trumee @stephenw10
                                      last edited by

                                      @stephenw10 said in Poor performace with Openvpn:

                                      Fragmented packets is the first thing I would be looking for. Then what size the large fragments are.

                                      Otherwise check the initial TCP transactions for the window size etc. Do you see retranmissions or missing packet errors.

                                      Steve

                                      I captured VPN interface and opened the capture in wireshark. I dont see any fragmented packet in the Info column.

                                      • Should I capture the WAN interface instead of the VPN interface?
                                      • Is there any wireshark tutorial which shows how to identify whether fragmentation is occuring?
                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        I'm not aware of any specific tutorial for that, there are many though.

                                        Packet fragmentation is not difficult to spot though even just from the pfSense interface.

                                        11:54:54.373954 IP 172.21.16.35 > 172.21.16.5: ICMP echo request, id 39018, seq 0, length 1480
                                        11:54:54.373963 IP 172.21.16.35 > 172.21.16.5: ip-proto-1
                                        11:54:55.374734 IP 172.21.16.35 > 172.21.16.5: ICMP echo request, id 39018, seq 1, length 1480
                                        11:54:55.374743 IP 172.21.16.35 > 172.21.16.5: ip-proto-1
                                        11:54:56.375739 IP 172.21.16.35 > 172.21.16.5: ICMP echo request, id 39018, seq 2, length 1480
                                        11:54:56.375749 IP 172.21.16.35 > 172.21.16.5: ip-proto-1
                                        

                                        Pinging size 2000 byte packets, each full size packet is followed by a fragment to make up the full packet.

                                        In that particular case there is a switch I have that doesn't pass packet fragments. 🙄

                                        Try looking at the OpenVPN traffic on the WAN also. If that is fragmented it will kill performance.

                                        Steve

                                        T 1 Reply Last reply Reply Quote 0
                                        • T Offline
                                          trumee @stephenw10
                                          last edited by

                                          For comparison sake i setup wireguard between the two networks. Wireguard starts with an MTU of 1420, and it reaches almost the line-speed in both forward and reverse directions.

                                          # wg-quick up wg0  
                                          [#] ip link add wg0 type wireguard
                                          [#] wg setconf wg0 /dev/fd/63
                                          [#] ip -4 address add 10.0.0.2/32 dev wg0
                                          [#] ip link set mtu 1420 up dev wg0
                                          [#] ip -4 route add 10.0.0.0/24 dev wg0
                                          
                                          root@wireguard:/etc/wireguard# iperf3 -c 10.0.0.1 
                                          Connecting to host 10.0.0.1, port 5201
                                          [  4] local 10.0.0.2 port 34842 connected to 10.0.0.1 port 5201
                                          [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
                                          [  4]   0.00-1.00   sec  14.8 MBytes   124 Mbits/sec    0   1.74 MBytes       
                                          [  4]   1.00-2.00   sec  16.8 MBytes   141 Mbits/sec    0   2.58 MBytes       
                                          [  4]   2.00-3.00   sec  17.0 MBytes   142 Mbits/sec    3   2.96 MBytes       
                                          [  4]   3.00-4.00   sec  16.0 MBytes   134 Mbits/sec    1   2.07 MBytes       
                                          [  4]   4.00-5.00   sec  16.7 MBytes   140 Mbits/sec    1   1.52 MBytes       
                                          [  4]   5.00-6.00   sec  17.0 MBytes   142 Mbits/sec    0   1.62 MBytes       
                                          [  4]   6.00-7.00   sec  16.9 MBytes   142 Mbits/sec    0   1.70 MBytes       
                                          [  4]   7.00-8.00   sec  16.8 MBytes   141 Mbits/sec    0   1.75 MBytes       
                                          [  4]   8.00-9.00   sec  16.9 MBytes   142 Mbits/sec    0   1.79 MBytes       
                                          [  4]   9.00-10.00  sec  17.0 MBytes   142 Mbits/sec    2    920 KBytes       
                                          - - - - - - - - - - - - - - - - - - - - - - - - -
                                          [ ID] Interval           Transfer     Bandwidth       Retr
                                          [  4]   0.00-10.00  sec   166 MBytes   139 Mbits/sec    7             sender
                                          [  4]   0.00-10.00  sec   163 MBytes   137 Mbits/sec                  receiver
                                          
                                          iperf Done.
                                          root@wireguard:/etc/wireguard# iperf3 -c 10.0.0.1 -R
                                          Connecting to host 10.0.0.1, port 5201
                                          Reverse mode, remote host 10.0.0.1 is sending
                                          [  4] local 10.0.0.2 port 35042 connected to 10.0.0.1 port 5201
                                          [ ID] Interval           Transfer     Bandwidth
                                          [  4]   0.00-1.00   sec  13.0 MBytes   109 Mbits/sec                  
                                          [  4]   1.00-2.00   sec  16.9 MBytes   141 Mbits/sec                  
                                          [  4]   2.00-3.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   3.00-4.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   4.00-5.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   5.00-6.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   6.00-7.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   7.00-8.00   sec  16.9 MBytes   141 Mbits/sec                  
                                          [  4]   8.00-9.00   sec  16.9 MBytes   142 Mbits/sec                  
                                          [  4]   9.00-10.00  sec  16.9 MBytes   142 Mbits/sec                  
                                          - - - - - - - - - - - - - - - - - - - - - - - - -
                                          [ ID] Interval           Transfer     Bandwidth       Retr
                                          [  4]   0.00-10.00  sec   168 MBytes   141 Mbits/sec    3             sender
                                          [  4]   0.00-10.00  sec   168 MBytes   141 Mbits/sec                  receiver
                                          
                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.