Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    iperf3 testing 500/40 connection: 33mpbs

    Scheduled Pinned Locked Moved General pfSense Questions
    39 Posts 5 Posters 5.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      Cabledude @johnpoz
      last edited by Cabledude

      @johnpoz
      Absolutely not. Here is the ping output from the client:

      PING 192.168.60.104 (192.168.60.104): 56 data bytes
      64 bytes from 192.168.60.104: icmp_seq=0 ttl=63 time=23.135 ms
      64 bytes from 192.168.60.104: icmp_seq=1 ttl=63 time=18.199 ms
      64 bytes from 192.168.60.104: icmp_seq=2 ttl=63 time=19.572 ms
      64 bytes from 192.168.60.104: icmp_seq=3 ttl=63 time=18.876 ms
      64 bytes from 192.168.60.104: icmp_seq=4 ttl=63 time=21.045 ms
      64 bytes from 192.168.60.104: icmp_seq=5 ttl=63 time=30.158 ms
      64 bytes from 192.168.60.104: icmp_seq=6 ttl=63 time=79.055 ms
      64 bytes from 192.168.60.104: icmp_seq=7 ttl=63 time=121.190 ms
      ^C
      --- 192.168.60.104 ping statistics ---
      8 packets transmitted, 8 packets received, 0.0% packet loss
      round-trip min/avg/max/stddev = 18.199/41.404/121.190/35.673 ms
      

      Is this latency to be considered higher than average?

      Pete

      Pete
      Home: SG-2100 + UniFi + Synology. SG-1100 retired
      Parents: SG-1100 + UniFi + Synology
      Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        What latency do you see outside the tunnel?

        1 Reply Last reply Reply Quote 0
        • C
          Cabledude
          last edited by Cabledude

          @stephenw10 good question. This my home connection (the pfSense SG-1100):
          Schermafbeelding 2021-12-18 om 23.21.21.jpg
          Does this give you any clue?

          The other node is the fibre 200/200 connection of my friend. He has a UniFi USG. I will ask him about his normal WAN latency.

          Pete

          Pete
          Home: SG-2100 + UniFi + Synology. SG-1100 retired
          Parents: SG-1100 + UniFi + Synology
          Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Not really, you'd have to ping between the public IPs directly outside the VPN.

            C 3 Replies Last reply Reply Quote 0
            • C
              Cabledude @stephenw10
              last edited by

              @stephenw10
              This is the iperf3 output, default options + -t 40

              Connecting to host 192.168.60.104, port 5201
              [  4] local 172.16.120.2 port 51753 connected to 192.168.60.104 port 5201
              [ ID] Interval           Transfer     Bandwidth
              [  4]   0.00-1.00   sec   872 KBytes  7.14 Mbits/sec                  
              [  4]   1.00-2.00   sec  1.56 MBytes  13.1 Mbits/sec                  
              [  4]   2.00-3.00   sec  3.10 MBytes  26.0 Mbits/sec                  
              [  4]   3.00-4.00   sec  4.11 MBytes  34.5 Mbits/sec                  
              [  4]   4.00-5.00   sec  5.04 MBytes  42.3 Mbits/sec                  
              [  4]   5.00-6.00   sec  3.98 MBytes  33.4 Mbits/sec                  
              [  4]   6.00-7.00   sec  4.76 MBytes  39.9 Mbits/sec                  
              [  4]   7.00-8.00   sec  3.58 MBytes  30.0 Mbits/sec                  
              [  4]   8.00-9.00   sec  4.25 MBytes  35.7 Mbits/sec                  
              [  4]   9.00-10.00  sec  4.71 MBytes  39.5 Mbits/sec                  
              [  4]  10.00-11.00  sec  4.27 MBytes  35.8 Mbits/sec                  
              [  4]  11.00-12.00  sec  4.93 MBytes  41.4 Mbits/sec                  
              [  4]  12.00-13.00  sec  4.39 MBytes  36.8 Mbits/sec                  
              [  4]  13.00-14.00  sec  3.74 MBytes  31.4 Mbits/sec                  
              [  4]  14.00-15.00  sec  4.72 MBytes  39.6 Mbits/sec                  
              [  4]  15.00-16.00  sec  3.94 MBytes  33.1 Mbits/sec                  
              [  4]  16.00-17.00  sec  4.21 MBytes  35.3 Mbits/sec                  
              [  4]  17.00-18.00  sec  4.08 MBytes  34.3 Mbits/sec                  
              [  4]  18.00-19.00  sec  4.05 MBytes  34.0 Mbits/sec                  
              [  4]  19.00-20.00  sec  3.68 MBytes  30.9 Mbits/sec                  
              [  4]  20.00-21.00  sec  3.30 MBytes  27.7 Mbits/sec                  
              [  4]  21.00-22.00  sec  4.85 MBytes  40.6 Mbits/sec                  
              [  4]  22.00-23.00  sec  3.86 MBytes  32.4 Mbits/sec                  
              [  4]  23.00-24.00  sec  4.45 MBytes  37.3 Mbits/sec                  
              [  4]  24.00-25.00  sec  4.20 MBytes  35.2 Mbits/sec                  
              [  4]  25.00-26.00  sec  3.68 MBytes  30.9 Mbits/sec                  
              [  4]  26.00-27.00  sec  4.56 MBytes  38.3 Mbits/sec                  
              [  4]  27.00-28.00  sec  4.17 MBytes  35.0 Mbits/sec                  
              [  4]  28.00-29.00  sec  3.78 MBytes  31.7 Mbits/sec                  
              [  4]  29.00-30.00  sec  4.62 MBytes  38.7 Mbits/sec                  
              [  4]  30.00-31.00  sec  2.51 MBytes  21.0 Mbits/sec                  
              [  4]  31.00-32.00  sec  4.07 MBytes  34.1 Mbits/sec                  
              [  4]  32.00-33.00  sec  3.76 MBytes  31.5 Mbits/sec                  
              [  4]  33.00-34.00  sec  4.40 MBytes  36.9 Mbits/sec                  
              [  4]  34.00-35.00  sec  4.52 MBytes  38.0 Mbits/sec                  
              [  4]  35.00-36.00  sec  4.11 MBytes  34.4 Mbits/sec                  
              [  4]  36.00-37.00  sec  4.25 MBytes  35.7 Mbits/sec                  
              [  4]  37.00-38.00  sec  4.39 MBytes  36.9 Mbits/sec                  
              [  4]  38.00-39.00  sec  4.79 MBytes  40.2 Mbits/sec                  
              [  4]  39.00-40.00  sec  3.62 MBytes  30.4 Mbits/sec                  
              - - - - - - - - - - - - - - - - - - - - - - - - -
              [ ID] Interval           Transfer     Bandwidth
              [  4]   0.00-40.00  sec   160 MBytes  33.5 Mbits/sec                  sender
              [  4]   0.00-40.00  sec   160 MBytes  33.5 Mbits/sec                  receiver
              
              iperf Done.
              

              Pete
              Home: SG-2100 + UniFi + Synology. SG-1100 retired
              Parents: SG-1100 + UniFi + Synology
              Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

              1 Reply Last reply Reply Quote 0
              • C
                Cabledude @stephenw10
                last edited by

                @stephenw10 My friend and I have no direct ports open / nodes to ping so I'm not sure how I could ping between the public IP's.

                Pete
                Home: SG-2100 + UniFi + Synology. SG-1100 retired
                Parents: SG-1100 + UniFi + Synology
                Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Add a specific rule to allow only that ping from the remote public IP.

                  C 1 Reply Last reply Reply Quote 0
                  • P
                    Patch
                    last edited by

                    @cabledude said in iperf3 testing 500/40 connection: 33mpbs:

                    Actually I did test all individual components

                    What about the hardware at the other end? Running a vpn tunnel is reasonalbly processor intensive.

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      Cabledude @stephenw10
                      last edited by

                      @stephenw10 said in iperf3 testing 500/40 connection: 33mpbs:

                      Add a specific rule to allow only that ping from the remote public IP.

                      Something of this sort?
                      Schermafbeelding 2021-12-19 om 00.51.00.jpg

                      Pete
                      Home: SG-2100 + UniFi + Synology. SG-1100 retired
                      Parents: SG-1100 + UniFi + Synology
                      Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                      stephenw10S B 2 Replies Last reply Reply Quote 0
                      • C
                        Cabledude @Patch
                        last edited by

                        @patch said in iperf3 testing 500/40 connection: 33mpbs:

                        What about the hardware at the other end? Running a vpn tunnel is reasonalbly processor intensive.

                        this is my end (speedtest is over WiFi but FTR VPN test was wired):
                        Schermafbeelding 2021-12-18 om 23.47.59.jpg

                        My friend's firewall is a USG 3P but the VPN wasn't S2S but remote access so I think his firewall isn't doing any encryption is it?
                        This is his line:
                        Schermafbeelding 2021-12-18 om 23.48.59.jpg

                        Pete
                        Home: SG-2100 + UniFi + Synology. SG-1100 retired
                        Parents: SG-1100 + UniFi + Synology
                        Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator @Cabledude
                          last edited by

                          @cabledude said in iperf3 testing 500/40 connection: 33mpbs:

                          Something of this sort?

                          I would set the protocol to ICMP and the destination to WAN address. But since that's a single source IP and one you trust the risk is minimal.

                          Steve

                          johnpozJ 1 Reply Last reply Reply Quote 0
                          • johnpozJ
                            johnpoz LAYER 8 Global Moderator @stephenw10
                            last edited by

                            Just taking his 18ms I see there as lowest one..

                            With only a 64k window your looking at 29.13 Mbit/sec due to the BDP..

                            Also lets not forget the overhead, are you trying to send 1500, those will get fragmented, again also lowering the performance. What is the box on his end doing the vpn connection? His router, what router is that - if some soho type of router, while it might be able to do a 200mbps connection, what can it push through a openvpn connection?

                            You prob want to adjust your txqueuelen and your tcp-queue-limit as well if your trying to see what is the max you could achieve for performance.

                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                            If you get confused: Listen to the Music Play
                            Please don't Chat/PM me for help, unless mod related
                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                            C 1 Reply Last reply Reply Quote 0
                            • C
                              Cabledude @johnpoz
                              last edited by

                              @johnpoz said in iperf3 testing 500/40 connection: 33mpbs:

                              Just taking his 18ms I see there as lowest one..

                              With only a 64k window your looking at 29.13 Mbit/sec due to the BDP..

                              That sounds about right! Could you share your calculation, I am very interested in that.

                              His router is a UniFi USG 3P. Not the fastest box but he can do about 80 mbps with his VPNs.

                              Pete
                              Home: SG-2100 + UniFi + Synology. SG-1100 retired
                              Parents: SG-1100 + UniFi + Synology
                              Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                              johnpozJ 1 Reply Last reply Reply Quote 0
                              • johnpozJ
                                johnpoz LAYER 8 Global Moderator @Cabledude
                                last edited by

                                @cabledude

                                https://www.cisco.com/c/en/us/support/docs/ip/transmission-control-protocol-tcp/200943-Why-Your-Application-only-Uses-10Mbps-Ev.html#anc3

                                There are few calculators around.. Here is one I use
                                https://www.switch.ch/network/tools/tcp_throughput/

                                An intelligent man is sometimes forced to be drunk to spend time with his fools
                                If you get confused: Listen to the Music Play
                                Please don't Chat/PM me for help, unless mod related
                                SG-4860 24.11 | Lab VMs 2.8, 24.11

                                C 1 Reply Last reply Reply Quote 0
                                • C
                                  Cabledude @johnpoz
                                  last edited by Cabledude

                                  @johnpoz Hi John, thank you, good read.
                                  I was just thinking what protocol am I using for the pfSense OpenVPN setup and it is UDP. Is this relevant for the BDP? If i read the article correctly it assumes TCP.
                                  9253518f-5e9c-4789-90df-8617f991cbe8-image.jpeg

                                  Thank you for helping out! I am learning every day :)

                                  Pete

                                  Pete
                                  Home: SG-2100 + UniFi + Synology. SG-1100 retired
                                  Parents: SG-1100 + UniFi + Synology
                                  Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                                  johnpozJ 1 Reply Last reply Reply Quote 0
                                  • johnpozJ
                                    johnpoz LAYER 8 Global Moderator @Cabledude
                                    last edited by

                                    @cabledude what your tunnel is doesn't really matter. Your testing with TCP from your iperf outputs.

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                                    C 1 Reply Last reply Reply Quote 0
                                    • C
                                      Cabledude @johnpoz
                                      last edited by

                                      @johnpoz Ah okay I see, thank you.

                                      So apparently building a VPN tunnel for high speed backups isn't as easy as simply connecting two gigabit nodes.

                                      In order to troubleshoot further, would it be of any use to repeat the iperf3 test using UDP? Or using any other flags?

                                      My friend is not available for further testing today but I might get a chance next week. He is willing to ping to my network tonight (UTC +1) though.

                                      Pete

                                      Pete
                                      Home: SG-2100 + UniFi + Synology. SG-1100 retired
                                      Parents: SG-1100 + UniFi + Synology
                                      Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                                      johnpozJ 1 Reply Last reply Reply Quote 0
                                      • johnpozJ
                                        johnpoz LAYER 8 Global Moderator @Cabledude
                                        last edited by

                                        @cabledude said in iperf3 testing 500/40 connection: 33mpbs:

                                        would it be of any use to repeat the iperf3 test using UDP?

                                        Depends on what your wanting accomplish - if just to see if a difference sure, but does your backup actually leverage udp ;)

                                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                                        If you get confused: Listen to the Music Play
                                        Please don't Chat/PM me for help, unless mod related
                                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Testing using UDP eliminates the TCP windowing/ack packets issue so can be useful as long as you interpret the result correctly.

                                          You should definitely use UDP for the VPN tunnel itself. You should only ever use TCP there if you are forced to, like is UDP is blocked. A TCP connection inside a TCP tunnel can produce horrible results.

                                          Steve

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            Cabledude @stephenw10
                                            last edited by

                                            @stephenw10 said in iperf3 testing 500/40 connection: 33mpbs:

                                            Not really, you'd have to ping between the public IPs directly outside the VPN.

                                            18 packets transmitted, 18 packets received, 0.0% packet loss
                                            round-trip min/avg/max/stddev = 15.493/18.116/24.227/2.289 ms
                                            

                                            This is a ping outside the tunnel from my friend's UniFi USG 200/200 fibre to my SG-1100 500/40 cable.

                                            Pete
                                            Home: SG-2100 + UniFi + Synology. SG-1100 retired
                                            Parents: SG-1100 + UniFi + Synology
                                            Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great

                                            johnpozJ 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.