iperf3 testing 500/40 connection: 33mpbs
-
Hey all,
I wanted to test my SG-1100's VPN throughput. I got 33 mbps. Here is how I tested:
As you can see, my home has an asynchronous 500/40 connection. My friend's home has 200/200 fibre.
33 mbps is less than I expected. From the spec sheet, the SG-1100 should do at least double this. So I had a look at the system activity during a 40 second iperf3 test:
Worth noting here:
- the SG-1100 is using about half of it's CPU capacity. So I'm not maxing out the SG-1100. There has to be another bottleneck.
- the consecutive iperf3 measurements have a substantial variation
- I did multiple test sessions and I got consistent lower 30's
So what is going on?
I read the iperf3 manual and found that by default, the client sends and the server receives. So in my diagram, the client can send out at 200 mbps (fibre) and the server can receive at 500 mbps (cable). Then why don't I get at least 60 mbps?I also read that iperf3 uses TCP by default. Could it be that in default (TCP) mode there are just as many packets coming back from the server, which is limited at 40 mbps?
Would appreciate any insights.
Pete -
- A chain is as strong as it’s weakest link.
- There are lots of elements is your test setup.
- If you want to know which is the limiting factor I think you will need to test individual components.
-
@patch Hi there and thank you for replying!
Actually I did test all individual components:- tested the SG-1100 locally using only the two laptops: I got 66-70 and a maxed out SG-1100 CPU, which is as it should be
- tested the 500/40 download / upload speeds which are on par
- tested the 200/200 download / upload speeds which are on par
So based on that i would expect the 500/40 to deliver 66-70 mbps (the sender is the iperf3 client having 200 mbps and the receiver is the iperf3 server at 500 mbps!) unless I misunderstand iperf3 operation in default (TCP) mode, hence this topic.
If iperf3 in TCP mode is designed to test full duplex IN/OUT then 33 mbps is about right. I can't find this in the specifications.
I would have liked to test more settings such as a UDP test but I have not been able to because of COVID-19.
-
@cabledude said in iperf3 testing 500/40 connection: 33mpbs:
i would expect the 500/40 to deliver 66-70 mbps (the sender is the iperf3 client having 200 mbps and the receiver is the iperf3 server at 500 mbps!)
Did you calculate the BDP? What is the latency between the sender and the receiver?
-
Worth noting here that by default in iperf the client end serves traffic to the server (for...reasons!).
So depending how you ran this you might just be hitting the 40Mbps upload limit on your connection.
Steve
-
@stephenw10 said in iperf3 testing 500/40 connection: 33mpbs:
So depending how you ran this you might just be hitting the 40Mbps upload limit on your connection.
Hi Stephen,
Thanks for pointing that out and yes I totally see that. This is the essence of this topic.
In this test:- the client is at the fiber node serving the traffic at 200mbps upload speed
- the server is at the cable node receiving the traffic at 500mbps download speed.
Therefore my expectation was to max out the SG-1100's 66-70 VPN capacity.
Pete
-
@johnpoz
Absolutely not. Here is the ping output from the client:PING 192.168.60.104 (192.168.60.104): 56 data bytes 64 bytes from 192.168.60.104: icmp_seq=0 ttl=63 time=23.135 ms 64 bytes from 192.168.60.104: icmp_seq=1 ttl=63 time=18.199 ms 64 bytes from 192.168.60.104: icmp_seq=2 ttl=63 time=19.572 ms 64 bytes from 192.168.60.104: icmp_seq=3 ttl=63 time=18.876 ms 64 bytes from 192.168.60.104: icmp_seq=4 ttl=63 time=21.045 ms 64 bytes from 192.168.60.104: icmp_seq=5 ttl=63 time=30.158 ms 64 bytes from 192.168.60.104: icmp_seq=6 ttl=63 time=79.055 ms 64 bytes from 192.168.60.104: icmp_seq=7 ttl=63 time=121.190 ms ^C --- 192.168.60.104 ping statistics --- 8 packets transmitted, 8 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 18.199/41.404/121.190/35.673 msIs this latency to be considered higher than average?
Pete
-
What latency do you see outside the tunnel?
-
@stephenw10 good question. This my home connection (the pfSense SG-1100):
Does this give you any clue?The other node is the fibre 200/200 connection of my friend. He has a UniFi USG. I will ask him about his normal WAN latency.
Pete
-
Not really, you'd have to ping between the public IPs directly outside the VPN.
-
@stephenw10
This is the iperf3 output, default options + -t 40Connecting to host 192.168.60.104, port 5201 [ 4] local 172.16.120.2 port 51753 connected to 192.168.60.104 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 872 KBytes 7.14 Mbits/sec [ 4] 1.00-2.00 sec 1.56 MBytes 13.1 Mbits/sec [ 4] 2.00-3.00 sec 3.10 MBytes 26.0 Mbits/sec [ 4] 3.00-4.00 sec 4.11 MBytes 34.5 Mbits/sec [ 4] 4.00-5.00 sec 5.04 MBytes 42.3 Mbits/sec [ 4] 5.00-6.00 sec 3.98 MBytes 33.4 Mbits/sec [ 4] 6.00-7.00 sec 4.76 MBytes 39.9 Mbits/sec [ 4] 7.00-8.00 sec 3.58 MBytes 30.0 Mbits/sec [ 4] 8.00-9.00 sec 4.25 MBytes 35.7 Mbits/sec [ 4] 9.00-10.00 sec 4.71 MBytes 39.5 Mbits/sec [ 4] 10.00-11.00 sec 4.27 MBytes 35.8 Mbits/sec [ 4] 11.00-12.00 sec 4.93 MBytes 41.4 Mbits/sec [ 4] 12.00-13.00 sec 4.39 MBytes 36.8 Mbits/sec [ 4] 13.00-14.00 sec 3.74 MBytes 31.4 Mbits/sec [ 4] 14.00-15.00 sec 4.72 MBytes 39.6 Mbits/sec [ 4] 15.00-16.00 sec 3.94 MBytes 33.1 Mbits/sec [ 4] 16.00-17.00 sec 4.21 MBytes 35.3 Mbits/sec [ 4] 17.00-18.00 sec 4.08 MBytes 34.3 Mbits/sec [ 4] 18.00-19.00 sec 4.05 MBytes 34.0 Mbits/sec [ 4] 19.00-20.00 sec 3.68 MBytes 30.9 Mbits/sec [ 4] 20.00-21.00 sec 3.30 MBytes 27.7 Mbits/sec [ 4] 21.00-22.00 sec 4.85 MBytes 40.6 Mbits/sec [ 4] 22.00-23.00 sec 3.86 MBytes 32.4 Mbits/sec [ 4] 23.00-24.00 sec 4.45 MBytes 37.3 Mbits/sec [ 4] 24.00-25.00 sec 4.20 MBytes 35.2 Mbits/sec [ 4] 25.00-26.00 sec 3.68 MBytes 30.9 Mbits/sec [ 4] 26.00-27.00 sec 4.56 MBytes 38.3 Mbits/sec [ 4] 27.00-28.00 sec 4.17 MBytes 35.0 Mbits/sec [ 4] 28.00-29.00 sec 3.78 MBytes 31.7 Mbits/sec [ 4] 29.00-30.00 sec 4.62 MBytes 38.7 Mbits/sec [ 4] 30.00-31.00 sec 2.51 MBytes 21.0 Mbits/sec [ 4] 31.00-32.00 sec 4.07 MBytes 34.1 Mbits/sec [ 4] 32.00-33.00 sec 3.76 MBytes 31.5 Mbits/sec [ 4] 33.00-34.00 sec 4.40 MBytes 36.9 Mbits/sec [ 4] 34.00-35.00 sec 4.52 MBytes 38.0 Mbits/sec [ 4] 35.00-36.00 sec 4.11 MBytes 34.4 Mbits/sec [ 4] 36.00-37.00 sec 4.25 MBytes 35.7 Mbits/sec [ 4] 37.00-38.00 sec 4.39 MBytes 36.9 Mbits/sec [ 4] 38.00-39.00 sec 4.79 MBytes 40.2 Mbits/sec [ 4] 39.00-40.00 sec 3.62 MBytes 30.4 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-40.00 sec 160 MBytes 33.5 Mbits/sec sender [ 4] 0.00-40.00 sec 160 MBytes 33.5 Mbits/sec receiver iperf Done. -
@stephenw10 My friend and I have no direct ports open / nodes to ping so I'm not sure how I could ping between the public IP's.
-
Add a specific rule to allow only that ping from the remote public IP.
-
@cabledude said in iperf3 testing 500/40 connection: 33mpbs:
Actually I did test all individual components
What about the hardware at the other end? Running a vpn tunnel is reasonalbly processor intensive.
-
@stephenw10 said in iperf3 testing 500/40 connection: 33mpbs:
Add a specific rule to allow only that ping from the remote public IP.
Something of this sort?
Pete
Home: SG-2100 + UniFi + Synology. SG-1100 retired
Parents: SG-1100 + UniFi + Synology
Testing: SG-1100 w/ 120GB SSD via ext USB (eMMC dead). Works great -
@patch said in iperf3 testing 500/40 connection: 33mpbs:
What about the hardware at the other end? Running a vpn tunnel is reasonalbly processor intensive.
this is my end (speedtest is over WiFi but FTR VPN test was wired):
My friend's firewall is a USG 3P but the VPN wasn't S2S but remote access so I think his firewall isn't doing any encryption is it?
This is his line:
-
@cabledude said in iperf3 testing 500/40 connection: 33mpbs:
Something of this sort?
I would set the protocol to ICMP and the destination to WAN address. But since that's a single source IP and one you trust the risk is minimal.
Steve
-
Just taking his 18ms I see there as lowest one..
With only a 64k window your looking at 29.13 Mbit/sec due to the BDP..
Also lets not forget the overhead, are you trying to send 1500, those will get fragmented, again also lowering the performance. What is the box on his end doing the vpn connection? His router, what router is that - if some soho type of router, while it might be able to do a 200mbps connection, what can it push through a openvpn connection?
You prob want to adjust your txqueuelen and your tcp-queue-limit as well if your trying to see what is the max you could achieve for performance.
-
@johnpoz said in iperf3 testing 500/40 connection: 33mpbs:
Just taking his 18ms I see there as lowest one..
With only a 64k window your looking at 29.13 Mbit/sec due to the BDP..
That sounds about right! Could you share your calculation, I am very interested in that.
His router is a UniFi USG 3P. Not the fastest box but he can do about 80 mbps with his VPNs.
-
There are few calculators around.. Here is one I use
https://www.switch.ch/network/tools/tcp_throughput/
