WAN optimization/acceleration
-
Ok, the first thing I would do here then is run some iperf tests across the tunnel to determine what is possible with the current config.
-
@netblues this is a good idea. We have a 3rd data center that is 55ms away and are experiencing similar issues. I tried something like this with 7zip but it took like 5 hours just to compress and break the file into multiple volumes. I have never heard of .xz, so thank you for exposing me to that and it seems like it could be way better.
@stephenw10 I‘m familiar with iperf but not sure what action should be taken based on the results. Iperf was run on windows servers behind each of the pf clusters.
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c 10.30.20.20 Connecting to host 10.30.20.20, port 5201 [ 4] local 10.10.10.42 port 65181 connected to 10.30.20.20 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.01 sec 896 KBytes 7.27 Mbits/sec [ 4] 1.01-2.01 sec 2.12 MBytes 17.8 Mbits/sec [ 4] 2.01-3.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 3.01-4.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 4.01-5.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 5.01-6.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 6.01-7.01 sec 2.12 MBytes 17.8 Mbits/sec [ 4] 7.01-8.01 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 8.01-9.01 sec 2.12 MBytes 17.8 Mbits/sec [ 4] 9.01-10.01 sec 2.00 MBytes 16.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.01 sec 19.2 MBytes 16.1 Mbits/sec sender [ 4] 0.00-10.01 sec 19.2 MBytes 16.1 Mbits/sec receiver iperf Done.Change window size for optimal TCP window size:
200,000,000 bps * .09 seconds = 18,000,000 / 8 = 2,250,000C:\Temp\iperf-3.1.3-win64>iperf3.exe -c 10.30.20.20 -w 2250000 Connecting to host 10.30.20.20, port 5201 [ 4] local 10.10.10.42 port 53257 connected to 10.30.20.20 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 3.25 MBytes 27.2 Mbits/sec [ 4] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec [ 4] 2.00-3.00 sec 2.12 MBytes 17.8 Mbits/sec [ 4] 3.00-4.00 sec 3.00 MBytes 25.2 Mbits/sec [ 4] 4.00-5.00 sec 3.62 MBytes 30.4 Mbits/sec [ 4] 5.00-6.00 sec 4.75 MBytes 39.9 Mbits/sec [ 4] 6.00-7.00 sec 6.00 MBytes 50.4 Mbits/sec [ 4] 7.00-8.00 sec 6.25 MBytes 52.3 Mbits/sec [ 4] 8.00-9.00 sec 6.50 MBytes 54.5 Mbits/sec [ 4] 9.00-10.00 sec 7.88 MBytes 66.1 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 43.5 MBytes 36.5 Mbits/sec sender [ 4] 0.00-10.00 sec 42.0 MBytes 35.2 Mbits/sec receiver iperf Done.10 Parallel streams:
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c 10.30.20.20 -P 10 Connecting to host 10.30.20.20, port 5201 [ 4] local 10.10.10.42 port 57020 connected to 10.30.20.20 port 5201 [ 6] local 10.10.10.42 port 57033 connected to 10.30.20.20 port 5201 [ 8] local 10.10.10.42 port 57043 connected to 10.30.20.20 port 5201 [ 10] local 10.10.10.42 port 57056 connected to 10.30.20.20 port 5201 [ 12] local 10.10.10.42 port 57060 connected to 10.30.20.20 port 5201 [ 14] local 10.10.10.42 port 57070 connected to 10.30.20.20 port 5201 [ 16] local 10.10.10.42 port 57074 connected to 10.30.20.20 port 5201 [ 18] local 10.10.10.42 port 57078 connected to 10.30.20.20 port 5201 [ 20] local 10.10.10.42 port 57095 connected to 10.30.20.20 port 5201 [ 22] local 10.10.10.42 port 57102 connected to 10.30.20.20 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 1.00 MBytes 8.36 Mbits/sec [ 6] 0.00-1.00 sec 1.00 MBytes 8.36 Mbits/sec [ 8] 0.00-1.00 sec 640 KBytes 5.23 Mbits/sec [ 10] 0.00-1.00 sec 640 KBytes 5.23 Mbits/sec [ 12] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 14] 0.00-1.00 sec 256 KBytes 2.09 Mbits/sec [ 16] 0.00-1.00 sec 512 KBytes 4.18 Mbits/sec [ 18] 0.00-1.00 sec 256 KBytes 2.09 Mbits/sec [ 20] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 22] 0.00-1.00 sec 512 KBytes 4.18 Mbits/sec [SUM] 0.00-1.00 sec 5.50 MBytes 46.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 2.00 MBytes 16.8 Mbits/sec [ 6] 1.00-2.00 sec 2.12 MBytes 17.8 Mbits/sec [ 8] 1.00-2.00 sec 1.12 MBytes 9.44 Mbits/sec [ 10] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec [ 12] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 14] 1.00-2.00 sec 512 KBytes 4.19 Mbits/sec [ 16] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 18] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec [ 20] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec [ 22] 1.00-2.00 sec 512 KBytes 4.19 Mbits/sec [SUM] 1.00-2.00 sec 7.75 MBytes 65.0 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 2.00 MBytes 16.8 Mbits/sec [ 6] 2.00-3.00 sec 2.00 MBytes 16.8 Mbits/sec [ 8] 2.00-3.00 sec 1.50 MBytes 12.6 Mbits/sec [ 10] 2.00-3.00 sec 1.62 MBytes 13.6 Mbits/sec [ 12] 2.00-3.00 sec 256 KBytes 2.10 Mbits/sec [ 14] 2.00-3.00 sec 512 KBytes 4.19 Mbits/sec [ 16] 2.00-3.00 sec 256 KBytes 2.10 Mbits/sec [ 18] 2.00-3.00 sec 256 KBytes 2.10 Mbits/sec [ 20] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec [ 22] 2.00-3.00 sec 640 KBytes 5.24 Mbits/sec [SUM] 2.00-3.00 sec 9.12 MBytes 76.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 2.00 MBytes 16.8 Mbits/sec [ 6] 3.00-4.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 3.00-4.00 sec 1.75 MBytes 14.7 Mbits/sec [ 10] 3.00-4.00 sec 1.62 MBytes 13.6 Mbits/sec [ 12] 3.00-4.00 sec 256 KBytes 2.10 Mbits/sec [ 14] 3.00-4.00 sec 640 KBytes 5.24 Mbits/sec [ 16] 3.00-4.00 sec 256 KBytes 2.10 Mbits/sec [ 18] 3.00-4.00 sec 384 KBytes 3.15 Mbits/sec [ 20] 3.00-4.00 sec 384 KBytes 3.15 Mbits/sec [ 22] 3.00-4.00 sec 896 KBytes 7.34 Mbits/sec [SUM] 3.00-4.00 sec 10.0 MBytes 83.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 1.62 MBytes 13.6 Mbits/sec [ 6] 4.00-5.00 sec 1.62 MBytes 13.6 Mbits/sec [ 8] 4.00-5.00 sec 1.12 MBytes 9.45 Mbits/sec [ 10] 4.00-5.00 sec 1.25 MBytes 10.5 Mbits/sec [ 12] 4.00-5.00 sec 256 KBytes 2.10 Mbits/sec [ 14] 4.00-5.00 sec 640 KBytes 5.25 Mbits/sec [ 16] 4.00-5.00 sec 384 KBytes 3.15 Mbits/sec [ 18] 4.00-5.00 sec 384 KBytes 3.15 Mbits/sec [ 20] 4.00-5.00 sec 384 KBytes 3.15 Mbits/sec [ 22] 4.00-5.00 sec 768 KBytes 6.30 Mbits/sec [SUM] 4.00-5.00 sec 8.38 MBytes 70.3 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 1.88 MBytes 15.7 Mbits/sec [ 6] 5.00-6.00 sec 2.12 MBytes 17.8 Mbits/sec [ 8] 5.00-6.00 sec 1.38 MBytes 11.5 Mbits/sec [ 10] 5.00-6.00 sec 1.38 MBytes 11.5 Mbits/sec [ 12] 5.00-6.00 sec 640 KBytes 5.24 Mbits/sec [ 14] 5.00-6.00 sec 896 KBytes 7.34 Mbits/sec [ 16] 5.00-6.00 sec 512 KBytes 4.20 Mbits/sec [ 18] 5.00-6.00 sec 512 KBytes 4.20 Mbits/sec [ 20] 5.00-6.00 sec 640 KBytes 5.24 Mbits/sec [ 22] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec [SUM] 5.00-6.00 sec 10.9 MBytes 91.3 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.00 sec 2.00 MBytes 16.8 Mbits/sec [ 6] 6.00-7.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 6.00-7.00 sec 1.62 MBytes 13.6 Mbits/sec [ 10] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec [ 12] 6.00-7.00 sec 640 KBytes 5.25 Mbits/sec [ 14] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec [ 16] 6.00-7.00 sec 768 KBytes 6.29 Mbits/sec [ 18] 6.00-7.00 sec 768 KBytes 6.29 Mbits/sec [ 20] 6.00-7.00 sec 768 KBytes 6.29 Mbits/sec [ 22] 6.00-7.00 sec 1.25 MBytes 10.5 Mbits/sec [SUM] 6.00-7.00 sec 12.4 MBytes 104 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 7.00-8.00 sec 2.00 MBytes 16.7 Mbits/sec [ 6] 7.00-8.00 sec 2.00 MBytes 16.7 Mbits/sec [ 8] 7.00-8.00 sec 1.62 MBytes 13.6 Mbits/sec [ 10] 7.00-8.00 sec 1.50 MBytes 12.6 Mbits/sec [ 12] 7.00-8.00 sec 768 KBytes 6.28 Mbits/sec [ 14] 7.00-8.00 sec 1.12 MBytes 9.42 Mbits/sec [ 16] 7.00-8.00 sec 896 KBytes 7.32 Mbits/sec [ 18] 7.00-8.00 sec 1.00 MBytes 8.37 Mbits/sec [ 20] 7.00-8.00 sec 896 KBytes 7.32 Mbits/sec [ 22] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec [SUM] 7.00-8.00 sec 13.0 MBytes 109 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 8.00-9.00 sec 1.75 MBytes 14.7 Mbits/sec [ 6] 8.00-9.00 sec 1.62 MBytes 13.7 Mbits/sec [ 8] 8.00-9.00 sec 1.50 MBytes 12.6 Mbits/sec [ 10] 8.00-9.00 sec 1.50 MBytes 12.6 Mbits/sec [ 12] 8.00-9.00 sec 768 KBytes 6.31 Mbits/sec [ 14] 8.00-9.00 sec 1.12 MBytes 9.47 Mbits/sec [ 16] 8.00-9.00 sec 768 KBytes 6.31 Mbits/sec [ 18] 8.00-9.00 sec 768 KBytes 6.31 Mbits/sec [ 20] 8.00-9.00 sec 896 KBytes 7.37 Mbits/sec [ 22] 8.00-9.00 sec 1.12 MBytes 9.47 Mbits/sec [SUM] 8.00-9.00 sec 11.8 MBytes 98.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 9.00-10.00 sec 1.75 MBytes 14.6 Mbits/sec [ 6] 9.00-10.00 sec 1.75 MBytes 14.6 Mbits/sec [ 8] 9.00-10.00 sec 896 KBytes 7.32 Mbits/sec [ 10] 9.00-10.00 sec 1.50 MBytes 12.5 Mbits/sec [ 12] 9.00-10.00 sec 384 KBytes 3.14 Mbits/sec [ 14] 9.00-10.00 sec 640 KBytes 5.23 Mbits/sec [ 16] 9.00-10.00 sec 640 KBytes 5.23 Mbits/sec [ 18] 9.00-10.00 sec 896 KBytes 7.32 Mbits/sec [ 20] 9.00-10.00 sec 1.00 MBytes 8.37 Mbits/sec [ 22] 9.00-10.00 sec 896 KBytes 7.32 Mbits/sec [SUM] 9.00-10.00 sec 10.2 MBytes 85.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec sender [ 4] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec receiver [ 6] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec sender [ 6] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec receiver [ 8] 0.00-10.00 sec 13.1 MBytes 11.0 Mbits/sec sender [ 8] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver [ 10] 0.00-10.00 sec 14.0 MBytes 11.7 Mbits/sec sender [ 10] 0.00-10.00 sec 14.0 MBytes 11.7 Mbits/sec receiver [ 12] 0.00-10.00 sec 4.25 MBytes 3.56 Mbits/sec sender [ 12] 0.00-10.00 sec 4.15 MBytes 3.48 Mbits/sec receiver [ 14] 0.00-10.00 sec 7.25 MBytes 6.08 Mbits/sec sender [ 14] 0.00-10.00 sec 7.16 MBytes 6.00 Mbits/sec receiver [ 16] 0.00-10.00 sec 4.88 MBytes 4.09 Mbits/sec sender [ 16] 0.00-10.00 sec 4.69 MBytes 3.93 Mbits/sec receiver [ 18] 0.00-10.00 sec 5.25 MBytes 4.40 Mbits/sec sender [ 18] 0.00-10.00 sec 5.21 MBytes 4.37 Mbits/sec receiver [ 20] 0.00-10.00 sec 5.50 MBytes 4.61 Mbits/sec sender [ 20] 0.00-10.00 sec 5.37 MBytes 4.51 Mbits/sec receiver [ 22] 0.00-10.00 sec 8.75 MBytes 7.34 Mbits/sec sender [ 22] 0.00-10.00 sec 8.58 MBytes 7.19 Mbits/sec receiver [SUM] 0.00-10.00 sec 99.0 MBytes 83.0 Mbits/sec sender [SUM] 0.00-10.00 sec 98.0 MBytes 82.2 Mbits/sec receiver iperf Done.UDP with expected bandwidth set for minimal datagram loss:
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c 10.30.20.20 -u -b 80000000 -R Connecting to host 10.30.20.20, port 5201 Reverse mode, remote host 10.30.20.20 is sending [ 4] local 10.10.10.42 port 63036 connected to 10.30.20.20 port 5201 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 4] 0.00-1.00 sec 9.85 MBytes 82.5 Mbits/sec 0.057 ms 0/1261 (0%) [ 4] 1.00-2.00 sec 9.52 MBytes 79.9 Mbits/sec 0.040 ms 0/1219 (0%) [ 4] 2.00-3.00 sec 9.39 MBytes 78.8 Mbits/sec 0.038 ms 0/1202 (0%) [ 4] 3.00-4.00 sec 9.60 MBytes 80.5 Mbits/sec 0.054 ms 0/1229 (0%) [ 4] 4.00-5.00 sec 9.53 MBytes 79.9 Mbits/sec 0.043 ms 0/1220 (0%) [ 4] 5.00-6.00 sec 9.54 MBytes 80.0 Mbits/sec 0.031 ms 0/1221 (0%) [ 4] 6.00-7.00 sec 9.60 MBytes 80.6 Mbits/sec 0.040 ms 0/1229 (0%) [ 4] 7.00-8.00 sec 9.48 MBytes 79.5 Mbits/sec 0.043 ms 6/1220 (0.49%) [ 4] 8.00-9.00 sec 9.54 MBytes 80.0 Mbits/sec 0.037 ms 0/1221 (0%) [ 4] 9.00-10.00 sec 9.55 MBytes 80.1 Mbits/sec 0.037 ms 0/1223 (0%) - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 4] 0.00-10.00 sec 96.5 MBytes 80.9 Mbits/sec 0.049 ms 6/12353 (0.049%) [ 4] Sent 12353 datagrams iperf Done. -
@rtw915 Can you possibly try iperf on pf boxes and outside the tunnel ?
Or port forward one end on windows and run iperf again over public ip and compare.
This will give an idea about tunnel overheads. -
I setup a port forward NAT rule. I don't want to have my public IP on here so it will be officiated. It does not seem like the tunnel is really adding that much overhead, right?
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c public_IP Connecting to host public_IP, port 5201 [ 4] local 10.10.10.42 port 65164 connected to public_IP port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 768 KBytes 6.29 Mbits/sec [ 4] 1.00-2.00 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 2.00-3.00 sec 1.88 MBytes 15.7 Mbits/sec [ 4] 3.00-4.00 sec 1.88 MBytes 15.7 Mbits/sec [ 4] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec [ 4] 5.00-6.00 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 6.00-7.00 sec 1.88 MBytes 15.7 Mbits/sec [ 4] 7.00-8.00 sec 1.88 MBytes 15.7 Mbits/sec [ 4] 8.00-9.00 sec 2.00 MBytes 16.8 Mbits/sec [ 4] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec sender [ 4] 0.00-10.00 sec 18.0 MBytes 15.1 Mbits/sec receiver iperf Done.Optimal TCP window size:
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c public_IP -w 2250000 Connecting to host public_IP, port 5201 [ 4] local 10.10.10.42 port 60842 connected to public_IP port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 2.88 MBytes 24.1 Mbits/sec [ 4] 1.00-2.00 sec 1.00 MBytes 8.39 Mbits/sec [ 4] 2.00-3.00 sec 2.25 MBytes 18.9 Mbits/sec [ 4] 3.00-4.00 sec 4.12 MBytes 34.6 Mbits/sec [ 4] 4.00-5.00 sec 5.12 MBytes 43.0 Mbits/sec [ 4] 5.00-6.00 sec 6.50 MBytes 54.6 Mbits/sec [ 4] 6.00-7.00 sec 8.25 MBytes 69.2 Mbits/sec [ 4] 7.00-8.00 sec 9.25 MBytes 77.6 Mbits/sec [ 4] 8.00-9.00 sec 6.00 MBytes 50.3 Mbits/sec [ 4] 9.00-10.00 sec 3.00 MBytes 25.2 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 48.4 MBytes 40.6 Mbits/sec sender [ 4] 0.00-10.00 sec 46.5 MBytes 39.0 Mbits/sec receiver iperf Done.10 Parallel streams:
C:\Temp\iperf-3.1.3-win64>iperf3.exe -c public_IP -P 10 Connecting to host public_IP, port 5201 [ 4] local 10.10.10.42 port 59407 connected to public_IP port 5201 [ 6] local 10.10.10.42 port 59417 connected to public_IP port 5201 [ 8] local 10.10.10.42 port 59421 connected to public_IP port 5201 [ 10] local 10.10.10.42 port 59431 connected to public_IP port 5201 [ 12] local 10.10.10.42 port 59441 connected to public_IP port 5201 [ 14] local 10.10.10.42 port 59451 connected to public_IP port 5201 [ 16] local 10.10.10.42 port 59464 connected to public_IP port 5201 [ 18] local 10.10.10.42 port 59465 connected to public_IP port 5201 [ 20] local 10.10.10.42 port 59475 connected to public_IP port 5201 [ 22] local 10.10.10.42 port 59488 connected to public_IP port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 256 KBytes 2.10 Mbits/sec [ 6] 0.00-1.00 sec 896 KBytes 7.33 Mbits/sec [ 8] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 10] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 12] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 14] 0.00-1.00 sec 256 KBytes 2.10 Mbits/sec [ 16] 0.00-1.00 sec 896 KBytes 7.33 Mbits/sec [ 18] 0.00-1.00 sec 640 KBytes 5.24 Mbits/sec [ 20] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [ 22] 0.00-1.00 sec 384 KBytes 3.14 Mbits/sec [SUM] 0.00-1.00 sec 4.75 MBytes 39.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 512 KBytes 4.20 Mbits/sec [ 6] 1.00-2.00 sec 2.00 MBytes 16.8 Mbits/sec [ 8] 1.00-2.00 sec 512 KBytes 4.20 Mbits/sec [ 10] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec [ 12] 1.00-2.00 sec 512 KBytes 4.20 Mbits/sec [ 14] 1.00-2.00 sec 512 KBytes 4.20 Mbits/sec [ 16] 1.00-2.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 1.00-2.00 sec 896 KBytes 7.34 Mbits/sec [ 20] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 22] 1.00-2.00 sec 256 KBytes 2.10 Mbits/sec [SUM] 1.00-2.00 sec 7.12 MBytes 59.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 512 KBytes 4.20 Mbits/sec [ 6] 2.00-3.00 sec 2.00 MBytes 16.8 Mbits/sec [ 8] 2.00-3.00 sec 768 KBytes 6.29 Mbits/sec [ 10] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec [ 12] 2.00-3.00 sec 512 KBytes 4.20 Mbits/sec [ 14] 2.00-3.00 sec 512 KBytes 4.20 Mbits/sec [ 16] 2.00-3.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 2.00-3.00 sec 1.25 MBytes 10.5 Mbits/sec [ 20] 2.00-3.00 sec 256 KBytes 2.10 Mbits/sec [ 22] 2.00-3.00 sec 512 KBytes 4.20 Mbits/sec [SUM] 2.00-3.00 sec 8.25 MBytes 69.2 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 640 KBytes 5.24 Mbits/sec [ 6] 3.00-4.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 3.00-4.00 sec 896 KBytes 7.34 Mbits/sec [ 10] 3.00-4.00 sec 384 KBytes 3.14 Mbits/sec [ 12] 3.00-4.00 sec 768 KBytes 6.29 Mbits/sec [ 14] 3.00-4.00 sec 640 KBytes 5.24 Mbits/sec [ 16] 3.00-4.00 sec 2.00 MBytes 16.8 Mbits/sec [ 18] 3.00-4.00 sec 896 KBytes 7.34 Mbits/sec [ 20] 3.00-4.00 sec 256 KBytes 2.10 Mbits/sec [ 22] 3.00-4.00 sec 640 KBytes 5.24 Mbits/sec [SUM] 3.00-4.00 sec 8.88 MBytes 74.4 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 896 KBytes 7.34 Mbits/sec [ 6] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 4.00-5.00 sec 1.12 MBytes 9.43 Mbits/sec [ 10] 4.00-5.00 sec 384 KBytes 3.14 Mbits/sec [ 12] 4.00-5.00 sec 1.00 MBytes 8.38 Mbits/sec [ 14] 4.00-5.00 sec 896 KBytes 7.34 Mbits/sec [ 16] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 4.00-5.00 sec 1.00 MBytes 8.38 Mbits/sec [ 20] 4.00-5.00 sec 384 KBytes 3.14 Mbits/sec [ 22] 4.00-5.00 sec 768 KBytes 6.29 Mbits/sec [SUM] 4.00-5.00 sec 10.1 MBytes 84.9 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec [ 6] 5.00-6.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 5.00-6.00 sec 1.62 MBytes 13.6 Mbits/sec [ 10] 5.00-6.00 sec 640 KBytes 5.25 Mbits/sec [ 12] 5.00-6.00 sec 1.38 MBytes 11.5 Mbits/sec [ 14] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec [ 16] 5.00-6.00 sec 2.00 MBytes 16.8 Mbits/sec [ 18] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec [ 20] 5.00-6.00 sec 640 KBytes 5.25 Mbits/sec [ 22] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec [SUM] 5.00-6.00 sec 12.8 MBytes 107 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.00 sec 1.62 MBytes 13.6 Mbits/sec [ 6] 6.00-7.00 sec 2.00 MBytes 16.8 Mbits/sec [ 8] 6.00-7.00 sec 1.88 MBytes 15.7 Mbits/sec [ 10] 6.00-7.00 sec 896 KBytes 7.34 Mbits/sec [ 12] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec [ 14] 6.00-7.00 sec 1.62 MBytes 13.6 Mbits/sec [ 16] 6.00-7.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 6.00-7.00 sec 1.88 MBytes 15.7 Mbits/sec [ 20] 6.00-7.00 sec 768 KBytes 6.29 Mbits/sec [ 22] 6.00-7.00 sec 1.38 MBytes 11.5 Mbits/sec [SUM] 6.00-7.00 sec 15.6 MBytes 131 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 7.00-8.00 sec 1.00 MBytes 8.39 Mbits/sec [ 6] 7.00-8.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 7.00-8.00 sec 2.00 MBytes 16.8 Mbits/sec [ 10] 7.00-8.00 sec 512 KBytes 4.20 Mbits/sec [ 12] 7.00-8.00 sec 1.88 MBytes 15.7 Mbits/sec [ 14] 7.00-8.00 sec 128 KBytes 1.05 Mbits/sec [ 16] 7.00-8.00 sec 2.00 MBytes 16.8 Mbits/sec [ 18] 7.00-8.00 sec 1.88 MBytes 15.7 Mbits/sec [ 20] 7.00-8.00 sec 512 KBytes 4.20 Mbits/sec [ 22] 7.00-8.00 sec 128 KBytes 1.05 Mbits/sec [SUM] 7.00-8.00 sec 11.9 MBytes 99.6 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 8.00-9.00 sec 1.38 MBytes 11.5 Mbits/sec [ 6] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec [ 10] 8.00-9.00 sec 768 KBytes 6.29 Mbits/sec [ 12] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec [ 14] 8.00-9.00 sec 256 KBytes 2.10 Mbits/sec [ 16] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec [ 20] 8.00-9.00 sec 640 KBytes 5.24 Mbits/sec [ 22] 8.00-9.00 sec 384 KBytes 3.14 Mbits/sec [SUM] 8.00-9.00 sec 12.8 MBytes 107 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 9.00-10.00 sec 1.75 MBytes 14.7 Mbits/sec [ 6] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec [ 8] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec [ 10] 9.00-10.00 sec 1.00 MBytes 8.39 Mbits/sec [ 12] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec [ 14] 9.00-10.00 sec 384 KBytes 3.15 Mbits/sec [ 16] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec [ 18] 9.00-10.00 sec 1.88 MBytes 15.7 Mbits/sec [ 20] 9.00-10.00 sec 1.00 MBytes 8.39 Mbits/sec [ 22] 9.00-10.00 sec 384 KBytes 3.15 Mbits/sec [SUM] 9.00-10.00 sec 13.9 MBytes 116 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 9.75 MBytes 8.18 Mbits/sec sender [ 4] 0.00-10.00 sec 9.70 MBytes 8.14 Mbits/sec receiver [ 6] 0.00-10.00 sec 18.1 MBytes 15.2 Mbits/sec sender [ 6] 0.00-10.00 sec 18.1 MBytes 15.2 Mbits/sec receiver [ 8] 0.00-10.00 sec 12.9 MBytes 10.8 Mbits/sec sender [ 8] 0.00-10.00 sec 12.9 MBytes 10.8 Mbits/sec receiver [ 10] 0.00-10.00 sec 5.12 MBytes 4.30 Mbits/sec sender [ 10] 0.00-10.00 sec 5.04 MBytes 4.23 Mbits/sec receiver [ 12] 0.00-10.00 sec 11.9 MBytes 9.96 Mbits/sec sender [ 12] 0.00-10.00 sec 11.8 MBytes 9.92 Mbits/sec receiver [ 14] 0.00-10.00 sec 6.25 MBytes 5.24 Mbits/sec sender [ 14] 0.00-10.00 sec 6.11 MBytes 5.12 Mbits/sec receiver [ 16] 0.00-10.00 sec 18.1 MBytes 15.2 Mbits/sec sender [ 16] 0.00-10.00 sec 18.1 MBytes 15.2 Mbits/sec receiver [ 18] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec sender [ 18] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec receiver [ 20] 0.00-10.00 sec 4.75 MBytes 3.98 Mbits/sec sender [ 20] 0.00-10.00 sec 4.65 MBytes 3.90 Mbits/sec receiver [ 22] 0.00-10.00 sec 5.75 MBytes 4.82 Mbits/sec sender [ 22] 0.00-10.00 sec 5.55 MBytes 4.66 Mbits/sec receiver [SUM] 0.00-10.00 sec 106 MBytes 88.9 Mbits/sec sender [SUM] 0.00-10.00 sec 105 MBytes 88.4 Mbits/sec receiver iperf Done.Running the UDP switch publicly resulted in hundreds of datagrams being received out-of-order.
-
@rtw915 Well, yes, it looks like the tunnel isn't adding anything significant. However speed does fluctuate a lot and this is probably an indication for congestion somewhere.
Not much you can do I guess too.
With such fluctuations, anything tcp will suffer, especially anything smb based
7zip supports a multihread option in bzip2 format.
Check it out. (also be careful with multithreading. It can easily saturate all your resources.
Also robocopy supports parallel copies, (that is if you break up your file in volumes)
Linux is better at compressing, splitting, rsyncing and decompressing, but probably introducing a new server just for that might be an overkill too. -
Mmm, 38Mbps doesn't seem too bad given those results though. That's close to what you see with a single stream in iperf using the optimised window size.
Given that the tunnel itself (or even the route) doesn't support more that 80Mbps it's about what I might expect there. I'm not sure if there's any improvement to be found without changing something significant.Steve
-
@netblues I remembered why I got stuck when trying to split a single file into smaller volumes. The source does not have enough space to duplicate the file into the smaller volumes to then transfer in parallel. Is it possible to start moving the created volumes in parallel as they are created?
@stephenw10 I agree, but it is just too slow.
-
Higher WAN bandwidth? Faster hardware? What's it running on now?
-
Using Mathis equation: Rate < (MSS/RTT)*(1 / sqrt(p))
Where:
MSS: maximum segment size - 1300 for IPSec overhead
RTT: round trip time - 90ms
P: Loss rate in % - .0002 seems about right I hope10213764.62 bytes < (1300/.09) * (1/SQRT(.0002%))
10213764.62 bytes * 8 /10^6 = ~81Mbps
This calculator https://wintelguy.com/wanperf.pl does the Mathis equation too, and increasing the WAN bandwidth does not increase the throughput.
@stephenw10 We are running SYS-1019D-FRN8TP with 64GB of mem. Based on the minimal throughput difference between public and IPSec iperf tests and Mathis equation output, I don’t think it is a hardware issue. I think TCP connections need to be parallelized and/or a proxy to perform TCP ACK trickery with an optimal TCP window size. I don't know, maybe there is a third solution.
-Ryan
-
Ok, so after a few more hours of googling I found a file transfer tool for high latency connections that seems to work great! The throughput that UFTP http://uftp-multicast.sourceforge.net/ was able to achieve almost saturated the bandwidth of the connection through the tunnel (186 Mbps)! I tested with an 8GB file.
The remote side (Client) I ran this:
uftpd -B 2250000 -L C:\temp\uftpd_log.txt -F @LOGOn the local side (Server) this:
uftp -f -B 2250000 -b 1200 -R 200000 -M 10.30.20.20 "C:\temp\2-25-2021.pcapng"Transfer summary:
Transfer status: Host: 0x8811096B Status: Completed time: 359.405 seconds Total elapsed time: 359.405 seconds Overall throughput: 23350.92 KB/s ----------------------------- Finishing group Sending DONE 1.1 Got COMPLETE from client 0x8811096B Late completions: Sending DONE_CONF 2.1I think this solves the batch file transfer speed issue. Now the SQL team needs me to find a way to improve SQL linked server transfer rates to synchronize transactions.
-
Nice find!
-
@rtw915 said in WAN optimization/acceleration:
Now the SQL team needs me to find a way to improve SQL linked server transfer rates to synchronize transactions.
This will bring you back to the initial wan accelerator solution.
The only other possible solution is to redesing the db subsystem, utilizing some way of sql replication, taking into consideration propagation delays
