OpenVPN asymmetric bandwidth with iperf
-
On OpenVPN connections I am experiencing speed results vastly different depending on where the iperf client is being run.
When the remote side (running OpenVPN Client) initiates the connection, it runs at the speed of the pipe.
When the local side (behind OpenVPN Server) initiates the connection, it is much slower than the speed of the pipe.
In this case, the local pipe is 1 Gbs (fiber) and the client pipe is 45Mbps (T-3)Here you can see the results from the server which is behind the pfSense firewall which is running OpenVPN Server:
C:\>iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 8.00 KByte (default) ------------------------------------------------------------ [312] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62236 [340] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62237 [356] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62238 [372] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62239 [388] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62240 [404] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62242 [420] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62243 [436] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62244 [452] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62245 [468] local 172.31.0.75 port 5001 connected with 192.168.200.10 port 62246 [ ID] Interval Transfer Bandwidth [468] 0.0-10.0 sec 3.98 MBytes 3.32 Mbits/sec [404] 0.0-10.5 sec 1.91 MBytes 1.53 Mbits/sec [340] 0.0-10.9 sec 1.73 MBytes 1.33 Mbits/sec [420] 0.0-10.4 sec 3.43 MBytes 2.77 Mbits/sec [372] 0.0-10.7 sec 3.95 MBytes 3.09 Mbits/sec [452] 0.0-10.2 sec 3.99 MBytes 3.29 Mbits/sec [388] 0.0-10.6 sec 1.87 MBytes 1.48 Mbits/sec [356] 0.0-10.8 sec 3.84 MBytes 2.97 Mbits/sec [312] 0.0-11.2 sec 2.58 MBytes 1.94 Mbits/sec [436] 0.0-10.5 sec 1.34 MBytes 1.07 Mbits/sec [SUM] 0.0-11.3 sec 28.6 MBytes 21.2 Mbits/sec C:\>iperf -c 192.168.200.10 -w64K -P10 ------------------------------------------------------------ Client connecting to 192.168.200.10, TCP port 5001 TCP window size: 64.0 KByte ------------------------------------------------------------ [252] local 172.31.0.75 port 53973 connected with 192.168.200.10 port 5001 [244] local 172.31.0.75 port 53972 connected with 192.168.200.10 port 5001 [236] local 172.31.0.75 port 53971 connected with 192.168.200.10 port 5001 [228] local 172.31.0.75 port 53970 connected with 192.168.200.10 port 5001 [220] local 172.31.0.75 port 53969 connected with 192.168.200.10 port 5001 [212] local 172.31.0.75 port 53968 connected with 192.168.200.10 port 5001 [204] local 172.31.0.75 port 53967 connected with 192.168.200.10 port 5001 [196] local 172.31.0.75 port 53966 connected with 192.168.200.10 port 5001 [188] local 172.31.0.75 port 53965 connected with 192.168.200.10 port 5001 [180] local 172.31.0.75 port 53963 connected with 192.168.200.10 port 5001 [ ID] Interval Transfer Bandwidth [212] 0.0-11.0 sec 336 KBytes 251 Kbits/sec [244] 0.0-11.0 sec 320 KBytes 238 Kbits/sec [220] 0.0-11.3 sec 336 KBytes 243 Kbits/sec [180] 0.0-11.7 sec 408 KBytes 285 Kbits/sec [252] 0.0-12.0 sec 504 KBytes 343 Kbits/sec [196] 0.0-12.7 sec 232 KBytes 150 Kbits/sec [204] 0.0-12.9 sec 408 KBytes 260 Kbits/sec [236] 0.0-12.9 sec 384 KBytes 243 Kbits/sec [188] 0.0-13.1 sec 488 KBytes 306 Kbits/sec [228] 0.0-13.7 sec 328 KBytes 196 Kbits/sec [SUM] 0.0-13.7 sec 3.66 MBytes 2.24 Mbits/sec C:\> -
It would be interesting to know, for the example of the server end that you give, what does the client end say for window size?
(The server reports a default 8.0 KByte default windows size, but I expect the client will have asked for much more than that. Maybe the client in the fast configuration is using a much bigger windows size?)
And what is the typical ping time across the link you are testing?
That will allow you to calculate a reasonable window size to make sure the test is always pushing data into the pipe.
And, of course, I assume that the link has no other (significant) traffic at the time of test. -
Thanks for your consideration. Our default test is:
iperf -c <ip>-w64K -P10 - I ran the test and I would have used this
Ping is around 73ms. between sites, no other (significant) usage on the pipe</ip>
-
64KByte*8bits=512Kbits window. So 1 connection can send 512Kbits in the 73ms before the first ACK is returned. 512Kbits per 73ms = 7.013Mbps. 10 connections in parallel could pump through up to 70.13Mbps. So you should not be limited by window size.
I can't think of anything in OpenVPN server/client that should make a asymmetry like this - encrypting and decrypting data both take processing that would be similar. I assume the hardware at both ends has enough CPU to process the speeds, and that the T-3 link is bidirectional 45Mbps.
Can you do other things across the link (copy big files) and also get asymmetric speeds?
Any ideas from others welcome, as I have usually found that failure to achieve full link speed with iperf is due to low window size. -
Odd thing is that with an IPsec tunnel, the asymmetry is reversed, faster when the client is on my side of the house.