OpenVPN Slow - local network test

spyder0552 · Jun 22, 2020, 5:46 PM

@Rico
Hey, just replied below. Yes, I did try that on Friday and it did not make a difference.

Is anyone out there trying this with Windows 10 clients? Perhaps this is a windows issue?

Pippin · Jun 22, 2020, 6:09 PM

@spyder0552 said in OpenVPN Slow - local network test:

[2.4.5-RELEASE][admin@pfSense.seradex.local]/root: openvpn --genkey --secret /tm p/secret
[2.4.5-RELEASE][admin@pfSense.seradex.local]/root: time openvpn --test-crypto -- secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm
Fri Jun 19 14:47:16 2020 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.007u 0.000s 0:10.00 100.0% 843+177k 0+0io 0pf+0w
[2.4.5-RELEASE][admin@pfSense.seradex.local]/root:

By the looks of that ^^^ it theoretically could do ~320Mbps

How about testing both ways? (client>server - server>client)
Which I5 is that?

Your top -aSH screenshot is timed wrong, ..... I guess.

spyder0552 · Jun 22, 2020, 6:24 PM

@Pippin
Home test lab is i5-4690K (yeah..little dated...best I can get atm).

I can't really get it to test from the inside out. The wall seems to be blocking it out. I guess the IP I would connect to would be the end point of the tunel...but not working it seems.

Pippin · Jun 22, 2020, 6:45 PM

You can reverse test mode by using iperf3 -R
"-R, Reverse test mode – Server sends, client receives"

spyder0552 · Jun 22, 2020, 7:20 PM

@Pippin
WOW...actually slower in reverse. Averaged only 100 Mb/s

But if I drop the VPN and just go through the firewall, it goes up to 900

spyder0552 · Jun 22, 2020, 7:24 PM

@spyder0552
Just to show the config on the client/server side
Client:
dev tun
persist-tun
persist-key
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
auth SHA256
tls-client
client
resolv-retry infinite
remote 192.168.2.98 1194 udp4
verify-x509-name "OpenVPN-Server-Cert" name
auth-user-pass
pkcs12 pfSense-UDP4-1194-vpn.p12
tls-crypt pfSense-UDP4-1194-vpn-tls.key
remote-cert-tls server

Server:
dev ovpns1
verb 1
dev-type tun
dev-node /dev/tun1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
cipher AES-128-GCM
auth SHA256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local 192.168.2.98
tls-server
server 192.168.99.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc/server1
username-as-common-name
plugin /usr/local/lib/openvpn/plugins/openvpn-plugin-auth-script.so /usr/local/sbin/ovpn_auth_verify_async user TG9jYWwgRGF0YWJhc2U= false server1 1194
tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'OpenVPN-Server-Cert' 1"
lport 1194
management /var/etc/openvpn/server1.sock unix
push "route 192.168.1.0 255.255.255.0"
push "dhcp-option DOMAIN test.local"
push "dhcp-option DNS 8.8.8.8"
client-to-client
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /etc/dh-parameters.2048
tls-crypt /var/etc/openvpn/server1.tls-crypt
ncp-ciphers AES-128-GCM
persist-remote-ip
float
topology subnet
fast-io

johnpoz · Jun 22, 2020, 9:51 PM

@spyder0552 said in OpenVPN Slow - local network test:

Just setup a HyperV box.

That doesn't seem like a legit test.. My test will be this

windows 10 PC --- sg4860--- NAS

Will set this up later.. But you don't show any of the mtu settings or txqueuelen..

johnpoz · Jun 23, 2020, 1:36 PM

Ok I just set this up...

With default everything on open seeing..

$ iperf3 -c 192.168.9.10
warning: Ignoring nonsense TCP MSS 334848
Connecting to host 192.168.9.10, port 5201
[  5] local 10.0.100.2 port 52251 connected to 192.168.9.10 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  25.5 MBytes   214 Mbits/sec
[  5]   1.00-2.00   sec  29.5 MBytes   248 Mbits/sec
[  5]   2.00-3.00   sec  32.4 MBytes   272 Mbits/sec
[  5]   3.00-4.00   sec  31.1 MBytes   261 Mbits/sec
[  5]   4.00-5.00   sec  29.5 MBytes   247 Mbits/sec
[  5]   5.00-6.00   sec  30.8 MBytes   258 Mbits/sec
[  5]   6.00-7.00   sec  30.6 MBytes   257 Mbits/sec
[  5]   7.00-8.00   sec  31.6 MBytes   265 Mbits/sec
[  5]   8.00-9.00   sec  31.0 MBytes   260 Mbits/sec
[  5]   9.00-10.00  sec  32.0 MBytes   268 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   304 MBytes   255 Mbits/sec                  sender
[  5]   0.00-10.01  sec   304 MBytes   255 Mbits/sec                  receiver

Let me see if can tweak that a bit..

win10 pc (192.168.200.10) --- switch - 192.168.200.1 (test igb4) pfsense (sg4860) ( igb0 lan) 192.168.9.253 -- switch -- 192.168.9.10 (NAS)

Tweaking didn't make much difference to be honest, but using aes-128-gcm vs cbc was huge difference.. When changed to cbc vs gcm when to 130 vs mid 200's

spyder0552 · Jun 23, 2020, 1:36 PM

@johnpoz
Thanks for doing the testing.
I am surprised to see that the best we can get on a full 1Gb/s link is ~250 Mb/s
So I guess my ~150Mb/s on the first crack is not horrible?

I know there is overhead to deal with...just surprised it is so much. This is looking like a 70% loss of speed using OpenVpn.
Is this what others are seeing as well?

johnpoz · Jun 23, 2020, 1:49 PM

openvpn is single threaded.. its easy to use - its never been "speedy" ;)

Keep in mind my sg4860, not a rocketship vpn endpoint concentrater either..

It has enough umph to get the job done with lower power requirements.. But prob not what I would use for my vpn endpoint if what I wanted to as much throughput as possible.. Nor would openvpn be my first choice in that area - ipsec is better geared for throughput..

Openvpn advantage is ease of use, and deployment, etc.

But overall your sort of test with everything on the same vm host is not really a valid sort of testing.. It works for poc, etc. But its not going to be a good indicator of what sort of bandwidth you could expect when using in the real world..

Pippin · Jun 23, 2020, 2:04 PM

But overall your sort of test with everything on the same vm host is not really a valid sort of testing

Yes I think so too.
Looking at the hardware it should be capable of more.
.

Nor would openvpn be my first choice in that area

Wait a bit, who knows ;)

johnpoz · Jun 23, 2020, 2:26 PM

@Pippin said in OpenVPN Slow - local network test:

Wait a bit, who knows ;)

If the goal was pure throughput, openvpn would not be on the top of the list of choices.. It has many other attributes it shines at.. But if what I am looking for is closest to line speed using least amount of horsepower.. Then no its not on the top of really any list ;)

spyder0552 · Jun 23, 2020, 2:34 PM

@johnpoz
That is what I am concluding.
OpenVPN = easy to deploy and flexible....just not that speedy.

I am in the process of setting up another test to use IPSEC and compare. I will post my results for those that are interested.

For this initial issue, I am calling it closed as it seems we have beaten the heck out of it. Thank you all for the fantastic support on this. So very much appreciated.

Pippin · Jun 23, 2020, 2:36 PM

If the goal was pure throughput, openvpn would not be on the top of the list of choices

Sure, but maybe it will get to that top some time in the future.

johnpoz · Jun 23, 2020, 3:16 PM

^ we can hope yeah ;)

Pippin · Sep 27, 2020, 9:45 PM

@johnpoz said in OpenVPN Slow - local network test:

^ we can hope yeah ;)

Although for Linux but here it is:
https://github.com/OpenVPN/ovpn-dco

Pippin · Feb 22, 2022, 7:44 PM

Nice:
https://reviews.freebsd.org/D34340

thiasaef · Feb 26, 2022, 6:28 PM

I get even worse results ...

Machine A (pfSense 2.6.0):

time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm
2022-02-26 19:22:27 Cipher negotiation is disabled since neither P2MP client nor server mode is enabled
0.192u 0.000s 0:00.19 100.0%	601+171k 1+0io 0pf+0w

Machine B (pfSense 2.6.0):

time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm
2022-02-26 19:22:35 Cipher negotiation is disabled since neither P2MP client nor server mode is enabled
0.587u 0.023s 0:00.61 98.3%	618+176k 0+0io 0pf+0w

I spent most of the day trying to reach reasonable speeds, and this is the result:

iperf3 -c 172.16.16.1 -R
Connecting to host 172.16.16.1, port 5201
Reverse mode, remote host 172.16.16.1 is sending
[  5] local 172.16.16.2 port 53032 connected to 172.16.16.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  6.10 MBytes  51.2 Mbits/sec                  
[  5]   1.00-2.00   sec  8.03 MBytes  67.4 Mbits/sec                  
[  5]   2.00-3.00   sec  7.28 MBytes  61.1 Mbits/sec                  
[  5]   3.00-4.00   sec  7.60 MBytes  63.8 Mbits/sec                  
[  5]   4.00-5.00   sec  6.77 MBytes  56.8 Mbits/sec                  
[  5]   5.00-6.00   sec  7.17 MBytes  60.1 Mbits/sec                  
[  5]   6.00-7.00   sec  8.87 MBytes  74.4 Mbits/sec                  
[  5]   7.00-8.00   sec  7.41 MBytes  62.2 Mbits/sec                  
[  5]   8.00-9.01   sec  7.54 MBytes  62.9 Mbits/sec                  
[  5]   9.01-10.00  sec  6.44 MBytes  54.3 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.14  sec  73.4 MBytes  60.7 Mbits/sec   91             sender
[  5]   0.00-10.00  sec  73.2 MBytes  61.4 Mbits/sec                  receiver