OpenVPN performance

mbf210 · Sep 27, 2015, 2:06 AM

I am working on diagnosing why my OpenVPN traffic is capped at about 7Mbps.

SPECS of system
ATT Gigabit fiber, verified from desktop can achieve 900Mbps up and down through speed test.
Router: HP GT7725 2.3Ghz Dual Core AMD Turion 2G RAM / 3x1G NIC
Uploads and downloads from Amazon S3, Google Drive and Steam (download only) are showing 500Mbps and more.
Opening a single port publicly to NAT to an Apache server I can get 60Mbps. This is on a simple Amazon EC2 server which has a max of 70up and 70 down, so I am maxing out that servers connections.
Netgear small business class gigabit switch

OpenVPN
user auth with password
2048 bit keys
AES-256-CBC

Once on VPN I can see my entire network however speeds cap at 7Mbps, and are usually 3Mpbs.

Things I have tried:

I reviewed the article on net.inet.ip.fastforwarding = 1, https://forum.pfsense.org/index.php/topic,47567.0.html this did not improve
I changed the client and server MTU to 64800 mssfix 1440 http://abautu.blogspot.com/2013/07/improving-openvpn-thoughput.html no improvement. Same as when MTU was default of 1500 and mssfix was 0 which is default
I reviewed the settings here and it seems with AES-256-CBC you can get 125Mbps at least if not more https://community.openvpn.net/openvpn/wiki/Gigabit_Networks_Linux

What are other areas I can try to increase my bandwidth. I would like to be able to confirm 50Mbps which is the most that most places where I will be connecting to will have as a max download.

johnpoz · Sep 27, 2015, 12:14 PM

"Once on VPN I can see my entire network however speeds cap at 7Mbps, and are usually 3Mpbs."

What speeds cap? Are you trying to do cifs/smb (windows file copy) over a wan? Yeah that is going to blow chunks..

What is your remote clients speeds? You could have 10Ge up and down doesn't matter if client is 2/1 – also what is the latency on this remote client and again what is capped.. are you doing http xfer from server on your network to this remote vpn client, are they using your vpn to talk to the ec2 server?

What exactly is capped? Are you running iperf tests from this client into your network? What?

ega · Sep 27, 2015, 2:55 PM

Check this posts

https://forum.pfsense.org/index.php?topic=99536.0

https://forum.pfsense.org/index.php?topic=88758.0

mbf210 · Sep 30, 2015, 4:43 AM

I did an iperf test.

Local LAN to router - 700-850Mbps

Server listening on TCP port 5001
TCP window size: 63.7 KByte (default)
–----------------------------------------------------------
[ 4] local 192.168.1.1 port 5001 connected with 192.168.1.126 port 53290
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 832 MBytes 698 Mbits/sec
[ 5] local 192.168.1.1 port 5001 connected with 192.168.1.126 port 53419
[ 5] 0.0-10.0 sec 1009 MBytes 848 Mbits/sec

From Amazon box connected via OpenVPN I get the max of 7.26Mbps. This is about the fastest I can get from any device on the network. Regardless of transfer type (smb share, apache web server, iperf) or direction to or from LAN.

Server listening on TCP port 5001
TCP window size: 63.7 KByte (default)
–----------------------------------------------------------
[ 4] local 192.168.1.1 port 5001 connected with 192.168.10.6 port 49218
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.3 sec 8.88 MBytes 7.26 Mbits/sec

I am using the default port of 1194. Does ATT (or other providers) throttle this port number possibly? I have seen reports of services like Netflix being slowed down.

mbf210 · Oct 4, 2015, 3:18 AM

Here are some other things I tried to eliminate bottleneck areas.
Changed to a different port than 1194 - in suspect ATT might throttle it. no difference, max still about 7Mbps
Changed chiper to lower than initial (AES-256-CBC). No effect. I did more reading and hardware and processor types can improve performance https://doc.pfsense.org/index.php/Are_cryptographic_accelerators_supported

This command will test the time to do encryption

/usr/bin/openssl speed -evp aes-128-cbc -engine cryptode

Doing just openssl speed goes through all ciphers. All OpenVPN ones are at 3 seconds on my system which seems typical. When adding on an encryption card you can get 0.1s. Some on the pfsense forums and other places recommend Soekris VPN1411 shows up to 34Mbps.

Several other blogs and posts show a max around 7-10Mbps with iperf and encryption?

Other articles state if you need more bandwidth go with ipsec. That is my current next direction to properly set up ipsec.

johnpoz · Oct 4, 2015, 12:22 PM

that test is for 3 seconds.. What is the output of that command in bytes processed?

The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 39009.87k 40097.17k 43848.99k 116723.03k 119200.92k

If your worried about what an open vpn can do your system.. Why don't you take the WAN out of it and do some testing with a box connected right at your wan.. When I have some more coffee I will do a bit of testing.. Pretty sure even my vm pfsense running on n40l can do more than 7mbps

mbf210 · Oct 9, 2015, 2:01 AM

I did the testing without encryption only for testing purposes. It is now back on. The encryption is not speed issue. My eliminated steps show it to be the setup of OpenVPN in some way, pfsense setting somewhere, or some hardware driver type thing. The box is a dual core AMD box. According to top one of the CPUs is always at 50% when load is near zero. Not sure if of this either. that is for a different thread some other day.

If anyone else has other suggestions on how to tune this I will try them out.

cmb · Oct 9, 2015, 4:03 AM

If you're not using IPsec, go to System>Advanced, Tunables, and add a tunable for net.inet.ip.fastforwarding set to value 1. Save and apply changes and try again.