Performance mystery with PIA on pfsense

whosmatt

Greetings all,

I set up PIA on my pfsense router, and everything is working as expected except for the throughput. I'm unable to get more than about 50Mbps down through pfsense, but if I install the client locally on a machine on my network, I can get close to my full 150Mbps down. Here's what's baffling me: watching the CPU usage in top, I never see openvpn go over about 30% usage. If I understand top correctly, that's 30% of one core. I'd expect it to at least max out a CPU core if my system is simply too slow. I've tried both BF-128 and AES-256-CBC (with and without hardware acceleration enabled) and don't see much difference.

Here's my specs:

CPU: AMD Sempron 2650 (dual cores, 1.45Ghz, AES-NI support)
Motherboard: MSI AM1I
RAM: 4GB DDR3
NIC: HP NC360T PCIe x4 (Intel 82571EB, em driver)

Anything ring a bell? I realize this isn't a world beating system but it should be able to do better than what I'm seeing.

mauroman33

Did you install the client locally on the same machine?

Have you tried to find out the OpenVPN performance of the CPU?

You could take a look here:
https://forum.pfsense.org/index.php?topic=115673.0

I use PIA and I have no problem to get close my full 100Mbps down with this connection log:

Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256-bit key
Data Channel Encrypt: Using 256-bit message hash 'SHA256' for HMAC authentication
Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256-bit key
Data Channel Decrypt: Using 256-bit message hash 'SHA256' for HMAC authentication
Control Channel: TLSv1.2, cipher TLSv1 / SSLv3 DHE-RSA-AES256-SHA, RSA 4096 bit

whosmatt

Found this: https://forum.pfsense.org/index.php?topic=88758.0 Sounds like exactly what I've got going on.

mauroman33

so did you solved just increaseing the TCP/UDP socket send and receive buffers size?

if you want to try, these are my custom options:

explicit-exit-notify 2;
ifconfig-nowarn;
tls-client;
persist-key;
persist-tun;
remote-cert-tls server;
reneg-sec 0;
auth-nocache;
tls-cipher TLS-DHE-RSA-WITH-AES-256-GCM-SHA384;
fast-io;
sndbuf 524288;
rcvbuf 524288

whosmatt

I added

fast-io;
sndbuf 524288;
rcvbuf 524288

to my config and can get about 70Mbps down now. I'll keep trying, but I'm traveling this week and have only remote access to my network so it's kind of difficult to really gauge the performance without other factors getting in the way. Thanks for the advice.

mauroman33

Very good, it's a 40% increase!

If you have not already done, you may try to activate PowerD in "Maximum" mode in System-Advanced-Miscellaneous and, if supported, to enable AES-NI in Cryptographic Hardware.

Anyway it may be possible that's close to the limit reached by the Sempron 2650.

To find it out, you should perform the test suggested by Ira in
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

Run from the GUI:
openvpn –genkey --secret /tmp/secret
and
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

Then to give the execution time in seconds in real-world meaning:
(3200 / execution_time_seconds) = Projected Maximum Performance OpenVPN in Mbps

As you can see from what I have tested in
https://forum.pfsense.org/index.php?topic=115673.msg642058#msg642058
CPUs in the same class may have different performance depending on the presence AES-NI support.

Please, let me know your benchmark.

whosmatt

The performance test with AES-NI enabled gives me a theoretical max of 92Mbps.

mauroman33

With a benchmark like that I would have expected about 100 Mbps in download.
I regret not being able to help you more.
The only thing I can add to the info about my settings is that I'm running the 2.3.2 stable version.
If you will solve the issue, I'd like to read the adopted solution.
Cheers

techy82

i've got 200mbps but can only seem to get 20mbps via pia

I have a

Intel(R) Core(TM) i5-6400 CPU @ 2.70GHz
2 CPUs: 2 package(s) x 1 core(s)

and the following custom options

auth-user-pass /etc/openvpn-password.txt;
fast-io;
sndbuf 524288;
rcvbuf 524288

AES cryptographic is enabled,

if you find a way to improve it it would be great to know

thanks!

mauroman33

@techy82
just out of curiosity, what PIA server are you connecting?

techy82

new york city

mauroman33

@techy82:

new york city

Never tried. I usually go through denmark or sweden and with the configuration above I easily get the limit of the line (100Mbps)

whosmatt

@mauroman33:

I regret not being able to help you more.

You've helped plenty. Thanks. Once I get home from my travels and am not testing remotely I'll be able to try tweaking a few more settings. Worst case I buy an Athlon 5350 or 5370 for a 50% + single thread improvement.

techy82

@mauroman33:

@techy82:

new york city

Never tried. I usually go through denmark or sweden and with the configuration above I easily get the limit of the line (100Mbps)

I'll try some different servers later and see how that goes, Thanks

M_Devil

pfSense 2.3.2. using PIA with 2 OpenVPN clients combined in one Gateway Group (PIA could not deliver coding/decoding speed with one connection).

Get full ISP speed (500/500 Mbit) with CPU load of ~30%

Hardware: intel i5-3450
VPN

AES-256-CBC
SHA256
fast-io;
sndbuf 524288;
rcvbuf 524288
Hardware acceleration enabled.
2 fixed (same country as client) IP adresses for PIA.

So it should not be PIA restricted, seems CPU restricted.

mauroman33

@M_Devil:

pfSense 2.3.2. using PIA with 2 OpenVPN clients combined in one Gateway Group (PIA could not deliver coding/decoding speed with one connection).

Get full ISP speed (500/500 Mbit) with CPU load of ~30%

Hardware: intel i5-3450
VPN

AES-256-CBC

SHA256

fast-io;

sndbuf 524288;

rcvbuf 524288

Hardware acceleration enabled.

2 fixed (same country as client) IP adresses for PIA.

So it should not be PIA restricted, seems CPU restricted.

This is interesting.
How do you set the priority in the group? Both Tier 1 I guess.
And what speed did you get using only one OpenVPN client?

M_Devil

Indeed, both tier 1.
When using Blowfish (only option in the past), I could not push it above 200Mbit and unstable. By then I came up with the 2 client setup and that worked like a charm.
Recently I switched to AES and with a quick test it seems that it could handle ISP speed also with one connection. I stick with 2 connection for stability and extra security reasons.

mauroman33

Thanks for your reply.

I'm curious about the OpenVPN performance of various CPUs because of a future upgrade of my line and your CPU seems really interesting from my point of view.

If you are willing, could you performed the simple OpenVPN benchmark referenced here?
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743 (Reply # 9 message)

From the GUI run

openvpn –genkey --secret / tmp / secret

--test time openvpn-crypto --secret / tmp / secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

Then to give the execution time in seconds in real-world meaning:
(3200 / execution_time_seconds) = Projected Maximum Performance OpenVPN in Mbps

My Celeron N3150 gets a value of 116 Mbps that's the same value that normally reaches during download trough a PIA client.

M_Devil

Execution time = 9.433 seconds, so Projected Maximum Performance = 339 Mbit.

Does this represent single core performance?

Edit: In this case it does not represent maximum performance. It could easly push 500Mbit with ~30% load.

mauroman33

As far as I know OpenVPN works in single thread, but I could be wrong… anyway your CPU is a beast! ;)
Thanks for letting me know.