Asus N3050I-C for OpenVPN (100MBIT WAN)

Pippin

Granted it does not test routing the raw encrypted traffic, but that is a small part of the equation,

I wonder how accurate it would be compared to a iperf between two routed clients with server/pfSense in the middle?
Or even compared to a client-to-client setting?
Off course measuring on the server.

–tun-mtu 20000

Could you elaborate?
Why 20000? OpenSSL will be fed bigger packets? That`s not fair compared to real world,…I think?

I would invite others to correlate their experiences with this simple OpenVPN benchmark.

Since I'm still testing real world throughput for different scenarios, I will. (just need to find time enough :))

lra

@Pippin:

@lra:

Granted it does not test routing the raw encrypted traffic, but that is a small part of the equation,

–tun-mtu 20000

Could you elaborate?
Why 20000? OpenSSL will be fed bigger packets? That`s not fair compared to real world,…I think?

The 20000 is arbitrary, but does effect the magic number of 3200. If you used "–tun-mtu 2000" the magic number would be 32 but the test execution time would be too short to be accurate (less than a second).

The "openvpn --test-crypto" sequentially tests packets from 1 byte to 20000 bytes in size (per "--tun-mtu 20000") encrypting then decrypting them via the 'tun' interface driver.

While I agree if OpenVPN's --test-crypto additionally supported specifying a number of iterations with a fixed packet size would be more "real-world", the results using the existing "openvpn --test-crypto" still gives a useful benchmark per my testing.

mauroman33

@lra:

@BlueKobold:
openvpn --genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

( 3200 / execution_time_seconds ) = Projected Maximum OpenVPN Performance in Mbps
As for comparing OpenVPN performance, I have started using this benchmark:

But this says nothing about OpenVPN performance at all.
The above test provides an easy to perform, upper limit test for any one OpenVPN session. Granted it does not test routing the raw encrypted traffic, but that is a small part of the equation, and why this is a projected maximum OpenVPN performance.

Single core user-land performance, tun driver kernel performance and crypto performance are all part of the test, all related to overall OpenVPN performance.

I have tested several, mostly lower-end (PC Engines APU2C, Jetway NF9HG-2930, Lanner FW-7525B, etc.) hardware and the above test gives a good ballpark, projected maximum OpenVPN performance for any one OpenVPN session.

I would invite others to correlate their experiences with this simple OpenVPN benchmark.

here is my result
Quad Core Celeron N3150
Execution time: 27.7 secs.
Maximum OpenVPN: 115 Mbps

in the real world my home router allows me to get the 90% of my 100Mbps WAN connection through an OpenVPN client

Pippin

@mauroman33:

here is my result
Quad Core Celeron N3150
Execution time: 27.7 secs.
Maximum OpenVPN: 115 Mbps

in the real world my home router allows me to get the 90% of my 100Mbps WAN connection through an OpenVPN client

Hi,
Then the 100Mbps WAN is not sufficient enough to compare. I have the same CPU, N3150N-D3V and my throughput tests using iperf in a routed
ovpnclient> to <re0-ovpnserver-re1>to ovpnclient, I get max. 160 Mbits/sec., with no compression going on.
Keep in mind that in this scenario the load for the server is heavier then "normal" because theres extra crypto going on, so Im somewhat sceptical to the mentioned test.

At the moment I have no access to my box to compare against 115Mbps, as soon as I have I will post here.</re0-ovpnserver-re1>

mauroman33

@Pippin:

@mauroman33:

here is my result
Quad Core Celeron N3150
Execution time: 27.7 secs.
Maximum OpenVPN: 115 Mbps

in the real world my home router allows me to get the 90% of my 100Mbps WAN connection through an OpenVPN client

Hi,
Then the 100Mbps WAN is not sufficient enough to compare. I have the same CPU, N3150N-D3V and my throughput tests using iperf in a routed
ovpnclient> to <re0-ovpnserver-re1>to ovpnclient, I get max. 160 Mbits/sec., with no compression going on.
Keep in mind that in this scenario the load for the server is heavier then "normal" because theres extra crypto going on, so Im somewhat sceptical to the mentioned test.

At the moment I have no access to my box to compare against 115Mbps, as soon as I have I will post here.</re0-ovpnserver-re1>

Hi,
I think so too.
By running a speed test without VPN on my 100/20 connection, the average result is about 94Mbps.
My scenario involves the connection using an OpenVPN client (SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit). In that case the result is about 90Mbps.
I tried with 4 different VPN providers (IPVanish, PureVPN, PIA, VyprVPN) and the results are similar.
Next month I might have a chance to try on a 250/50 connection. I will post the result here.

Pippin

@mauroman33:

SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit

For the mentioned test, that is not relevant because the test involves the datachannel.
SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit is for the control channel. *See note.

What could be more interesting for comparison is the log showing this info:


Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Encrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Decrypt: Using 512 bit message hash 'SHA512' for HMAC authentication

*Note
I use

tls-version-min 1.2 or-highest

on both sides.
Server and client will negotiate the highest available TLS version.
With that setting you will probably get:

Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA

Maybe useful for you and others.

mauroman33

@Pippin:

@mauroman33:

SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit

For the mentioned test, that is not relevant because the test involves the datachannel.
SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit is for the control channel. *See note.

What could be more interesting for comparison is the log showing this info:
Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Encrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Decrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
*Note
I use
tls-version-min 1.2 or-highest
on both sides.
Server and client will negotiate the highest available TLS version.
With that setting you will probably get:
Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Maybe useful for you and others.

Thanks for the clarification

this is my log
Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Encrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Decrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 2048 bit RSA

mauroman33

@mauroman33:

@Pippin:

@mauroman33:

here is my result
Quad Core Celeron N3150
Execution time: 27.7 secs.
Maximum OpenVPN: 115 Mbps

in the real world my home router allows me to get the 90% of my 100Mbps WAN connection through an OpenVPN client

Hi,
Then the 100Mbps WAN is not sufficient enough to compare. I have the same CPU, N3150N-D3V and my throughput tests using iperf in a routed
ovpnclient> to <re0-ovpnserver-re1>to ovpnclient, I get max. 160 Mbits/sec., with no compression going on.
Keep in mind that in this scenario the load for the server is heavier then "normal" because theres extra crypto going on, so Im somewhat sceptical to the mentioned test.

At the moment I have no access to my box to compare against 115Mbps, as soon as I have I will post here.</re0-ovpnserver-re1>

Hi,
I think so too.
By running a speed test without VPN on my 100/20 connection, the average result is about 94Mbps.
My scenario involves the connection using an OpenVPN client (SSLv3 DHE-RSA-AES256-SHA, RSA 2048 bit). In that case the result is about 90Mbps.
I tried with 4 different VPN providers (IPVanish, PureVPN, PIA, VyprVPN) and the results are similar.
Next month I might have a chance to try on a 250/50 connection. I will post the result here.

I finally got to test the router with a 250/100 fiber connection.
The results are in line with expectations.
The Celeron N3150 is able to reach about 130Mbs via VPN client

The VPN connection log:
Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Encrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
Data Channel Decrypt: Using 256 bit message hash 'SHA256' for HMAC authentication
Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 4096 bit RSA

speedt1.png_thumb

speedt2.png_thumb

lra

@mauroman33, Thanks for the follow-up post.

It seems the simple OpenVPN benchmark formula referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

gives a reasonable base-line reference. I too have found the actual tested speed can be 5-20 % faster than the benchmark formula, and for some it is right on target.

Nothing beats an actual real-world test, but a quick CLI base-line test can be useful.

mauroman33

@lra:

@mauroman33, Thanks for the follow-up post.

It seems the simple OpenVPN benchmark formula referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

gives a reasonable base-line reference. I too have found the actual tested speed can be 5-20 % faster than the benchmark formula, and for some it is right on target.

Nothing beats an actual real-world test, but a quick CLI base-line test can be useful.

Hello, just a clarification.

Running the command I get this input:
27.41 real 25.62 user 1.77 sys

What do you mean for "execution_time_seconds" in the formula? The "real" value or the "user" value?

lra

@mauroman33:

@lra:

It seems the simple OpenVPN benchmark formula referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

Hello, just a clarification.

Running the command I get this input:
27.41 real 25.62 user 1.77 sys

What do you mean for "execution_time_seconds" in the formula? The "real" value or the "user" value?

Use the "real" value…

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

mauroman33

@lra:

@mauroman33:

@lra:

It seems the simple OpenVPN benchmark formula referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

Hello, just a clarification.

Running the command I get this input:
27.41 real 25.62 user 1.77 sys

What do you mean for "execution_time_seconds" in the formula? The "real" value or the "user" value?

Use the "real" value…

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

Thank you!

I saw that in a previous message you have tested a Celeron N2930 with those results
Execution time: 42.4 secs.
Maximum OpenVPN: 75 Mbps

If we consider that the Celeron N2930 is completely comparable with the Celeron N3150
http://www.cpubenchmark.net/compare.php?cmp%5B%5D=2255&cmp%5B%5D=2546
that got 117 Mbps as OpenVPN performance, we could assume the difference is totally due to the AES-NI support of the N3150.
What do you think about it?

lra

The AES-NI support of the N3150 is no doubt a large part of the increased performance, but there may be other factors as well.

Also, use this "OpenVPN benchmark formula" as a guide, not gospel.

yennhikorea

@BlueKobold:

Could you please expand on why you think the APU2 would be better? To me it seems to have much slower performance on paper?

For sure I will do that. Only counting together the performance tech. specs. would be like:

APU2C2 is 4 CPU cores & AES-NI

Intel i210AT consumer grade NICs

2 GB normal RAM

3 x miniPCIe + SIM

mSATA support & SATA Port

wide spread and well supported

APU2C4 is 4 CPU cores & AES-NI

Intel i211AT LAN Ports server grade NICs

4 GB ECC RAM

3 x miniPCIe + SIM

mSATA support & SATA Port

wide spread and well supported

Both are available as a bundle for around ~220 € fully fan less and silent and are easy routing 100 MBit/s
with case and PSU. And it will be able also to route 250 MBit/s at the WAN Port with ease.

How well is your board supported?
How well are the drivers are matching to that hardware?
How well it is playing together with pfSense (version 2.2.6)?

The apu2 sports an AMD GX-412TC which clocks in at 1200MHz.
While the Intel n3150 clocks in at 1600MHz, and goes up to 2080MHz with turbo.

Yep but would it do better then the APU2? It has more CPU power and thats it, perhaps it
would be better sorting the OpenVPN now, but since OpenVPN 2.4 and AES-GCM support
I would not swear on this! So I really thing there are other things similar matching but more
or better supported and running like hell. At the end of this thread I am counting together
some spare parts as an assemble, there are for sure better and stronger systems out there
but how well they are playing nice together with pfSense is the most question for me!

This is an honest question, I really wonder, because I am trying to make this exact decision myself

Each of us has his own understanding, beloved hardware or systems he´s is more or less swearing
on for sure that must not be matching or considering the parts and interested in systems other would love
to go with.

(Although I am looking at Jetway boards with mutliple NIC's, not Asus (With the cost of the extra NIC you're basically paying the same as a multi NIC board).).

Yes and no, sorry based on my lower English language skills I must take much more lines to explain something
but there are even also some strange differences and also if the hardware is based on the same SoC or CPU!
So there are J1900 and N2930 boards I hate and pfSense is causing problems with, and based on the same
CPUs or SoC, as explained in some line above, other boards will not have this failures, issues or malfunction.
And that mostly for only some bucks on top of the other hardware likes 20 € - 60 € and this is not really much
money of you can safe time and play around with your new hardware and don´t be boring about some problems.

For your 100M connectivity, APU2/2150 should be able to handle the job easily, while the APU2 board comes with dual Intel i210/211 NICs which seems to be better.

Here in Germany are only some 100 MBit/s FTTH/FTTC connections able to get for private persons
and this is one of the most used self made firewall basis because pfSense, untangle UTM and Sophos
UTM are running fine on them too. The N2930 is working for edwardwong routing nearly 1 GBit/s at
the WAN port. I don´t know about the OpenVPN speed, but according to the AES-NI support in OpenVPN
version 2.4 it could really be that the APU2 is then better, perhaps also the Intel N3050i too, but from that
I don´t know the support of it. And due of the lack of AES-NI at the N2930 I was considering the APU2 as
a better choice.

Entry Level:

APU2C4 bundle

Compex WLE200NX

Sierra Wireless MC7710 LTE

Crucial 30/60/120 GB mSATA

Jetway NF9HG-2930

2 x 4 GB DDR3-1600MHz

Ubiquiti SR71-E WLAN card

Sierra Wireless MC7710 LTE

Crucial 30/60/120 GB mSATA

Supermicro A1SRi-2358 (new)

2 x 2 GB DDR3-1600MHz ECC RAM

Samsung840 Pro SSD 80/120/240 GB

pfSense SG-2220 / SG-2440

Mid ranged:
Supermicro A1SRi-2558
Supermicro A1SRi-2758

2 x 4/8 GB DDR3-1600MHz ECC RAM

Samsung840 Pro SSD 80/120/240 GB

pfSense SG-4860 / SG-8860

Professional:

ASUS Q87T

Gigabyte Q87T

CPU support
Intel Core i7 (Haswell), Intel Core i5 (Haswell), Intel Core i3 (Haswell),
Intel Pentium G (Haswell), Intel Celeron G (Haswell), Intel Xeon E3 v3 (Haswell)

2 x 2/4/8 GB S0-DIMM DDR3-1600MHz

Intel Ethernet Server Adapter I350-T4

WiFi Atheros AR9280 half length

Crucial 30/60/120GB mSATA

Noctua NH-L9i, CPU-Kühler

pfSense C2758 1U / XG-2758

High end:

Gigabyte GA-6LISL

Intel Xeon E3-12xxv3

Intel i350 / i354 4x NIC

8/16 GB ECC DDR3 RAM

Intel SLC/MLC 120/240 SSD

pfSense XG-2758 / XG-1500

I also encountered the same problem, this is useful information to me
Thank you so much

mattlach

@lra:

@thnee:

Perhaps a PC Engines APU or APU2 Board or bundle (PSU & case & Board) would be realizing this for you.

Could you please expand on why you think the APU2 would be better? To me it seems to have much slower performance on paper?

The apu2 sports an AMD GX-412TC which clocks in at 1200MHz.
While the Intel n3150 clocks in at 1600MHz, and goes up to 2080MHz with turbo.

As for comparing OpenVPN performance, I have started using this benchmark:
openvpn --genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
Then to give the execution time in seconds a real-world meaning:
( 3200 / execution_time_seconds ) = Projected Maximum OpenVPN Performance in Mbps
For example (tested using Linux 3.2.x)…

PC Engines APU2 Quad Core AMD GX-412TC:
Execution time: 77.3 secs.
Maximum OpenVPN: 41 Mbps

Jetway NF9HG-2930 Quad Core Celeron N2930:
Execution time: 42.4 secs.
Maximum OpenVPN: 75 Mbps

So far, in my testing, this benchmark comes close to actual Maximum OpenVPN Performance measurements under optimum conditions. The projected speed should be an upper limit.

Note: The magic number of 3200 comes from summing 1 to 20000, multiply by 2 for encrypt and decrypt and by 8 bits/byte and divide by 1,000,000 for a result of Mbps

Do you really run AES256? Seems a little overkill.

If I want to know AES-128-CBC performance, can I just change it after –cipher?

Thanks,
Matt

lra

@mattlach:

If I want to know AES-128-CBC performance, can I just change it after –cipher?

Yes, simply change to –cipher aes-128-cbc , the formula stays the same.

BTW, with OpenVPN 2.4 you can also test --cipher aes-256-gcm and --cipher aes-128-gcm .

messerchmidt

would go for the quad core variant for not much more, if possible

apollo lake atom based board perhaps

pfBasic

FWIW, J3355B:

AES-256-CBC : 291.2Mbps
AES-256-GCM: 302.0Mbps

AES-128-CBC: 293.5Mbps
AES-128-GCM: 307.9Mbps


#: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.989u 0.015s 0:11.02 99.7%    819+178k 2+0io 0pf+0w
#: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-gcm
disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.596u 0.023s 0:10.66 99.5%    817+178k 2+0io 0pf+0w
#: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc
disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.902u 0.015s 0:10.99 99.2%    821+178k 2+0io 0pf+0w
#: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm
disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
10.392u 0.015s 0:10.46 99.4%    818+177k 2+0io 0pf+0w

mauroman33

@pfBasic:

FWIW, J3355B:

AES-256-CBC : 291.2Mbps
AES-256-GCM: 302.0Mbps

AES-128-CBC: 293.5Mbps
AES-128-GCM: 307.9Mbps

Thanks for the useful information. I'm going to update the tread here:
https://forum.pfsense.org/index.php?topic=115673.0

Rango

@lra:

@mauroman33, Thanks for the follow-up post.

It seems the simple OpenVPN benchmark formula referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743

gives a reasonable base-line reference. I too have found the actual tested speed can be 5-20 % faster than the benchmark formula, and for some it is right on target.

Nothing beats an actual real-world test, but a quick CLI base-line test can be useful.

@mauroman33 did you test 256 or 128 cipher? did you have AES-NI active in pfsense when doing this test? Do you know what version of pfsense this was?

I have Celeron N3150 with AES-NI hardware accelerators. I'm little disappointed if 115Mbps is cap of this processor. It was said it can do 300Mbps on single core.

My ISP connection is 180Mbps i was hoping 10% less then my ISP connection so 160Mbs?

Can you test or anyone else with this buffer code along with hardware accelerators on for N3150?

https://forum.pfsense.org/index.php?topic=128698.msg714619#msg714619

fast-io
sndbuf 524288
rcvbuf 524288

I don't think one can estimate output when hardware accelerators are on as we don't know what factor that changes things. I'm assuming estimates are based on CPU cycles alone no? I spent ~$350 on this box 2 yrs ago and for it to come short it's bit disappointing.

Anyone have any suggestions on newer cheap <$200 NUCs with CPU that has AES-NI accelerators instructions. I'm still keeping hope i can max out my ISP connection with N3150? Maybe those buffer codes?

I will soon have vpn and will do tests myself even with those buffer codes. They seemed to speed things up quite a bit.

Is GCM suppose to be faster more secure then CBC? What's the deal-eo with that?