AES-NI performance

VAMike

Nope - I know that openvpn is single threaded in that each instance gets a single thread.

What I'm wondering is do multiple instances of openvpn, which result in multiple openvpn threads each also result in multiple threads of openssl?

the "openssl" command line utility is single threaded unless you pass -multi (which produces an output which is pretty meaningless and hard to compare across platforms, just don't do that). The ssl library is single threaded with a process. If you run multiple instances of openvpn you are running multiple independent processes, not threads, and can utilize different cores with each process.

You didn't answer whether the cryptodev stuff was disabled in the gui.

kejianshi

Yes - cryptodev is disabled and AES-NI is enabled. The pfsense VM gets about the same scores at the physical machine also, which is pretty nice to see.

I was only in the box to test why its getting random crashes, so I was just playing around and running process to stress the machine to wait for the crash.

And it died… I think the power supply is failing. Going to have to get that replaced before I can further study the mysteries of AES-NI on the AMD 8150.

jazzl0ver

Hi all,

Version 	2.4.3-RELEASE-p1 (amd64) 
CPU Type 	Intel(R) Xeon(R) CPU X5650 @ 2.67GHz 24 CPUs: 2 package(s) x 6 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)

I performed several tests with the following commands:

openssl speed -evp aes-128-cbc -elapsed
openssl speed -evp aes-128-gcm -elapsed

with different Cryptographic Hardware and Kernel PTI settings (+PTI means Kernel PTI is enabled):

+------------------------+--------------------------+--------------------------+--------------+--------------+-----------------+-----------------+
|                        | AES-NI + Cryptodev + PTI | AES-NI + Cryptodev - PTI | AES-NI + PTI | AES-NI - PTI | Cryptodev + PTI | Cryptodev - PTI |
+------------------------+--------------------------+--------------------------+--------------+--------------+-----------------+-----------------+
| aes-128-cbc 16 bytes   |                     7189 |                     7794 |       612843 |       612249 |          605915 |          588186 |
| aes-128-cbc 8192 bytes |                   568785 |                   591544 |       765053 |       763943 |          763748 |          764321 |
| aes-128-gcm 16 bytes   |                   243029 |                   243885 |       238457 |       251084 |          250158 |          229928 |
| aes-128-gcm 8192 bytes |                   942211 |                   943865 |       944693 |       943185 |          944543 |          946034 |
+------------------------+--------------------------+--------------------------+--------------+--------------+-----------------+-----------------+

The router was rebooted after changing each setting.

Can anybody explain the very small values in aes-128-cbc 16 bytes test as well as remarkably smaller values in aes-128-cbc 8192 bytes test when both AES-NI and Cryptodev enabled?

Thanks in advance!

stephenw10

I suggest that when both are enabled the AES-NI module registers itself as a crypto device in the framework for AES-CBC and openssl tries to use it. That results in massive additional switching especially for small packets.
Though there is a load of misinformation surrounding this and I have managed to get it wrong before!

Perhaps more interesting is that you seem to be seeing a better result with PTI enabled in some cases there. I have no explanation for that.

Steve

jazzl0ver

@stephenw10 , thanks for your prompt reply!

What is the best Cryptographic Hardware setting then? The router mainly serves as a proxy (haproxy) and openvpn server.
And why does the option "AES-NI and Cryptodev" ever exist if it degrades the performance?

Regarding better results with PTI enabled - they look more like a measurement error.

stephenw10

Cryptodev exists because there are other cryptographic accelerators in use on other hardware. Though almost everything easily available is now relatively ancient and surpassed by general software encryption on modern CPUs.
AES-NI exists because some code was not written/compiled to the AES instructions directly and it provides a way to access that.

Personally I use AES-NI only.

Steve

VAMike

@stephenw10 said in AES-NI performance:

Perhaps more interesting is that you seem to be seeing a better result with PTI enabled in some cases there. I have no explanation for that.

Run-to-run variation. The affects of PTI should be minimal for this sort of workload. Note that the AES-NI and the cryptodev columns are effectively identical (they're executing the same code) yet they have significant differences in some cases--which are just testing artifacts. Likewise, the AES-GCM tests should be identical in all three columns PTI/non-PTI, but there's noise between runs and not enough samples to average. But mostly the only significant result is the performance of aes-ni cbc+cryptodev--don't do that!

@jazzl0ver said in AES-NI performance:

And why does the option "AES-NI and Cryptodev" ever exist if it degrades the performance?

Bad UI design, basically. And a lot of really misinformed people running tests which confused a lot of other people.

stephenw10

Re-reading this thread is.... painful.

Steve

jazzl0ver

Just to confirm: if I leave AES-NI only in Cryptographic Hardware, won't this affect OpenVPN performance which Hardware Crypto setting is BSD cryptodev engine? Or I'll have to change it to No Hardware Crypto Acceleration (since it will still utilize internal OpenSSL's AES-NI code)?

stephenw10

It shouldn't make any difference since the cryptodev module will not be loaded. I would set that no No Hardware Crypto Acceleration there anyway though.

Steve

jazzl0ver

Thanks @stephenw10 ! I appreciate your help!