AES-NI and OpenVPN?

N8LBV

Is it expected to be making a difference?
In my test I do not see a difference in CPU utilization with CPU hardware accel turned on off.
For an openvpn client and server.
I am certain that I am enabling and disabling it. + rebooting client and server to be sure all changes are in effect.
Testing while passing ~100mbps of VPN traffic in both directions and monitoring CPU
Utilization.

This is a pretty basic test but is where I am starting off.
Testing with CE 2.6.0 release and with CE 2.7.latest snapshots.
Intel CPU and not a virtual machine.
Any thoughts expectations experiences or other tests?

CPU Type Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
Current: 3502 MHz, Max: 2900 MHz
4 CPUs: 1 package(s) x 2 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No
Hardware crypto AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS

2.7.0-DEVELOPMENT (amd64)
built on Thu Apr 20 06:05:16 UTC 2023
FreeBSD 14.0-CURRENT

Max CPU usage during data transfer through the tunnel.
16%

Dobby_

@n8lbv

Hello, where your results? Reading your post may be
trigger us to await some results what was pfSense 2.6
able to deliver and what was coming out together
with pfSense 2.7.

This is a pretty basic test but is where I am starting off.

From where to where your test wehre running?

Testing with CE 2.6.0 release and with CE 2.7.latest snapshots.

pfSense to pfSense OPNvpn using, ok, but what numbers
or throughput you were seeing at witch side?

Intel CPU and not a virtual machine.

OK

Any thoughts expectations experiences or other tests?

Fairly we should know the entire setup and the hardware on both sides to do so!

CPU Type Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
Current: 3502 MHz, Max: 2900 MHz
4 CPUs: 1 package(s) x 2 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No
Hardware crypto AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS
2.7.0-DEVELOPMENT (amd64)
built on Thu Apr 20 06:05:16 UTC 2023
FreeBSD 14.0-CURRENT

And what is with the other both ends from the OPNvpn tunnel? We expect to see something from you like this;

pfSense 2.6 OPNvpn -- to -- ? with xyz MBit/s throughput

pfSense 2.7 OPNvpn -- to -- ? with xyz MBIt/s throughput

Max CPU usage during data transfer through the tunnel.
16%

What LAN ports where in usage?
Intel i225, Intel i226, Intel?
What is your internet connection speed?
250 MBit/s up, 100 MBit/s down or both vpn ends speed is?
How many queues where set up?
Is PPPoE in usage on both ends or one end of the VPN tunnel?
What was the max queue amount, length and size?
What eth port tunings where done?
What other packets were installed?
And/or running byside of this test such IDS, pfBlocker-ng,..?
Was the 16% for all CPU cores, or only one of them?

stephenw10

How are you disabling it? You would likely have to disable the CPU reporting the support in the BIOS. OpenSSL (and hence OpenVPN) will use the AES-NI instructions directly if your CPU supports them regardless of whether or not the kernel module is loaded.
The only exception to that is if you are using DCO in which case it can take advantage of the crypto framework which means QAT or AES-NI via the module.

Steve

N8LBV

@dobby_ Hi Dobby_
I am not ready to go into details yet.
I only wanted to ask a very basic question as to if openvpn uses AES-NI
And if a significant reduction in CPU usage should be expected on basic Intel hardware.
--Desktop Intel i5/i7CPU gen3,4,6,7,8,9 + Intel on motherboard Nic.
I am testing on different systems.

When comparing CPU utilization (monitored on PFSense Dashboard page)
I do not see any difference in CPU usage while passing ~100Mbps through the OpenVPN tunnel. when "turing off" Intel CPU hardware crypto. or having it "turned on".

Not (yet) ready to dive into more detail than that.
But wanted to know if is supposed top help openvpn/AES significantly.

Thanks.

N8LBV

@stephenw10 Hi SW,

Three places:
1- System\Advanced\Misc: Cryptographic & Thermal Hardware\AES-NI CPU-hased hardware accelleration can be enabled or set to "none" here.

2- OpenVPN\Servers\your-server\Crytographic Settings\Hardware Crypto\IntelRDRAND engine - RAND can be set or set to "no hardware crypto"

3- OpenVPN\Clients\your-client\Crytographic Settings\Hardware Crypto\IntelRDRAND engine - RAND can be set or set to "no hardware crypto"

stephenw10

Yup, none of those will prevent OpenSSL using AES-NI instructions if it sees a CPU that supports them.

N8LBV

@stephenw10 of course you know what my next questions is right?
Is this a future feature to be able to turn them off?
Or the jerk question: Why are those settings there? :) :)
I'm learning to be less of a jerk :) as I get older?

N8LBV

@n8lbv I think the proper answer is "those settings are there to confuse jerks".

stephenw10

The confusing part here is that AES-NI is not really a hardware crypto accelerator which is what those settings are for. It's just an additional instruction set some CPUs support.
The AES-NI kernel module exists to allow code to use it via the BSD crypto framework as though it was a hardware accelerator (like QAT) but OpenSSL does not need that.

N8LBV

@stephenw10 My take on it or way of explaining it if I had to.

A CPU that has AES-NI does the computations "faster" or more efficiently (in less clock cycles)
That a CPU that does not have it and therefore would need to do part of it in software alone.
So you'd expect to see less CPU utilization compared to the same CPU not using this part of it's instructionset. (more computations per MHz).

stephenw10

Yes, that's true. It can compelete the same computations in less cycles using the AES specific instructions. So you would see less CPU usage for the same encrypted throughput.

N8LBV

@stephenw10 Yep yep-
And if you can turn off AES-NI in advanced settings and you can turn it off (or turn it on) as default is off- for any openvpn server or client instance one would expect that openvpn would not be using it when set this way, and would expect it is now doing the calculations in software and not using the build-in cpu AES-NI instructions.
Thus why are these settings present if they do not turn it off or on like they imply?

N8LBV

@n8lbv A CPU does not magically "detect" if you're using AES.
you have to send it those special or AES commands (or not) and do them the longer way.
I could see the software "detecting" if an AES-NI CPU is present and then choosing to use these special commands if it is present "automatically".
Or NOT send the CPU these special commands if you choose a setting in software not to and do them the long way (with no AES-NI instructions) to the CPU.

stephenw10

The setting in System > Advanced only loads the AES-NI kernel module. Or the QAT kernel module, or potentially some other hardware device.
Importantly disabling it there does not unload the module until you reboot or manually kldunload it.

OpenSSL does exactly that. If it detects it's running on a CPU that supports AES-NI instructions it will use them directly.

Steve

N8LBV

@stephenw10 I did make sure to reboot between comparisons.
-Steve

N8LBV

@stephenw10 With the module unloaded OpenSSL will still use AES-NI.
So unloading the module .. OpenSSL will still use AES-NI without the module?
So AES-NI can be (or will be) used with our without the module?
And does the dropdown within OpenVPN client or server only apply to the module?
:)

stephenw10

Yes OpenSSL will use AES-NI with or without the kernel module.

The dropdown in the OpenVPN config applies the OpenSSL 'engine' used and does nothing for AES-NI in current pfSense versions.

The only way to actually test the difference AES-NI makes would be to disable it in the BIOS AFAIK.

Steve

Dobby_

@n8lbv

Intel Whitepaper for peoples who writing code
Breakthrough AES Performance with Intel AES New Instructions

A study of AES-NI acceleration using LibreSSL, OpenSSL
AES-NI SSL Performance

Different other points counting also and matching too
Advanced Encryption Standard New Instructions
(AES-NI) Analysis: Security, Performance, and
Power Consumption

A discussion on Reddit about AES-NI
Please watch out for the comment from "jra_samba_org"

The Algorithm itself together with AES-NI
Rochester Institute of Technology

AES-NI performance in other applications
A LOOK AT THE PERFORMANCE IMPACT OF HARDWARE-ACCELERATED AES

Another different type of SSL usage, with Intel AES-NI
Implementation and Performance of AES-NI in CyaSSL

Under different aspects, regarding other points and after
all reading over, it is a let us say raw frame outspeak but
all in all AES-NI is speeding up x8 till x10 the entire usage
of AES against pure done in software.

N8LBV

I still am not clearly understanding what we get in terms of AES-NI If the module is not loaded
and AES-NI is not selected in the client or server itself.
How is it different if the module is loaded or if the module is not loaded?
And how is it different if it is not selected in the client or server itself?
I have tried every combination the above and cannot see any noticeable difference in CPU usage or throughput.
We are connected at wire speed through two PFSense systems.
Speed test is right at 1Gbps when tested without AES/OpenVPN (two NATs)
CPU never goes over 32%
CPU is an older gen2 i5.

I will have to have more fun and testing with this.

stephenw10

I would not expect to any difference there because OpenSSL will use AES-NI instructions if they are available whatever setting you have there.

To test that you would need to disable AES-NI in the BIOS to prevent OpenSSL seeing it.

The only other test you could do here is using DCO in OpenVPN. Because that does all the encryption in kernel mode it can use the module so you would potentially see a difference. It can use the QAT module there for example and that gives a significant performance bump.