Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    AES-NI and OpenVPN?

    Scheduled Pinned Locked Moved Hardware
    45 Posts 5 Posters 4.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N8LBVN
      N8LBV @stephenw10
      last edited by

      @stephenw10 Hi SW,

      Three places:
      1- System\Advanced\Misc: Cryptographic & Thermal Hardware\AES-NI CPU-hased hardware accelleration can be enabled or set to "none" here.

      2- OpenVPN\Servers\your-server\Crytographic Settings\Hardware Crypto\IntelRDRAND engine - RAND can be set or set to "no hardware crypto"

      3- OpenVPN\Clients\your-client\Crytographic Settings\Hardware Crypto\IntelRDRAND engine - RAND can be set or set to "no hardware crypto"

      I feel more like I do now.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Yup, none of those will prevent OpenSSL using AES-NI instructions if it sees a CPU that supports them.

        N8LBVN 1 Reply Last reply Reply Quote 0
        • N8LBVN
          N8LBV @stephenw10
          last edited by

          @stephenw10 of course you know what my next questions is right?
          Is this a future feature to be able to turn them off?
          Or the jerk question: Why are those settings there? :) :)
          I'm learning to be less of a jerk :) as I get older?

          I feel more like I do now.

          N8LBVN 1 Reply Last reply Reply Quote 0
          • N8LBVN
            N8LBV @N8LBV
            last edited by

            @n8lbv I think the proper answer is "those settings are there to confuse jerks".

            I feel more like I do now.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              The confusing part here is that AES-NI is not really a hardware crypto accelerator which is what those settings are for. It's just an additional instruction set some CPUs support.
              The AES-NI kernel module exists to allow code to use it via the BSD crypto framework as though it was a hardware accelerator (like QAT) but OpenSSL does not need that.

              N8LBVN 1 Reply Last reply Reply Quote 1
              • N8LBVN
                N8LBV @stephenw10
                last edited by

                @stephenw10 My take on it or way of explaining it if I had to.

                A CPU that has AES-NI does the computations "faster" or more efficiently (in less clock cycles)
                That a CPU that does not have it and therefore would need to do part of it in software alone.
                So you'd expect to see less CPU utilization compared to the same CPU not using this part of it's instructionset. (more computations per MHz).

                I feel more like I do now.

                Dobby_D 1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Yes, that's true. It can compelete the same computations in less cycles using the AES specific instructions. So you would see less CPU usage for the same encrypted throughput.

                  N8LBVN 1 Reply Last reply Reply Quote 0
                  • N8LBVN
                    N8LBV @stephenw10
                    last edited by N8LBV

                    @stephenw10 Yep yep-
                    And if you can turn off AES-NI in advanced settings and you can turn it off (or turn it on) as default is off- for any openvpn server or client instance one would expect that openvpn would not be using it when set this way, and would expect it is now doing the calculations in software and not using the build-in cpu AES-NI instructions.
                    Thus why are these settings present if they do not turn it off or on like they imply?

                    I feel more like I do now.

                    N8LBVN 1 Reply Last reply Reply Quote 0
                    • N8LBVN
                      N8LBV @N8LBV
                      last edited by N8LBV

                      @n8lbv A CPU does not magically "detect" if you're using AES.
                      you have to send it those special or AES commands (or not) and do them the longer way.
                      I could see the software "detecting" if an AES-NI CPU is present and then choosing to use these special commands if it is present "automatically".
                      Or NOT send the CPU these special commands if you choose a setting in software not to and do them the long way (with no AES-NI instructions) to the CPU.

                      I feel more like I do now.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        The setting in System > Advanced only loads the AES-NI kernel module. Or the QAT kernel module, or potentially some other hardware device.
                        Importantly disabling it there does not unload the module until you reboot or manually kldunload it.

                        OpenSSL does exactly that. If it detects it's running on a CPU that supports AES-NI instructions it will use them directly.

                        Steve

                        N8LBVN 2 Replies Last reply Reply Quote 0
                        • N8LBVN
                          N8LBV @stephenw10
                          last edited by

                          @stephenw10 I did make sure to reboot between comparisons.
                          -Steve

                          I feel more like I do now.

                          1 Reply Last reply Reply Quote 0
                          • N8LBVN
                            N8LBV @stephenw10
                            last edited by

                            @stephenw10 With the module unloaded OpenSSL will still use AES-NI.
                            So unloading the module .. OpenSSL will still use AES-NI without the module?
                            So AES-NI can be (or will be) used with our without the module?
                            And does the dropdown within OpenVPN client or server only apply to the module?
                            :)

                            I feel more like I do now.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Yes OpenSSL will use AES-NI with or without the kernel module.

                              The dropdown in the OpenVPN config applies the OpenSSL 'engine' used and does nothing for AES-NI in current pfSense versions.

                              The only way to actually test the difference AES-NI makes would be to disable it in the BIOS AFAIK.

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • Dobby_D
                                Dobby_ @N8LBV
                                last edited by

                                @n8lbv

                                Intel Whitepaper for peoples who writing code
                                Breakthrough AES Performance with Intel AES New Instructions

                                A study of AES-NI acceleration using LibreSSL, OpenSSL
                                AES-NI SSL Performance

                                Different other points counting also and matching too
                                Advanced Encryption Standard New Instructions
                                (AES-NI) Analysis: Security, Performance, and
                                Power Consumption

                                A discussion on Reddit about AES-NI
                                Please watch out for the comment from "jra_samba_org"

                                The Algorithm itself together with AES-NI
                                Rochester Institute of Technology

                                AES-NI performance in other applications
                                A LOOK AT THE PERFORMANCE IMPACT OF HARDWARE-ACCELERATED AES

                                Another different type of SSL usage, with Intel AES-NI
                                Implementation and Performance of AES-NI in CyaSSL

                                Under different aspects, regarding other points and after
                                all reading over, it is a let us say raw frame outspeak but
                                all in all AES-NI is speeding up x8 till x10 the entire usage
                                of AES against pure done in software.

                                #~. @Dobby

                                Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                                PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                                PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                                N8LBVN 1 Reply Last reply Reply Quote 0
                                • N8LBVN
                                  N8LBV @Dobby_
                                  last edited by

                                  I still am not clearly understanding what we get in terms of AES-NI If the module is not loaded
                                  and AES-NI is not selected in the client or server itself.
                                  How is it different if the module is loaded or if the module is not loaded?
                                  And how is it different if it is not selected in the client or server itself?
                                  I have tried every combination the above and cannot see any noticeable difference in CPU usage or throughput.
                                  We are connected at wire speed through two PFSense systems.
                                  Speed test is right at 1Gbps when tested without AES/OpenVPN (two NATs)
                                  CPU never goes over 32%
                                  CPU is an older gen2 i5.
                                  gen2.jpg

                                  I will have to have more fun and testing with this.

                                  I feel more like I do now.

                                  Dobby_D 1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    I would not expect to any difference there because OpenSSL will use AES-NI instructions if they are available whatever setting you have there.

                                    To test that you would need to disable AES-NI in the BIOS to prevent OpenSSL seeing it.

                                    The only other test you could do here is using DCO in OpenVPN. Because that does all the encryption in kernel mode it can use the module so you would potentially see a difference. It can use the QAT module there for example and that gives a significant performance bump.

                                    1 Reply Last reply Reply Quote 1
                                    • Dobby_D
                                      Dobby_ @N8LBV
                                      last edited by

                                      @n8lbv said in AES-NI and OpenVPN?:

                                      The purpose of AES-NI is to improve the speed of applications performing encryption and decryption using the Advanced Encryption Standard (AES) ...

                                      TrueCrypt without using AES-NI ~227 MB/s
                                      TrueCrypt with using AES-NI ~1.8 GBit/s

                                      This was the software usage of AES-NI and now let us
                                      have a look on the transport encryption such VPN.

                                      In real life it would be coming on top of this
                                      more points that counting in or on top of it.

                                      • OpenSSL or LibreSSL
                                      • Implementation of the software
                                      • VPN method you are using (IPSec)

                                      Can you set up an IPSec tunnel using AES-GCM-128/256?
                                      And measure again.

                                      I still am not clearly understanding what we get in
                                      terms of AES-NI If the module is not loaded and
                                      AES-NI is not selected in the client or server itself.

                                      The code or software is benefitting from that AES-NI.

                                      How is it different if the module is loaded or if the
                                      module is not loaded?

                                      If you don´t see any performance gain or a better or
                                      higher throughput, it indicates to you that there will
                                      be nothing happen, based on that AES-NI function.

                                      In older days PC Engines Alix board were very popular
                                      for setting up pfSense or mOnOwall on it, VPN was at
                                      14.2 MBit/s throughput and together with a small
                                      miniPCI card from Soekris (HiFn chip) you were able
                                      to gain the throughput to something around 42.3
                                      MBit/s so this was nearly three time faster and you
                                      were able to see the benefit. As today CPU becomes
                                      more strong and fast that this AES-NI instructions perhaps
                                      will be not able to realize in all cases and for or from all users, because the Internet speed was also increasing much, but with a Internet connection of 50 MBit/s you
                                      may need or wish to scratch out all you can get, more
                                      then others with a 1 GBit/s line. Where it makes no
                                      difference they get out 100 MBit/s more or less.

                                      And how is it different if it is not selected in the client or > server itself?

                                      What you were able to see at the IPSec tunnel pushing
                                      a 300 MB file through?

                                      I have tried every combination the above and cannot
                                      see any noticeable difference in CPU usage or throughput.

                                      Perhaps your CPU is strong enough, so be happy with it.

                                      We are connected at wire speed through two
                                      PFSense systems. Speed test is right at 1Gbps
                                      when tested without AES/OpenVPN (two NATs)
                                      CPU never goes over 32%
                                      CPU is an older gen2 i5.

                                      Perhaps, from my point of view, you were not able to saturate the entire internet or vpn line and therefor
                                      you will be not able to see any significant gain here.

                                      #~. @Dobby

                                      Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                                      PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                                      PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                                      N8LBVN 1 Reply Last reply Reply Quote 0
                                      • N8LBVN
                                        N8LBV @Dobby_
                                        last edited by N8LBV

                                        @dobby_ Fun stuff.
                                        I get the full 1Gbps if I hit the local speedtest server through the two-NAT two PFsense systems route.
                                        If I hit it through the Openvpn tunnel between the same two systems I am seeing the speed and CPU utilization you see in that screen capture.

                                        Thank you for answering all of my questions and I'm a newb at much of this.
                                        I'm still failing to understand why the toggleable options are there both in system\advanced
                                        and within the openvpn client or server items.
                                        Why are there they to turn on/off if it is going to go ahead and use AES-NI anyware regardless of the toggles?
                                        Are there two separate sets of AES-NI software?
                                        One (kernel level) in the module and another in userspace(default if you do not load the module?

                                        And what about the other toggle within the client or server ?
                                        Does that just tell the client or server to USE the module or to NOT to use it?

                                        Then if you USE the module does it stop using the _not module AES-NI in userspace?
                                        Or does it use a combination of BOTH versus (one or the other) when it is loaded as instructed from system\advanced?

                                        Thanks for being patient with me on this and for all of the information.
                                        -Steve

                                        I feel more like I do now.

                                        Dobby_D 1 Reply Last reply Reply Quote 0
                                        • Dobby_D
                                          Dobby_ @N8LBV
                                          last edited by

                                          @n8lbv said in AES-NI and OpenVPN?:

                                          I get the full 1Gbps if I hit the local speedtest server through the two-NAT two PFsense systems route.

                                          You may be then the lucky guy were the VPN will be
                                          done in raw MHz/GHz, and all is fine for you.

                                          But there are also other peoples out and they don`t use OPNvpn and/or WireGuard. Or they have special needs,
                                          an other use case and so on.

                                          If I hit it through the Openvpn tunnel between the same two systems I am seeing the speed and CPU utilization you see in that screen capture.

                                          May be but others have perhaps the need for IPSec and/or
                                          they are getting a gain from that AES-NI, if it is present
                                          and you have no other things up there such Intel QAT
                                          you have not the need to change, but if it is so you
                                          will be also the lucky one that is able to change and let
                                          us say you say please use QAT or an HiFn Chip or what ever is in place. So in my eyes you are looking at your situation and your hardware and all is fine.

                                          Thank you for answering all of my questions and I'm a newb at much of this.

                                          It is more to try out turn your thinking into another
                                          direction, because;

                                          • you see only your situation
                                          • you see only your equipment

                                          So you can´t understand it, let us imagine you are a company and given by law or company rules or plain insurance rules you will be pressed as the admin of the network to use something such IPSec, or other hardware
                                          in the company all over the world is not able to use OpnVPN and/or WireGuard, o what will you do now?

                                          And for that clients and peoples or plain users it is a nice
                                          catch or nice to have thing, that pfSense is offering to them the ability to change such things fast, to fit their
                                          needs.

                                          I'm still failing to understand why the toggleable options are there both in system\advanced
                                          and within the openvpn client or server items.

                                          Your CPU comes with AES-NI, but what will be when;

                                          • Your CPU is also sorted with Intel QAT and you want
                                            to change it now? You do you want to realize it then?
                                          • Your hole amount of VPN road Warriors will be hard increasing and you may want to change the chip that
                                            or instance that should be used for encryption, how
                                            you want to do this then?

                                          Why are there they to turn on/off if it is going to go ahead and use AES-NI anyware regardless of the toggles?

                                          What is when at one day other things such QAT comes
                                          out and you will be not able to change or say use this
                                          or use that?

                                          Are there two separate sets of AES-NI software?

                                          It depends often on other points and more points.
                                          Mostly it is only the way how it is used.
                                          case one:
                                          AES-NI is present in the CPU
                                          crypto dev system from the os uses a driver
                                          and the driver uses the AES-NI instructions set of the cpu
                                          case two:
                                          the software uses the AES-NI directly

                                          But in real and on top this it comes other benefits by side
                                          like better elliptic curve's, better cycling and so on and so on, because it is all under the heat not visible to see for
                                          us customers we may thinking why is it there.

                                          One (kernel level) in the module and another in userspace(default if you do not load the module?
                                          And what about the other toggle within the client or server ?

                                          In short it can be answered if you are not needing it makes it not obsolete to be present and let us imagine you must now feed some more 1 GBit/s lines and doing a backup
                                          over them, what then?

                                          Does that just tell the client or server to USE the module or to NOT to use it?

                                          I think it is more who and when it will be used;

                                          • OpenSSL will use it even if it is present
                                          • The software only if it is enabled
                                          • And the other benefits from it may be not known from us
                                            because we a re not the software coder, but anyway I will be more lucky with the situation that I am able to change
                                            something, against there is no way to do so.

                                          Then if you USE the module does it stop using the _not module AES-NI in userspace?

                                          Since the change from FreeBSD 12.3 to version 14.0
                                          there will be more changes as we end users would
                                          knowing about. Who knows how things will be
                                          changed and in wich steps this process will be done.

                                          Or does it use a combination of BOTH versus (one or the other) when it is loaded as instructed from system\advanced?
                                          Thanks for being patient with me on this and for all of the information.

                                          Let us say you are an engineer and you do research for
                                          let us say 1 million $ and now you want to go to the
                                          patent office to get a patent, and then this fancy new OpenVPN or WireGuard comes out and you use it, but
                                          someone was steeling your ideas because there where
                                          a not found whatever Inside, you are done!

                                          Other countries will be hard forbit the usage of hardware
                                          accelerated crypto work and now you can show them it
                                          is all done in pure CPU work with an unchecked AES-NI
                                          usage, all is then fine for you!

                                          #~. @Dobby

                                          Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                                          PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                                          PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            @n8lbv said in AES-NI and OpenVPN?:

                                            I'm still failing to understand why the toggleable options are there both in system\advanced
                                            and within the openvpn client or server items.
                                            Why are there they to turn on/off if it is going to go ahead and use AES-NI anyware regardless of the toggles?

                                            The crypto hardware option in System > Advanced it to choose which kernel module to load. Those modules are used by the bsd crypto framework for kernel mode crypto operations. So that's IPSec or OpenVPN in DCO mode.

                                            The hardware crypto option in the OpenVPN config is the equivalent of specifiying the 'engine' for OpenSSL operations. In current pfSense versions OpenSSL can't use the AES-NI module for that (and doesn't need to anyway). It exists only for much older crypto offload hardware that may still be use. Most users should not select anything there.

                                            If you're looking for the best OpenVPN throughput the best option currently, by some way, is to use QAT supported hardware and run OpenVPN in DCO mode.

                                            Steve

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.