Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Aes-ni not working?

    2.1 Snapshot Feedback and Problems - RETIRED
    9
    40
    28.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimpJ
      jimp Rebel Alliance Developer Netgate
      last edited by

      Is aesni.ko loaded during those tests? (check the output of kldstat)

      I would expect to see at least AES-128-CBC in the cryptodev list if it attached, but then again, some others have reported that OpenSSL 1.0.1 did use AES-NI but didn't ever report it as being present, so it may take some more speed tests to tell for sure…

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      1 Reply Last reply Reply Quote 0
      • M
        miloman
        last edited by

        i entered the commands in the shell of a fresh image i just bootet up. i haven't configured/enabled anything at all.

        if i enter the command "kldload aesni" i get this output:
        padlock0: No ACE support
        aesni0: AES-CBC,AES-XTS on motherboard

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          Does that openssl engine output change after having run the kldload?

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • M
            miloman
            last edited by

            yes…

            /usr/bin/openssl engine -t -c
            (cryptodev) BSD cryptodev engine
            [RSA, RSA, DH, [b]AES-128-CBC]
              [available]
            (padlock) VIA PadLock (no-RNG, no-ACE)
              [unavailable]
            (dynamic) Dynamic engine loading support
              [unavailable]

            /usr/local/bin/openssl engine -t -c
            (cryptodev) BSD cryptodev engine
            [RSA, RSA, DH,[b] AES-128-CBC, AES-192-CBC, AES-256-CBC]
              [available]
            (rdrand) Intel RDRAND engine
            [RAND]
              [available]
            (dynamic) Dynamic engine loading support
              [unavailable]
            (padlock) VIA PadLock: not supported
              [unavailable]

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              ok, great.

              One more thing if you have some time:

              1. Reboot so aes-ni is not loaded.
              2. Run the following in order:

              Test speed before

              /usr/bin/openssl speed -evp aes-128-cbc -elapsed
              /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed

              Load AES-NI

              kldload aesni

              Test OpenSSL with default engine

              /usr/bin/openssl speed -evp aes-128-cbc -elapsed
              /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed

              Test OpenSSL with cryptodev engine

              /usr/bin/openssl speed -evp aes-128-cbc -elapsed -engine cryptodev
              /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed -engine cryptodev

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              1 Reply Last reply Reply Quote 0
              • M
                miloman
                last edited by

                heres your wall of text. :)

                [2.1-BETA1][admin@pfSense.localdomain]/root(1): /usr/bin/openssl speed -evp aes-128-cbc -elapsed
                You have chosen to measure elapsed time instead of user CPU time.
                To get the most accurate results, try to run this
                program when this computer is idle.
                Doing aes-128-cbc for 3s on 16 size blocks: 18546805 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 5035121 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 1289095 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 1024 size blocks: 325137 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 40722 aes-128-cbc's in 3.00s
                OpenSSL 0.9.8q 2 Dec 2010
                built on: date not available
                options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
                compiler: cc
                available timing options: USE_TOD HZ=128 [sysconf value]
                timing function used: gettimeofday
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc      98891.21k  107382.09k  109967.84k  110944.78k  111161.64k

                [2.1-BETA1][admin@pfSense.localdomain]/root(2): /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed
                You have chosen to measure elapsed time instead of user CPU time.
                Doing aes-128-cbc for 3s on 16 size blocks: 108688414 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 28926457 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 7348512 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 1024 size blocks: 1844550 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 230842 aes-128-cbc's in 3.00s
                OpenSSL 1.0.1c 10 May 2012
                built on: Sun Jan 27 13:05:44 EST 2013
                options:bn(64,32) md2(int) rc4(8x,mmx) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx)
                compiler: cc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,–noexecstack -DL_ENDIAN -DTERMIOS -O3 -fomit-frame-pointer -Wall -O2 -pipe -fno-strict-aliasing -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc    579671.54k  617097.75k  627073.02k  629606.40k  630352.55k

                [2.1-BETA1][admin@pfSense.localdomain]/root(3): kldload aesni

                [2.1-BETA1][admin@pfSense.localdomain]/root(4): /usr/bin/openssl speed -evp aes-128-cbc -elapsed
                You have chosen to measure elapsed time instead of user CPU time.
                To get the most accurate results, try to run this
                program when this computer is idle.
                Doing aes-128-cbc for 3s on 16 size blocks: 2725774 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 2507908 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 1925032 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 1024 size blocks: 1029235 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 147766 aes-128-cbc's in 3.00s
                OpenSSL 0.9.8q 2 Dec 2010
                built on: date not available
                options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
                compiler: cc
                available timing options: USE_TOD HZ=128 [sysconf value]
                timing function used: gettimeofday
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc      14535.69k    53485.26k  164217.56k  351201.58k  403372.36k

                [2.1-BETA1][admin@pfSense.localdomain]/root(5): /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed
                You have chosen to measure elapsed time instead of user CPU time.
                Doing aes-128-cbc for 3s on 16 size blocks: 2719290 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 2505062 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 1919653 aes-128-cbc's in 3.01s
                Doing aes-128-cbc for 3s on 1024 size blocks: 1028277 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 147809 aes-128-cbc's in 3.00s
                OpenSSL 1.0.1c 10 May 2012
                built on: Sun Jan 27 13:05:44 EST 2013
                options:bn(64,32) md2(int) rc4(8x,mmx) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx)
                compiler: cc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,–noexecstack -DL_ENDIAN -DTERMIOS -O3 -fomit-frame-pointer -Wall -O2 -pipe -fno-strict-aliasing -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc      14502.88k    53441.32k  163384.91k  350985.22k  403617.11k

                [2.1-BETA1][admin@pfSense.localdomain]/root(6): /usr/bin/openssl speed -evp aes-128-cbc -elapsed -engine cryptodev
                engine "cryptodev" set.
                You have chosen to measure elapsed time instead of user CPU time.
                To get the most accurate results, try to run this
                program when this computer is idle.
                Doing aes-128-cbc for 3s on 16 size blocks: 2721627 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 2516799 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 1926157 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 1024 size blocks: 1029088 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 147941 aes-128-cbc's in 3.00s
                OpenSSL 0.9.8q 2 Dec 2010
                built on: date not available
                options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
                compiler: cc
                available timing options: USE_TOD HZ=128 [sysconf value]
                timing function used: gettimeofday
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc      14514.45k    53674.88k  164313.53k  351151.19k  403847.11k

                [2.1-BETA1][admin@pfSense.localdomain]/root(7): /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed -engine cryptodev
                engine "cryptodev" set.
                You have chosen to measure elapsed time instead of user CPU time.
                Doing aes-128-cbc for 3s on 16 size blocks: 2733266 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 64 size blocks: 2512115 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 256 size blocks: 1928735 aes-128-cbc's in 3.01s
                Doing aes-128-cbc for 3s on 1024 size blocks: 1031083 aes-128-cbc's in 3.00s
                Doing aes-128-cbc for 3s on 8192 size blocks: 147874 aes-128-cbc's in 3.00s
                OpenSSL 1.0.1c 10 May 2012
                built on: Sun Jan 27 13:05:44 EST 2013
                options:bn(64,32) md2(int) rc4(8x,mmx) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx)
                compiler: cc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,–noexecstack -DL_ENDIAN -DTERMIOS -O3 -fomit-frame-pointer -Wall -O2 -pipe -fno-strict-aliasing -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
                The 'numbers' are in 1000s of bytes per second processed.
                type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                aes-128-cbc      14577.42k    53591.79k  164157.89k  351943.00k  403794.60k

                1 Reply Last reply Reply Quote 0
                • jimpJ
                  jimp Rebel Alliance Developer Netgate
                  last edited by

                  @miloman:

                  heres your wall of text. :)

                  Thanks :-)

                  Summarizing a little…
                  @miloman:

                  OpenSSL 0.9.8q, aesni.ko UNloaded:
                  aes-128-cbc      98891.21k  107382.09k  109967.84k  110944.78k  111161.64k

                  OpenSSL 1.0.1c, aesni.ko UNloaded:
                  aes-128-cbc    579671.54k  617097.75k  627073.02k  629606.40k  630352.55k

                  OpenSSL 0.9.8q, aesni.ko loaded:
                  aes-128-cbc      14535.69k    53485.26k  164217.56k  351201.58k  403372.36k

                  OpenSSL 1.0.1c, aesni.ko loaded:
                  aes-128-cbc      14502.88k    53441.32k  163384.91k  350985.22k  403617.11k

                  OpenSSL 0.9.8q, aesni.ko loaded, cryptodev engine:
                  aes-128-cbc      14514.45k    53674.88k  164313.53k  351151.19k  403847.11k

                  OpenSSL 1.0.1c, aesni.ko loaded, cryptodev engine
                  aes-128-cbc      14577.42k    53591.79k  164157.89k  351943.00k  403794.60k

                  It looks like loading aesni.ko does make it get used, since there is a substantial difference between the base system aesni before and after it is loaded.
                  Oddly, OpenSSL 1.0.1c without aesni.ko loaded is even faster. I'm not sure if that's somehow linked to OpenSSL's internal aesni support that may be getting dragged down by cryptodev or what.

                  If you repeat that test (just the first two commands), are the results the same each time?

                  Once aesni.ko is loaded it doesn't seem to matter which version of openssl is used or the engine used, too, suggesting at least the speed command is autoselecting the engine based on the cipher being used. (I confirmed this is also the case on ALIX with glxsb). So the last two commands can be ignored apparently.

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  1 Reply Last reply Reply Quote 0
                  • M
                    miloman
                    last edited by

                    If you repeat that test (just the first two commands), are the results the same each time?

                    yes… i ran the commands a couple of times to see if the speed/results were consistent.

                    let me know if you need me to test anything else. :)

                    1 Reply Last reply Reply Quote 0
                    • jimpJ
                      jimp Rebel Alliance Developer Netgate
                      last edited by

                      It may be helpful if others with capable hardware could run the same test, I started a spreadsheet here:
                      https://docs.google.com/spreadsheet/ccc?key=0AojFUXcbH0ROdE15eHB4dndHTXZYcU1mQm9Dc3V2elE

                      The only other thing to try is a similar test but with actual VPN traffic (e.g. OpenVPN using AES-128-CBC) to see if (a) throughput is improved and/or (b) cpu usage reduced under load.

                      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                      Need help fast? Netgate Global Support!

                      Do not Chat/PM for help!

                      1 Reply Last reply Reply Quote 0
                      • jimpJ
                        jimp Rebel Alliance Developer Netgate
                        last edited by

                        Thought of one more thing:

                        cryptotest -va aes128
                        
                        

                        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                        Need help fast? Netgate Global Support!

                        Do not Chat/PM for help!

                        1 Reply Last reply Reply Quote 0
                        • B
                          bardelot
                          last edited by

                          Just out of curiosity, I wonder how big the difference is when you do not use EVP (e.g. without -evp). Or is AES-NI only used when using EVP anyway?

                          1 Reply Last reply Reply Quote 0
                          • G
                            ggzengel
                            last edited by

                            Intel(R) Core(TM) i5 CPU 660 @ 3.33GHz

                            (cryptodev) BSD cryptodev engine
                            [RSA, DSA, DH, AES-128-CBC, AES-192-CBC, AES-256-CBC]
                                [ available ]
                            (rsax) RSAX engine support
                            [RSA]
                                [ available ]

                            cryptotest -a aes 100000 100000
                            23.461 sec,  200000    aes crypts,  100000 bytes, 852493443 byte/sec,  6504.0 Mb/sec

                            /usr/local/bin/openssl speed -evp aes-128-cbc -engine cryptodev -multi 4
                            OpenSSL 1.0.1c 10 May 2012
                            evp              33879.67k  137175.74k  474658.63k  1254087.68k  1675531.61k

                            /usr/local/bin/openssl speed -evp aes-256-cbc -engine cryptodev -multi 4
                            evp              33888.18k  135526.57k  447022.51k  1109458.88k  1423601.97k

                            1 Reply Last reply Reply Quote 0
                            • V
                              vitek
                              last edited by

                              Input from my machine an virtualized pfsense in esxi 5.1. (AES NI working on other win7 guest, so its correctly passthroughed)
                              ESXI host specs:
                              Xeon 1220
                              32gb ram
                              Intel NICs

                              pfSense guest specs:
                              2 cores
                              1gb ram
                              VMxNet3 nics

                              Before kldload aesni

                              [2.1-BETA1][admin@pfsense.localdomain]/root(1): /usr/bin/openssl speed -evp aes-128-cbc -elapsed
                              You have chosen to measure elapsed time instead of user CPU time.
                              To get the most accurate results, try to run this
                              program when this computer is idle.
                              Doing aes-128-cbc for 3s on 16 size blocks: 25200854 aes-128-cbc's in 3.00s
                              Doing aes-128-cbc for 3s on 64 size blocks: 7556040 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 256 size blocks: 1974553 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 1024 size blocks: 506622 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 8192 size blocks: 63906 aes-128-cbc's in 3.01s
                              OpenSSL 0.9.8q 2 Dec 2010
                              built on: date not available
                              options:bn(64,64) md2(int) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) blowfish(idx)
                              compiler: cc
                              available timing options: USE_TOD HZ=128 [sysconf value]
                              timing function used: gettimeofday
                              The 'numbers' are in 1000s of bytes per second processed.
                              type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                              aes-128-cbc    134377.68k  160686.63k  167961.52k  172378.58k  173953.57k

                              [2.1-BETA1][admin@pfsense.localdomain]/root(3): /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed
                              You have chosen to measure elapsed time instead of user CPU time.
                              Doing aes-128-cbc for 3s on 16 size blocks: 111268869 aes-128-cbc's in 3.00s
                              Doing aes-128-cbc for 3s on 64 size blocks: 30363529 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 256 size blocks: 7753535 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 1024 size blocks: 1944836 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 8192 size blocks: 243389 aes-128-cbc's in 3.01s
                              OpenSSL 1.0.1c 10 May 2012
                              built on: Sun Jan 27 13:08:29 EST 2013
                              options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
                              compiler: cc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,–noexecstack -DL_ENDIAN -DTERMIOS -O3 -DMD32_REG_T=int -Wall -O2 -pipe -fno-strict-aliasing -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
                              The 'numbers' are in 1000s of bytes per second processed.
                              type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                              aes-128-cbc    593433.97k  646072.80k  659916.45k  662113.10k  662887.96k

                              after kldload aesni

                              [2.1-BETA1][admin@pfsense.localdomain]/root(5): /usr/bin/openssl speed -evp aes-128-cbc -elapsed
                              You have chosen to measure elapsed time instead of user CPU time.
                              To get the most accurate results, try to run this
                              program when this computer is idle.
                              Doing aes-128-cbc for 3s on 16 size blocks: 2914003 aes-128-cbc's in 3.00s
                              Doing aes-128-cbc for 3s on 64 size blocks: 2776488 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 256 size blocks: 2127090 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 1024 size blocks: 1097708 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 8192 size blocks: 129159 aes-128-cbc's in 3.01s
                              OpenSSL 0.9.8q 2 Dec 2010
                              built on: date not available
                              options:bn(64,64) md2(int) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) blowfish(idx)
                              compiler: cc
                              available timing options: USE_TOD HZ=128 [sysconf value]
                              timing function used: gettimeofday
                              The 'numbers' are in 1000s of bytes per second processed.
                              type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                              aes-128-cbc      15517.00k    59045.34k  180937.99k  373499.22k  351573.93k

                              [2.1-BETA1][admin@pfsense.localdomain]/root(6): /usr/local/bin/openssl speed -evp aes-128-cbc -elapsed
                              You have chosen to measure elapsed time instead of user CPU time.
                              Doing aes-128-cbc for 3s on 16 size blocks: 2870466 aes-128-cbc's in 3.00s
                              Doing aes-128-cbc for 3s on 64 size blocks: 2702743 aes-128-cbc's in 3.02s
                              Doing aes-128-cbc for 3s on 256 size blocks: 2093458 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 1024 size blocks: 1087780 aes-128-cbc's in 3.01s
                              Doing aes-128-cbc for 3s on 8192 size blocks: 130583 aes-128-cbc's in 3.01s
                              OpenSSL 1.0.1c 10 May 2012
                              built on: Sun Jan 27 13:08:29 EST 2013
                              options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
                              compiler: cc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,–noexecstack -DL_ENDIAN -DTERMIOS -O3 -DMD32_REG_T=int -Wall -O2 -pipe -fno-strict-aliasing -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
                              The 'numbers' are in 1000s of bytes per second processed.
                              type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
                              aes-128-cbc      15309.15k    57359.77k  178177.74k  370331.17k  355652.47k

                              I can add that i have tested actual VPN performance which conclude.
                              Speed measured with iperf on 2 windows 7 machines one on LAN and on WAN

                              If i just route between 2 nets without an tunnel the speeds are well above Gbit speed. CPU usage = ~75%

                              If i use the vpn tunnel with AES 128 the speed is around 300mbit (around same speed with BSD engine, no hardware , and RSX engine). CPU usage ~40%

                              If i use the vpn tunnel with NO encryption the speed is still around 300mbit.

                              Not really sure why as soon as the tunnel is used the speed no more than 300mbit.

                              Hope this helps!
                              Let me know if I should test something else.

                              1 Reply Last reply Reply Quote 0
                              • jimpJ
                                jimp Rebel Alliance Developer Netgate
                                last edited by

                                You might be hitting a general openvpn limit at some point there, check threads around the forum here, you might at least try this tweak:
                                http://forum.pfsense.org/index.php/topic,47567.0.html

                                Your numbers seem to coincide with the similar numbers from the previous tester as well.

                                Did you happen to try the VPN speed without aesni.ko loaded? Or just with and toggling the engine setting?

                                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                Need help fast? Netgate Global Support!

                                Do not Chat/PM for help!

                                1 Reply Last reply Reply Quote 0
                                • V
                                  vitek
                                  last edited by

                                  Actually now that you say it. I only tested the vpn speed without the aesni.ko loaded. I should test it with it loaded.

                                  Ill also check the thread with the tweak.

                                  EDIT: I tested with the aesni.ko loaded no speedchange. Might be higher cpu usage though not entirely sure.
                                  Also tested the ip fastforwarding tweak which had no effect.

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    adam65535
                                    last edited by

                                    Did anyone ever discover why there was no apparent change in performance with aes-ni enabled?  I did a search for aes-ni and aesni but didn't see any further threads.  I don't have a system with aes-ni on 2.1 yet.

                                    1 Reply Last reply Reply Quote 0
                                    • K
                                      kejianshi
                                      last edited by

                                      I see you are testing IPsec earlier and some openvpn. I would be interested in knowing what the maximum throughput you might get with all 4 cores enabled, using 4 separate clients connecting to 1 server each client on a different port with separate openvpn instance for each.  Its probably not part of your testing, bit would be interesting to know if it will saturate a gigabit interface.

                                      As far as file transferes from 1 computer to another be careful that drive read/write speed isn't a bottleneck.

                                      1 Reply Last reply Reply Quote 0
                                      • jimpJ
                                        jimp Rebel Alliance Developer Netgate
                                        last edited by

                                        @adam65535:

                                        Did anyone ever discover why there was no apparent change in performance with aes-ni enabled?  I did a search for aes-ni and aesni but didn't see any further threads.  I don't have a system with aes-ni on 2.1 yet.

                                        Not yet, mostly for lack of a good test setup. We're building up some test rigging/infrastructure to get some good throughput numbers for the new book and for other purposes and I believe some of that hardware does have AES-NI, so we may have better information in the coming months.

                                        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                        Need help fast? Netgate Global Support!

                                        Do not Chat/PM for help!

                                        1 Reply Last reply Reply Quote 0
                                        • A
                                          adam65535
                                          last edited by

                                          jimp, If I have time I will probably throw a 2.1 snapshot one of the Dell R320 servers I have and see how the openssl test does on it.  I assume it will reveal the same results as everyone else though.

                                          kejianshi,  I just don't have time to do that kind of testing right now.

                                          1 Reply Last reply Reply Quote 0
                                          • jimpJ
                                            jimp Rebel Alliance Developer Netgate
                                            last edited by

                                            A single test probably won't really tell us much. What we'd really need to see is a pair of identical systems configured identically back-to-back (but with different IPs/subnets as needed) and see what kind of LAN-to-LAN throughput we can obtain through an active/live VPN in each of the test cases

                                            1. aesni.ko loaded, OpenVPN set to use cryptodev
                                            2. aesni.ko loaded, OpenVPN set to "no hardware"
                                            3. aesni.ko unloaded, OpenVPN set to use cryptodev
                                            4. aesni.ko unloaded, OpenVPN set to "no hardware"

                                            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                            Need help fast? Netgate Global Support!

                                            Do not Chat/PM for help!

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.