AES-NI performance
-
I am looking for a solution that will mostly be used for AES encryption and decryption. It is most likely that a CPU with AES-NI will offer the best value. Routing is a requirement but not a high priority. Surprisingly, after looking around, not only am I under the impression that the Raspberry Pi 3 offers good value - but perhaps more surprisingly is that a Raspberry Pi 3 actually offers the best AES-NI performance. I say this in comparison to what I've seen posted here from various pfsense users.
To keep things simple, I've settled on one single value: openssl's AES-256-CBC 8192k column. The following are my results:
openssl speed -evp aes-256-cbc Doing aes-256-cbc for 3s on 16 size blocks: 551008 aes-256-cbc's in 0.20s Doing aes-256-cbc for 3s on 64 size blocks: 398438 aes-256-cbc's in 0.23s Doing aes-256-cbc for 3s on 256 size blocks: 193847 aes-256-cbc's in 0.10s Doing aes-256-cbc for 3s on 1024 size blocks: 64090 aes-256-cbc's in 0.04s Doing aes-256-cbc for 3s on 8192 size blocks: 8696 aes-256-cbc's in 0.01s OpenSSL 1.0.2c 12 Jun 2015 built on: reproducible build, date unspecified options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr) compiler: gcc -I. -I.. -I../include -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -march=armv7-a -Wa,--noexecstack -O3 -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 44080.64k 110869.70k 496248.32k 1640704.00k 7123763.20k
That 7123763.20k value is what I'm using to compare to others and that significantly outperforms others.
I am interested if anyone can come close? Even if it does not exceed the Pi 3 but is at least within the same ballpark, then the routing capabilities (and 2nd NIC) of a pfsense solution may result in a better overall solution. Please use "openssl speed -evp aes-256-cbc"
-
I'm using a ZBOX small form factor computer with a Core i5 4570T that supports AES-NI. Here are my results:
[2.3.2-RELEASE][root@pfSense.home]/root: openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 1090105 aes-256-cbc's in 0.29s
Doing aes-256-cbc for 3s on 64 size blocks: 1870619 aes-256-cbc's in 0.34s
Doing aes-256-cbc for 3s on 256 size blocks: 1516771 aes-256-cbc's in 0.23s
Doing aes-256-cbc for 3s on 1024 size blocks: 865844 aes-256-cbc's in 0.20s
Doing aes-256-cbc for 3s on 8192 size blocks: 173742 aes-256-cbc's in 0.02s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 60338.78k 356374.67k 1656718.40k 4364919.41k 91090845.70k -
AM1 box
Doing aes-256-cbc for 3s on 16 size blocks: 857911 aes-256-cbc's in 0.38s
Doing aes-256-cbc for 3s on 64 size blocks: 832702 aes-256-cbc's in 0.33s
Doing aes-256-cbc for 3s on 256 size blocks: 744018 aes-256-cbc's in 0.41s
Doing aes-256-cbc for 3s on 1024 size blocks: 523430 aes-256-cbc's in 0.24s
Doing aes-256-cbc for 3s on 8192 size blocks: 140310 aes-256-cbc's in 0.06s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 36604.20k 162416.54k 459999.66k 2213129.58k 18390712.32k -
highwire - 91090845.70k is the fastest we've seen on this forum!
That's a ZOTAC ZBox ID92 right? Quite a bit more pricey than Raspberry Pi 3 but also better AES performance.
-
Also pretty good W4RH34D - 18390712.32k is about 2.5x Raspberry Pi 3.
What's in your AM1 box - Athlon 5350?
-
Also pretty good W4RH34D - 18390712.32k is about 2.5x Raspberry Pi 3.
What's in your AM1 box - Athlon 5350?
5370
-
Lanner FW-7525D (Quad-core Atom C2558 @ 2.40GHz)
Doing aes-256-cbc for 3s on 16 size blocks: 970824 aes-256-cbc's in 0.31s
Doing aes-256-cbc for 3s on 64 size blocks: 921585 aes-256-cbc's in 0.27s
Doing aes-256-cbc for 3s on 256 size blocks: 753715 aes-256-cbc's in 0.20s
Doing aes-256-cbc for 3s on 1024 size blocks: 449350 aes-256-cbc's in 0.15s
Doing aes-256-cbc for 3s on 8192 size blocks: 92660 aes-256-cbc's in 0.02s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 42591.25k 160843.69k 719856.10k 2245898.56k 24345837.57kPfSense SG-2440 (Dual-core Atom C2358 @ 1.74GHz)
Doing aes-256-cbc for 3s on 16 size blocks: 695323 aes-256-cbc's in 0.36s
Doing aes-256-cbc for 3s on 64 size blocks: 676799 aes-256-cbc's in 0.27s
Doing aes-256-cbc for 3s on 256 size blocks: 550378 aes-256-cbc's in 0.22s
Doing aes-256-cbc for 3s on 1024 size blocks: 330729 aes-256-cbc's in 0.17s
Doing aes-256-cbc for 3s on 8192 size blocks: 67846 aes-256-cbc's in 0.04s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 34474.84k 168916.56k 608168.62k 2167026.48k 14241549.52kKind regards,
Rene. -
Hi!
For fun or reference :). A Hyper-v hosted pfsense on a hp microserver gen 8 with a Xeon 1265Lv2.
[2.3.2-RELEASE][n23]/root: openssl speed -evp aes-256-cbc Doing aes-256-cbc for 3s on 16 size blocks: 1084848 aes-256-cbc's in 0.45s Doing aes-256-cbc for 3s on 64 size blocks: 1345250 aes-256-cbc's in 0.24s Doing aes-256-cbc for 3s on 256 size blocks: 709374 aes-256-cbc's in 0.23s Doing aes-256-cbc for 3s on 1024 size blocks: 472042 aes-256-cbc's in 0.19s Doing aes-256-cbc for 3s on 8192 size blocks: 110932 aes-256-cbc's in 0.03s OpenSSL 1.0.1s-freebsd 1 Mar 2016 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: clang The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 38978.40k 355493.16k 774825.57k 2577978.71k 29080158.21k
-
This is from a connection over wifi, not sure if that makes a difference.
On a SuperMicro 2758:
openssl speed -evp aes-256-cbc Doing aes-256-cbc for 3s on 16 size blocks: 982926 aes-256-cbc's in 0.38s Doing aes-256-cbc for 3s on 64 size blocks: 921181 aes-256-cbc's in 0.27s Doing aes-256-cbc for 3s on 256 size blocks: 761431 aes-256-cbc's in 0.33s Doing aes-256-cbc for 3s on 1024 size blocks: 448646 aes-256-cbc's in 0.19s Doing aes-256-cbc for 3s on 8192 size blocks: 92805 aes-256-cbc's in 0.04s OpenSSL 1.0.1s-freebsd 1 Mar 2016 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: clang The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 41938.18k 215608.99k 594061.21k 2450205.35k 19462619.14k
With the aesni turned off in the advanced settings:
openssl speed -evp aes-256-cbc Doing aes-256-cbc for 3s on 16 size blocks: 955806 aes-256-cbc's in 0.29s Doing aes-256-cbc for 3s on 64 size blocks: 909612 aes-256-cbc's in 0.26s Doing aes-256-cbc for 3s on 256 size blocks: 758911 aes-256-cbc's in 0.30s Doing aes-256-cbc for 3s on 1024 size blocks: 446740 aes-256-cbc's in 0.13s Doing aes-256-cbc for 3s on 8192 size blocks: 92400 aes-256-cbc's in 0.04s OpenSSL 1.0.1s-freebsd 1 Mar 2016 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: clang The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 52905.15k 225804.29k 654420.94k 3659694.08k 19377684.48k
-
highwire - 91090845.70k is the fastest we've seen on this forum!
That's a ZOTAC ZBox ID92 right? Quite a bit more pricey than Raspberry Pi 3 but also better AES performance.
Yes, an ID92. Quite pricey (I bought mine on sale, but still). I bought it for a HTPC but abandoned that plan and re purposed it. It is very much overkill for this application (even running a VPN server) as my connection is only 100mbps/10mbps.
-
I was thinking of firing up the 6 core xeon but I just don't really care for epeen stuff anymore. I mean if someone needs to see it I'll do it, no time for "just for grins" these days.
-
I have a chinese "mini-computer" (gen 5 i5)
I did 2 test and got a very varying result:
Try one:
Doing aes-256-cbc for 3s on 16 size blocks: 1704782 aes-256-cbc's in 0.28s
Doing aes-256-cbc for 3s on 64 size blocks: 1762586 aes-256-cbc's in 0.31s
Doing aes-256-cbc for 3s on 256 size blocks: 1417931 aes-256-cbc's in 0.32s
Doing aes-256-cbc for 3s on 1024 size blocks: 811284 aes-256-cbc's in 0.13s
Doing aes-256-cbc for 3s on 8192 size blocks: 163126 aes-256-cbc's in 0.05s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 96983.15k 360977.61k 1133238.12k 6646038.53k 24435715.51kTry two:
Doing aes-256-cbc for 3s on 16 size blocks: 1727740 aes-256-cbc's in 0.41s
Doing aes-256-cbc for 3s on 64 size blocks: 1742973 aes-256-cbc's in 0.38s
Doing aes-256-cbc for 3s on 256 size blocks: 1414059 aes-256-cbc's in 0.29s
Doing aes-256-cbc for 3s on 1024 size blocks: 815243 aes-256-cbc's in 0.13s
Doing aes-256-cbc for 3s on 8192 size blocks: 163008 aes-256-cbc's in 0.01s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 68046.38k 291396.63k 1252321.22k 6285619.44k 170926276.61k -
Thanks all for the results, keep them coming!
Here are the results so far:
170926276.61k gen 5 i5 91090845.70k Zotac ZBOX ID92 Core i5 4570T 29080158.21k hp microserver gen 8 Xeon 1265Lv2 24435715.51k gen 5 i5 24345837.57k Lanner FW-7525D Quad-core Atom C2558 @ 2.40GHz 19462619.14k SuperMicro 2758 18390712.32k AM1 Athlon 5370 14241549.52k pfSense SG-2440 Dual-core Atom C2358 @ 1.74GHz 7123763.20k Raspberry Pi 3 ARMv7l
-
AR15USR,
There doesn't seem any difference in your tests. Can you try running without the "-evp" option?
openssl speed aes-256-cbc
-
Koenig,
Can you provide the make and model of your "gen 5 i5"?
-
Here's an updated list of results:
170926276.61k gen 5 i5 91090845.70k Zotac ZBOX ID92 Core i5 4570T 42008576.00k Gigabyte GA-N3150N-D3V board Celeron N3150 with AES-NI https://forum.pfsense.org/index.php?topic=108119.0 29080158.21k hp microserver gen 8 Xeon 1265Lv2 27986842.97k Gigabyte GA-N3150N-D3V Celeron N3150 with AES-NI https://forum.pfsense.org/index.php?topic=105114.msg601520#msg601520 24435715.51k gen 5 i5 24345837.57k Lanner FW-7525D Quad-core Atom C2558 @ 2.40GHz 19462619.14k SuperMicro 2758 18390712.32k AM1 Athlon 5370 14241549.52k pfSense SG-2440 Dual-core Atom C2358 @ 1.74GHz 7123763.20k Raspberry Pi 3 ARMv7l 405686.95k Intel i7-4510U + 2x Intel 82574 + 2x Intel i350 Mini-ITX Build https://forum.pfsense.org/index.php?topic=115627.msg646395#msg646395 230708.57k ci323 nano u Celeron N3150 with AES-NI w/ -engine cryptodev https://forum.pfsense.org/index.php?topic=115673.msg656602#msg656602 217617.75k RCC-VE 2440 Intel Atom C2358 https://forum.pfsense.org/index.php?topic=91974.0 124788.74k ALIX.APU2B4/APU2C4 1 GHz Quad Core AMD GX-412TC http://wiki.ipfire.org/en/hardware/pcengines/apu2b4 34204.33k ALIX.APU1C/APU1D 1 GHz Dual Core AMD G-T40E http://wiki.ipfire.org/en/hardware/pcengines/apu1c
-
Koenig,
Can you provide the make and model of your "gen 5 i5"?
There's no brand or model on it…
Something like this: https://www.aliexpress.com/item/Fanless-PC-Intel-NUC-Core-i7-5500u-i5-5257u-Iris-6100-Barebone-Mini-PC-Windows-2HDMI/32755490163.html?spm=2114.01010208.3.100.Dtd346&ws_ab_test=searchweb0_0,searchweb201602_2_10091_10090_10088_10089,searchweb201603_1&btsid=6d47dcd0-df75-47e8-84cf-86813f160f8e
Some more results:
aes-256-cbc 99810.65k 375805.41k 1454872.58k 4844784.55k 28507460.95k
aes-256-cbc 62518.77k 350371.84k 1217122.52k 5055197.38k 34182738.74k
aes-256-cbc 76404.78k 341786.43k 1224697.10k 4425564.16k 34284240.90k
aes-256-cbc 91091.47k 242748.12k 1191453.72k 5068092.37k 85483061.25k
aes-256-cbc 100148.30k 299186.69k 1330803.04k 6668591.10k 86076555.26k
aes-256-cbc 105877.45k 377916.58k 1538361.48k 6694084.61k 57179897.86k
aes-256-cbc 84355.12k 320069.81k 1420017.17k 6647087.10k 57598978.73k
aes-256-cbc 106102.67k 260300.35k 1792681.83k 9638188.87k 34206646.27k
All from the same machine.
-
14241549.52k pfSense SG-2440 Dual-core Atom C2358 @ 1.74GHz
217617.75k RCC-VE 2440 Intel Atom C2358 https://forum.pfsense.org/index.php?topic=91974.0Obviously something off there.
-
First my system details -
System: Netgate SG-4860
Version: 2.3.2-RELEASE-p1 (amd64) built on Fri Sep 30 14:36:56 CDT 2016 FreeBSD 10.3-RELEASE-p9
CPU Type: Intel(R) Atom(TM) CPU C2558 @ 2.40GHz 4 CPUs: 1 package(s) x 4 core(s)
Hardware crypto: AES-CBC,AES-XTS,AES-GCM,AES-ICMResults (system pretty active so possibility for skewed results) -
[2.3.2-RELEASE][admin@pfSense.localdomain]/root: openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 984814 aes-256-cbc's in 0.35s
Doing aes-256-cbc for 3s on 64 size blocks: 920037 aes-256-cbc's in 0.30s
Doing aes-256-cbc for 3s on 256 size blocks: 759776 aes-256-cbc's in 0.26s
Doing aes-256-cbc for 3s on 1024 size blocks: 452100 aes-256-cbc's in 0.15s
Doing aes-256-cbc for 3s on 8192 size blocks: 92821 aes-256-cbc's in 0.03s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 44819.98k 193254.95k 754434.54k 3118823.75k 24332468.22k -
AR15USR,
There doesn't seem any difference in your tests. Can you try running without the "-evp" option?
openssl speed aes-256-cbc
/root: openssl speed aes-256-cbc Doing aes-256 cbc for 3s on 16 size blocks: 5517180 aes-256 cbc's in 3.01s Doing aes-256 cbc for 3s on 64 size blocks: 1544753 aes-256 cbc's in 3.00s Doing aes-256 cbc for 3s on 256 size blocks: 399657 aes-256 cbc's in 3.00s Doing aes-256 cbc for 3s on 1024 size blocks: 258521 aes-256 cbc's in 3.00s Doing aes-256 cbc for 3s on 8192 size blocks: 32712 aes-256 cbc's in 2.99s OpenSSL 1.0.1s-freebsd 1 Mar 2016 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: clang The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256 cbc 29348.53k 32954.73k 34104.06k 88241.83k 89558.79k
For comparison:
/root: openssl speed -evp aes-256-cbc Doing aes-256-cbc for 3s on 16 size blocks: 957210 aes-256-cbc's in 0.39s Doing aes-256-cbc for 3s on 64 size blocks: 893869 aes-256-cbc's in 0.24s Doing aes-256-cbc for 3s on 256 size blocks: 751299 aes-256-cbc's in 0.27s Doing aes-256-cbc for 3s on 1024 size blocks: 450002 aes-256-cbc's in 0.10s Doing aes-256-cbc for 3s on 8192 size blocks: 92472 aes-256-cbc's in 0.02s OpenSSL 1.0.1s-freebsd 1 Mar 2016 built on: date not available options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: clang The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 39207.32k 236212.09k 724075.46k 4537127.86k 32321306.62k