AES-NI instructions…

jimp

Hard to say, but you'd have to compare it with the output on the same hardware running bare metal without ESX.

biggsy

Unfortunately, I haven't got any chance of trying 2.2 on the bare metal but I should probably have mentioned that 2.1.5-RELEASE (amd64) is currently running on the same hardware and ESXi version. AES-NI is showing up in Crypto Hardware Acceleration on that VM.

timmyj9

i can confirm same situation here with the hardware crypto acceleration working in pfSense 2.1 but not 2.2-beta (also under ESXi 5.5)

eri--

Please provide information from your dmesg the 20 top rows.

Also the output of kldload -v aesni and dmesg info after.

biggsy

On 2.1.5:


Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014
    root@pf2_1_1_amd64.pfsense.org:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.8 amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2493.80-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x306a9  Family = 6  Model = 3a  Stepping = 9
  Features=0xfa3fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>Features2=0x96982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,avx,hv>AMD Features=0x28100800 <syscall,nx,rdtscp,lm>AMD Features2=0x1 <lahf>TSC: P-state invariant
real memory  = 4294967296 (4096 MB)
avail memory = 4092432384 (3902 MB)
ACPI APIC Table: <ptltd  apic ="">MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <version 1.1="">irqs 0-23 on motherboard
wlan: mac acl policy registered
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff804abaf0, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff804abb90, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff804abc30, 0) error 1
kbd1 at kbdmux0
cryptosoft0: <software crypto="">on motherboard
padlock0: No ACE support.

$ kldload -v aesni
kldload: can't load aesni: File exists</software></version></ptltd ></lahf></syscall,nx,rdtscp,lm></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,avx,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>

On 2.2 (updated from clone of 2.1.5 with restored config)


Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.1-RELEASE #0 29f4af5(releng/10.1)-dirty: Sat Nov 15 10:43:23 CST 2014
    root@pfsense-22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10 amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.33-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x306a9  Family = 0x6  Model = 0x3a  Stepping = 9
  Features=0xfa3fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>Features2=0x9e982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>AMD Features=0x28100800 <syscall,nx,rdtscp,lm>AMD Features2=0x1 <lahf>TSC: P-state invariant
real memory  = 4294967296 (4096 MB)
avail memory = 4098441216 (3908 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ptltd  apic ="">MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <version 1.1="">irqs 0-23 on motherboard
wlan: mac acl policy registered
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80606680, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80606730, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff806067e0, 0) error 1
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff8062de50, 0) error 1
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff8062df00, 0) error 1
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff8062dfb0, 0) error 1
random: <software, yarrow="">initialized
kbd1 at kbdmux0
cryptosoft0: <software crypto="">on motherboard
padlock0: No ACE support.

$ kldload -v aesni
kldload: can't load aesni: module already loaded or in kernel</software></software,></version></ptltd ></lahf></syscall,nx,rdtscp,lm></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>

2.2 still shows this message on boot:


padlock0: No ACE support.
aesni0: No SSE4.1 support.

fileman

Same problem with 2.2 RC last snapshot.

![cpu aesni.png](/public/imported_attachments/1/cpu aesni.png)
![cpu aesni.png_thumb](/public/imported_attachments/1/cpu aesni.png_thumb)

jimp

Seems OK here on bare metal

: dmesg | egrep -i '(SSE|aes.*ni)'
  Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x43d8e3bf <sse3,pclmulqdq,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,movbe,popcnt,tscdlt,aesni,rdrand>aesni0: <aes-cbc,aes-xts,aes-gcm>on motherboard
: kldstat | grep aesni
 3    1 0xffffffff82612000 60b5     aesni.ko</aes-cbc,aes-xts,aes-gcm></sse3,pclmulqdq,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,movbe,popcnt,tscdlt,aesni,rdrand></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>

I do see that message when loading aesni.ko inside a VMware VM, though.

heper

so aes-ni doesn't work inside VM's ?

jimp

I'd wager that has more to do with the hypervisor than the OS since it works on bare metal but it's tough to say for sure.

zanon

Hi ,
I had some free time around Christmas and played with new 2.2 RC .
I have tested new VMware 6.0 RC as well as ESXI 5.5 and directly on bare e3-1230v2, and can confirm that the problem with aesni persist with both Hypervisors.

I spend long time to test multiple cases with both 2.1.5 and 2.2 versions of pfsense on VM <->VM scenario. The results is one and the same. no HW acceleration at all.
I also tried my spare e3-1230v2 against my prod, both versions 2.1.5 and looks like HW acceleration is not working as well. speed is capped near ~~ 326 Mbits/sec.
Unfortunately I cannot install 2.2RC in prod to test it…. ... But looks lke HW acceleration works for 2.2.rc
(when i perform tests from 2.2 against 2.1.5 speed is near 400 Mbits/sec , when i test from 2.1.5 against 2.2rc speed is droping to 312 Mbits/sec)
i also have to confirm that pure speed between 2 * VM 2.2rc (vmx3) is like 3.04 Gbits/sec when 2 * vm 2.1.5 (vmx3) is hardly hitting 1.59 Gbits/sec .
unfortunately with no HW acceleration the IPSEC speed is like i said ~~ 350 Mbits/sec.

At the end, I am not an expert, but looks like this "No SSE4.1 support" problem is some misunderstanding in aesni_probe module related to the way vmware reports Features= and Features2= to guest operating system .
(but dont shoot me if i am wrong ) ;)

biggsy

In my brief test of 2.2RC in a VM yesterday, I didn't see the "padlock0: No ACE support/aesni0: No SSE4.1 support" messages but I wasn't watching for them.

With 2.1.5 running on 5.5 U2 everything seems to be OK:

$ dmesg | egrep -i '(SSE|aes.*ni)'
  Features=0xfa3fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>Features2=0x96982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,avx,hv>aesni0: <aes-cbc,aes-xts> on motherboard</aes-cbc,aes-xts></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,avx,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,dts,mmx,fxsr,sse,sse2,ss>

els

My machine has AES-NI and I did a "dmesg" to confirm that. I have site-to-site VPN running and it works (except for the bug with IPSEC widget). How do I tell if AES-NI is being utilized? Do I need to make configuration change to force it to use AES-NI?

zanon

Hi,
just tested a fresh FreeBSD 10.1 installation on esxi 5.5u2. AES-NI looks working

uname -a 
FreeBSD  10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

dmesg | grep -i aes
  Features2=0x9e982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>aesni0: <aes-cbc,aes-xts> on motherboard</aes-cbc,aes-xts></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>

vs the very same VM with pfsense :

 uname -a
FreeBSD pfSense.localdomain 10.1-RELEASE-p3 FreeBSD 10.1-RELEASE-p3 #0 8bdb2f8(releng/10.1)-dirty: Thu Jan  1 15:43:28 CST 2015     root@pfsense-22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10  amd64

 Features2=0x9e982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>aesni0: No SSE4.1 support.</sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>

after i copy the module /boot/kernel/aesni.ko from freebsd to pfsense i got 1 warning , but eventually looks like working :

dmesg | grep -i aes
  Features2=0x9e982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>warning: KLD '/boot/kernel/aesni.ko' is newer than the linker.hints file
aesni0: <aes-cbc,aes-xts>on motherboard</aes-cbc,aes-xts></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,hv>

jimp

The FreeBSD module does not include our code for IPsec acceleration of AES-GCM. It would not be useful on pfSense in general.

eri--

I will double check this though that should not prevent our module to not attach where freebsd one attaches.
I will post here when resolve that.

EDIT: Oh i forgot the 10.1 FreeBSD does not have any AES-GCM code :)

zanon

Hi, аs long as i could imagine,
the problem is not in specific implementation of AES additions, but in the detection of processor Features and Features2 in aesni_probe module.
but enough for this :)

i really have to share that most of us, people who are using pfsense, are pretty excited of your work guys .

Thank you for everything you are doing .

eri--

Yeah but AES-GCM has more requirments than plain AES-CBC/XTS speedup.