Went physical to virtual, AES is having no effect on OpenVPN performance
-
So I went ahead and virtualized pfsense (in ESXi 6, with NICs passedthrough) and everything as far as I can tell functions flawlessly. However, I am noticing a strange problem with OpenVPN performance post the P-to-V. Here are the specs for servers.
Server 1:
Lenovo RS140
E3-1225v3
2x 8GB DDR3 ECC UDIMM
X520-DA2Server 2:
ASRock EP2C602-4L/D16
2x E5-2670
16x 8GB DDR3 ECC RDIMMI have OpenVPN server set up on both, and Server 2 is set as a client to connect to 1 as site-to-site. Server 1 has a 150/150 link, and Server 2 is 200/20. Prior to virtualizing, I was able to pull stuff to Server 2 side at about link speed of Server 1. After virtualizing, it's now stuck at about ~40Mb. I looked around on pfsense forums, and the only options that could be related to OpenVPN performance is aesni.ko on/off, cryptodev on/off and ip.fastforwarding=0/1. I've tried combinations of all 3, and it's having zero effect on the performance. CPU load during transfers is about 3-4% on Server 1 and about 10% on Server 2. I also confirmed that on CLI level, both servers seems to be seeing AES support from CPU properly, and loading the aesni.ko module is indeed allowing the proper ciphers to be available to cryptodev. I was even able to run commands to measure the encryption performancing using AES-128-CBC. I'm not really understanding what else could be causing this performance cap.