The AES-NI checkbox in the GUI enables AES-NI for AES 128/192/256 CBC via cryptodev. That means that for each block of data to encrypt, the openssl library will issue an ioctl to send that block to the kernel, suffering a context switch penalty. Since the computation being performed is exactly the same as what openssl would do without cryptodev (and in that case, without the context switch) it is necessarily slower; there is no advantage at all in enabling AES-NI via cryptodev. You do not see a penalty for GCM modes because those are not implemented in cryptodev and so openssl continues to use its internal routines.
So why does the AES-NI kernel module exist at all? If you are using ipsec, which does all of its encryption in the kernel, then you need the AES-NI kernel module to let the ipsec module to use AES-NI–and in that case it's a performance gain because everything is happening in-kernel. Ideally, pfsense would enable a configuration in which you can load aesni.ko for ipsec without loading cryptodev, so you can get the benefits without the drawbacks.
So when would you ever want cryptodev? The /dev/crypto interface is only worth using with external crypto processors, like the old via padlock or the hifn cards (though you're generally much better off just throwing out such hardware and buying something new if you care at all about crypto performance; the crypto accelerator on the old alix boards, for example, was about as fast as a new raspberry pi or an APU1 without hardware crypto, and an order of magnitude slower than an APU2). In theory it might also have a benefit for quick assist, but I think that's implemented in openssl in a way that avoids using /dev/crypto. There's been speculation over the years that cryptodev might help improve cpu utilization, but I haven't seen results on modern hardware where any speculative gain outweighs the performance penalty of the context switching overhead.
