Gigabit with i5-3550 - OpenVPN not getting more than 10Mbit down/up
I'm just posting this here for anyone else scouring the internet looking for a solution.
I have two instances of OpenVPN on my 2.3.4-p1 pfSense box, which is running on an Intel i5-3550 with gigabit fiber service. I have AES-NI enabled and both servers are configured to use the cryptodev engine. One server is on 1194 / UDP, which I use anywhere that isn't the office. For that I use 443 / TCP, because 1194 is blocked there.
For a few weeks I've been trying to figure out why I had such terrible throughput on the 443 server. I ran speedtests at work, school, and my girlfriend's house, which also has gigabit fiber. I could not achieve any more than 10 Mbps down and maybe 11 or 12 M up on the 443 server, while the 1194 server would consistently get 35M down / 40M up. I tried all sorts of things: disabling the shaper on my WAN and LAN interfaces, changing encryption methods, even disabling encryption altogether. Nothing worked.
I just tonight took a look at the system activity to find that the OpenVPN process was chewing up >80% CPU time whenever I was running a speedtest. I double checked the configuration differences between the 443 and 1194 servers, and found that I had previously set the logging on the 443 server to 11 - the highest option available - to troubleshoot a problem a year or so ago. The 1194 was set to default.
Setting the 443 server to default logging level cleared everything up, and OpenVPN takes no more than 5% CPU time now.
So before you go blaming your ISP for rate limiting HTTPS traffic, check that you're not bottlenecked by your routers disk I/O :-[