OpenVPN speed vs hardware

krock72 · Nov 14, 2017, 1:27 AM

Hi All,

1st a "Thank you" to everyone who works so hard at posting so much great info on this forum. This has been an extremely helpful resource for me in my short journey to run pfsense with an openvpn to PIA full time.

I also wanted to post some info on hardware and the actual VPN throughput achieved from usenet source which can more than saturate my 300mbps cable.

Quad Core Atom E3845 1.91GHz, 2MB L2 Cache, CPU Supports AES-NI, 8GB DDR3L RAM and 128GB mSATA SSD
312mbps without openvpn 115mbps with openvpn

Core i3-7350K 4.20 GHz, ASRock H270M-ITX/AC, 8GB DDR42400, 123 mSATA
312mbps without openvpn 312mbps with openvpn

I got much better speed on the E3845 after adding these settings and I left them on the new system
OpenVPN -Advanced Configuration-Custom options
persist-key
persist-tun
remote-cert-tls server
reneg-sec 0

UDP Fast I/O checked
Send/Receive Buffer 1mb
Verbosity level 3

Hope this helps someone,

Kevin

Rango · Mar 8, 2018, 7:50 AM

How much did your speed improve after those settings?

I have N3150 and you're getting same speed with hardware as ppl tested so makes me think it's tweaks that make speed increase. Thoughts?

EDIT

Here is big question, does your 7350K have AES-NI and is it enabled? Is this PC with PSU or NUC?

It doesn't matter actually, i'm just trying to prove it to myself that AES-NI doesn't work currently and it's all about cpu cycles. PERIOD.

If it doesn't have it it proves CPU did it by itself. If you has it and it's enabled it still shows AES-NI does nothing as double cpu cycles did this job. This is sad.

Rango · Mar 8, 2018, 7:49 AM

I just did testing on my own on live vpn connection. With N3150 with 1.6Ghz CPU i'm getting max 105Mbs on AES-128-GCM. My CPU only peak utlization was 49%, never got above 50%. Don't know why it didn't hit 100%.

It seems AES-NI doens't work at all in pfsense. It's useless. Why do i say that. Look at your pc cpu clocks.

2.0Ghz got you 115Mbps which makes sense with your extra 400mhz but look at your 4Ghz pc it doulbed the speed which also makes sense as 115x2 equals 330Mbps.

It seems it's all about CPU cycles. Sad AES-NI does not work at all or it's completely useless looking at Cpu cycles analysis itself. It seems to hit 200Mbps one would need ~ 3Ghz CPU.

My ISP is 180Mbps. I wanted to hit 100% less my ISP speed so ~165-170Mbps. I was getting 50Mbps with Asus 87u router so i doubled but i paid $300 and it this NUK comes short.

It seem i will have to spend another $400 to build some monster with 4.0 Ghz cycle rate cause AES-NI don't work, at least in pfsense (2.4.2) software. Maybe that will change in 2.5 release.

SO CONCLUSION IT'S ALL ABOUT CPU CYCLES AND AES-NI DOES NOT WORK. I love pfsense but i'm dissapointed in those results.

It says it's active but it absolutely does NOT help cpu cycles at all. There is no difference with it on based on those results here.

tibere86 · Mar 8, 2018, 9:03 PM

I run pfSense on a Quad Core Atom E3950 1.6GHz (Burst to 2.0GHz), 2MB L2 Cache, AES-NI, 8GB DDR3L RAM and 64GB mSATA SSD
I have FIOS 150/150 internet plan, and I max out my connect using OpenVPN + AES-128-CBC on PIA VPN. CPU usage is around 20% during speed tests.

Rango · Mar 8, 2018, 9:39 PM

Let's see the screenshot. I was able to improve my speed last night by enabling FreeBSD hardware acceleration in openvpn client and in advanced options under networking instead of AES-NI so now i get

115-120Mbps which is better with 50% of cpu usage. So i gained 10Mbps but that's where rubber meets the road. No more. The AES-NI doesn't work or it works slightly that one can not notice its effect.

Based on what i see CPU cycles do 95% of the work and AES-NI is useless. If i learned that here on forum i would have chose different solution with more CPU power as i nee at minimum 200Mbps and

ideal would be 300Mbps so probably i newer processor with 2.5Ghz-3Ghz cycle rate. I think this thread below is right as far as Mbps goes as i matched N3150 performance in real testing on openvpn.

If anyone has any suggestions on newer mobile processors with lower power consumption please let me know. I see this thread is 2 yrs old already. I found NUC on aliexpress with 4500 i 7 cput but it's $300 with shipping.

Not bad but too much for 2 yr old processor is somewhat already old. I should have done this in first place. Ideally i would like to have NUC that can max out 1Gbs connection on openvpn but that's pipe dream i think at this point.

It would have to be some 4.5Ghz monster with 100watt power consumption. So FreeBSD hardware acceleration work, AES-NI doesn't and it assist like 5-10%. Nothing significant. Those speeds in that thread below are right on the money i think.

I'm learning a lot by tweaking here and there in pfsense. Now when i'm looking at consumer router i think it's a toy. I converted my asus 87u into access point and lan switch. That thing maxed out at

45Mbps with 1.0Ghz processor. One can again see coraliation of CPU cycles here again and it being ARM processor doesn't help either. It seems newer CPU with lower cycles would do better then older

CPU with same cycle rate. I would also NOT do ARM with AES-NI. I think it won't do well. Look at the performance of atom and celerons. I would expect similar performance out of ARM. I think CPU needs to be power pc grade and it seems i7 i5 and some newer AMD APU feet the bill. AMD APU

A10are cheap and clock rates are 4Ghz so that would be ideal but then what motherboard. How to get 2 NICs, all this is an obstacle.

https://forum.pfsense.org/index.php?topic=115673.0

Ryu945 · Mar 12, 2018, 5:43 AM

@Rango:

How much did your speed improve after those settings?

I have N3150 and you're getting same speed with hardware as ppl tested so makes me think it's tweaks that make speed increase. Thoughts?

EDIT

Here is big question, does your 7350K have AES-NI and is it enabled? Is this PC with PSU or NUC?

It doesn't matter actually, i'm just trying to prove it to myself that AES-NI doesn't work currently and it's all about cpu cycles. PERIOD.

If it doesn't have it it proves CPU did it by itself. If you has it and it's enabled it still shows AES-NI does nothing as double cpu cycles did this job. This is sad.

AES-NI does work in Pfsense. It is just really buggy. I have it turned on for the server but not for the individual VPNS and for some reason, that is what makes it work. If I turn it on for the VPNs itself then it stops working. Also, have to have AES-NI and Crypto Dev both turned on for it to work at all (if I remember correctly).

@tibere86:

I run pfSense on a Quad Core Atom E3950 1.6GHz (Burst to 2.0GHz), 2MB L2 Cache, AES-NI, 8GB DDR3L RAM and 64GB mSATA SSD
I have FIOS 150/150 internet plan, and I max out my connect using OpenVPN + AES-128-CBC on PIA VPN. CPU usage is around 20% during speed tests.

OpenVPN is a single threaded process. The peak OpenVPN could ever take a Quad Core is 25%. You can go slightly higher when you add Pfsense processes being run in other cores.

Rango · Mar 12, 2018, 5:49 AM

@Ryu945:

@Rango:

How much did your speed improve after those settings?

I have N3150 and you're getting same speed with hardware as ppl tested so makes me think it's tweaks that make speed increase. Thoughts?

EDIT

Here is big question, does your 7350K have AES-NI and is it enabled? Is this PC with PSU or NUC?

It doesn't matter actually, i'm just trying to prove it to myself that AES-NI doesn't work currently and it's all about cpu cycles. PERIOD.

If it doesn't have it it proves CPU did it by itself. If you has it and it's enabled it still shows AES-NI does nothing as double cpu cycles did this job. This is sad.

AES-NI does work in Pfsense. It is just really buggy. I have it turned on for the server but not for the individual VPNS and for some reason, that is what makes it work. If I turn it on for the VPNs itself then it stops working. Also, have to have AES-NI and Crypto Dev both turned on for it to work at all (if I remember correctly).

@tibere86:

I run pfSense on a Quad Core Atom E3950 1.6GHz (Burst to 2.0GHz), 2MB L2 Cache, AES-NI, 8GB DDR3L RAM and 64GB mSATA SSD
I have FIOS 150/150 internet plan, and I max out my connect using OpenVPN + AES-128-CBC on PIA VPN. CPU usage is around 20% during speed tests.

OpenVPN is a single threaded process. The peak OpenVPN could ever take a Quad Core is 25%. You can go slightly higher when you add Pfsense processes being run in other cores.

Based on what you said cryptodev is doing the boost not AES-NI. Enabled AES-NI in openvpn client only and under advanced networking settings and you will see it makes ZERO difference. When you do that for Crypto dev on client and adv settings without AES-NI it boosts 10% so Crypto dev works but AES-NI DOES NOT. I use it as client so i have no use for it as server unless somehow i can setup multiple instances of connections to my vpn provider. I asked a question how to set that up but so far no answer how to do this.

Ryu945 · Mar 12, 2018, 5:55 AM

Crypto-Dev by itself also did nothing. I only got it to work when both were turned on.

Here is the hardware I assembled and some results. I still need to add more information to the thread (so ask questions) and look to do any other optimizations. I haven't done anything yet other then duel VPN and hardware acceleration

https://forum.pfsense.org/index.php?topic=144583.0

Rango · Mar 12, 2018, 6:12 AM

@Ryu945:

Crypto-Dev by itself also did nothing. I only got it to work when both were turned on.

That's interesting. I now only have Crypto Dev on both sides and it boosts 20% so i can get 120Mbs on N3150 and medium is about 115-117Mbps but when i switch to only AES-NI it goes down by 20%

to base line with is about 100Mbps which is what you see in screenshot above. I tried it every possible combination and that's what i'm getting. At least i'm happy Cryptodev is working and boosting a bit, 20%.

Maybe if AES-NI would work it would boost much more. I dunno what the expectation of hardware based acceleration should be. I just reported what my testing yielded. I am happy with pfsense but it

seems AES-NI module is not working and looks like Cryptop Dev is FreeBSD solution to it, for now maybe. Maybe in 2.5 this will change when they focus on it. I can't wait if so.

I am however disappointed i purchased N3150 however. I didn't do enough research then. The fact that i owned asus 87u also purchased for encryption. It is now exclusively AP. I guess as they say u learn on your own mistakes. I've learned. Thanks for posting your results. :)

Ryu945 · Mar 12, 2018, 2:31 PM

@Rango:

@Ryu945:

Crypto-Dev by itself also did nothing. I only got it to work when both were turned on.

That's interesting. I now only have Crypto Dev on both sides and it boosts 20% so i can get 120Mbs on N3150 and medium is about 115-117Mbps but when i switch to only AES-NI it goes down by 20%

to base line with is about 100Mbps which is what you see in screenshot above. I tried it every possible combination and that's what i'm getting. At least i'm happy Cryptodev is working and boosting a bit, 20%.

Maybe if AES-NI would work it would boost much more. I dunno what the expectation of hardware based acceleration should be. I just reported what my testing yielded. I am happy with pfsense but it

seems AES-NI module is not working and looks like Cryptop Dev is FreeBSD solution to it, for now maybe. Maybe in 2.5 this will change when they focus on it. I can't wait if so.

I am however disappointed i purchased N3150 however. I didn't do enough research then. The fact that i owned asus 87u also purchased for encryption. It is now exclusively AP. I guess as they say u learn on your own mistakes. I've learned. Thanks for posting your results. :)

I did this AES-NI test with the version that came out before the Spectrum/Meltdown bug so I don't know if things have changed in the version I currently run. I will have to run more test at a later time. I did notice a massive speed reduction after that update.