PC Engines apu2 experiences
-
I have a site-to-site VPN using an OpenVPN tunnel between two APU2C4's with pfSense on them (2.4.3-RELEASE-p1 (amd64)). I've been reading a few hours now on how to really have OpenVPN utilize hardware AES-NI as the CPU supports it. There are several threads about this but not one is clear enough to really explain how pfSense uses this.
So, if I want to use hardware AES-NI, do I need to choose AES-NI CPU-based Acceleration under System > Advanced > Miscellaneous > Cryptographic Hardware? Or should I set it to None (no module loaded) and OpenVPN will use the AES-NI natively from the hardware without any module conflicts from the pfSense BSD OS?
If I choose the former, under the OpenVPN Server/Client settings the only selection I have for Hardware Crypto is No Hardware Crypto Acceleration. The only time I have an extra option in the Client settings is when I choose BSD Crypto Device (cryptodev) under Miscellaneous. So which is which? I'm starting to have a headache because of the confusing pfSense GUI :)
-
I'm also curious as to what the "correct" settings are... I can say that I did a couple of tests wrt speed, and I eventually settled on (apart from a bunch of other tweaks) enabling AES-NI CPU-based Acceleration under System > Advanced > Miscellaneous > Cryptographic Hardware, and then in the OpenVPN Server/Client settings the only selection I have No Hardware Crypto Acceleration.
I know a while back, in the OpenVPN settings you could choose between AES-NI and cryptodev, but after some update the cryptodev disappeard.
Additionally, I think that the best speeds were achieved when both AES-NI CPU-based Acceleration and cryptodev were enabled, which is now the default if you have AES-NI CPU-based Acceleration enabled on the system.
So, I've just assumed that it doesn't matter anymore about the setting in OpenVPN since the system is already using all it can. -
@veldkornet said in PC Engines apu2 experiences:
Additionally, I think that the best speeds were achieved when both AES-NI CPU-based Acceleration and cryptodev were enabled, which is now the default if you have AES-NI CPU-based Acceleration enabled on the system.
So, I've just assumed that it doesn't matter anymore about the setting in OpenVPN since the system is already using all it can.Enabling both where? Let's call both places Miscellaneous settings and Client settings to avoid confusion. Like I said, I don't have any options for a cryptodev Client setting IF I keep the Miscellaneous settings to AES-NI. All I have is the No Hardware Crypto Acceleration option.
-
My mistake, I actually have the Miscellaneous set to AES-NI and BSD Crypto Device (aesni, cryptodev).
This is the update that I referred to in my previous post where both are enabled.
OpenVPN Client & Server:
And these are the options available to me within OpenVPN:
-
@veldkornet said in PC Engines apu2 experiences:
My mistake, I actually have the Miscellaneous set to AES-NI and BSD Crypto Device (aesni, cryptodev).
This is the update that I referred to in my previous post where both are enabled.OpenVPN Client & Server:
And these are the options available to me within OpenVPN:
Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?
-
@kevindd992002 said in PC Engines apu2 experiences:
Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?
With regards to having both loaded, see resolved bug 7810.
In the OpenVPN settings, selecting BSD cryptodev engine made it slower indeed and shouldn't be selected.
Not related to the crypto, but what also made a big difference is setting the following:
Anyway, this is just my opinion of how it should be since this works the best in my situation.
-
@veldkornet said in PC Engines apu2 experiences:
@kevindd992002 said in PC Engines apu2 experiences:
Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?
With regards to having both loaded, see resolved bug 7810.
In the OpenVPN settings, selecting BSD cryptodev engine made it slower indeed and shouldn't be selected.
Not related to the crypto, but what also made a big difference is setting the following:
Anyway, this is just my opinion of how it should be since this works the best in my situation.
Ahh, I see what you mean. Thanks for the heads up. I'm setting it that way then.
Yeah, I have both UDP Fast I/O and send/receiver buffer set to those values also as I read they can speed up things.
-
Setting those tweaks will definitely show an improvement. Usually a significant one though I've never tested it on the APU2 myself.
OpenSSL should use the AES-NI instructions on your CPU directly if it supports them. The danger here is that instead of using them directly it tries to use the BSD crypto framework where the AES-NI kernel module has registered itself for the algorithms it supports. That means a load of additional cycles to do the same calculation.
As long as you don't have AES-NI+BSD crypto set in Adv. > MIsc. and BSD crypto set in openvpn you should avoid that.
When I last tested it Fast I/O and send/receiver buffer made a greater difference to throughput.
Steve
-
@stephenw10 said in PC Engines apu2 experiences:
Setting those tweaks will definitely show an improvement. Usually a significant one though I've never tested it on the APU2 myself.
OpenSSL should use the AES-NI instructions on your CPU directly if it supports them. The danger here is that instead of using them directly it tries to use the BSD crypto framework where the AES-NI kernel module has registered itself for the algorithms it supports. That means a load of additional cycles to do the same calculation.
As long as you don't have AES-NI+BSD crypto set in Adv. > MIsc. and BSD crypto set in openvpn you should avoid that.
When I last tested it Fast I/O and send/receiver buffer made a greater difference to throughput.
Steve
I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?
-
@kevindd992002 said in PC Engines apu2 experiences:
I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?
Im interested in this too! And Im a bit confused after reading 20 posts about it in this thread.
cheers -
@kevindd992002 said in PC Engines apu2 experiences:
I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?
That's what I would expect. In 2.4 at least.
The last time I tested this though I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled. Use AES-GCM and enable fastio and larger send/rec buffers.
That was a while back, 2.3.4 vs 2.4.0.More testing is always good.
Steve
-
@stephenw10 said in PC Engines apu2 experiences:
I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled
What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
But I havent done your test with all OFF. -
@daemonix said in PC Engines apu2 experiences:
@stephenw10 said in PC Engines apu2 experiences:
I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled
What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
But I havent done your test with all OFF.I currently get around 20Mbit as well, up and down, sometimes a bit more.
I have a 400/40 Mbit line.
I’ll need to double check my settings, but I think I have 256bit encryption.
-
@veldkornet said in PC Engines apu2 experiences:
@daemonix said in PC Engines apu2 experiences:
@stephenw10 said in PC Engines apu2 experiences:
I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled
What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
But I havent done your test with all OFF.I currently get around 20Mbit as well, up and down, sometimes a bit more.
I have a 400/40 Mbit line.
I’ll need to double check my settings, but I think I have 256bit encryption.
pif Im getting 40Mbit max here. 128-GCM too.
I found this page: https://github.com/ocochard/netbenches/blob/master/AMD_GX-412TC_4Cores_Intel_i210AT/openvpn/results/fbsd11.0/README.md
But I cant see any luck. my test config below with APU2 running the openvpn server.
EDIT: this page too: https://teklager.se/en/knowledge-base/apu2-vpn-performance/
-
Just wanted to find out how everyone is getting on with 2.4.4?
I had Firmware 4.8.0.4 on my APU2 with 2.4.3 and everything was fine, but after upgrading to 2.4.4 I had lots of “stalls”, both via the web interface and via SSH. I often had to restart the SAH session (or refresh the web page).
I downgraded the firmware all the way back down to 4.0.7, and the stalls seem to be gone, although I can’t exactly say that everything is as snappy as I’d expect it to be, changing screens on the GUI still take quite a bit of time.
If I restart unbound, it seems to take a good 5 minutes before it actually starts resolving.
I seem to have to restart PHP-FPM pretty often to get the interface in a working state (never had to do this before).Anyone else seeing this? Or whats your experience been so far with 2.4.4?
-
I'd be surprised if that was anything to do with the Coreboot version really. About the only thing I could imagine doing that would be some component that is initiallised differently and only supported in FreeBSD 11.2. But I'm not aware of that.
I would first backup the config and do a clean 2.4.4 install. If you still see the same issues you did in the upgraded 2.4.4 then did deeper. I would expect to see errors logged though.Steve
-
I need a recommendation on a console cable for the APU1/APU2 units. My laptop will be retired soon and so I'll no longer have a serial port to use. I'm sure there are USB cables that will connect in. Does anyone have a link to one that they use that we know will work? Thanks for the help!
-
@stewart said in PC Engines apu2 experiences:
I need a recommendation on a console cable for the APU1/APU2 units. My laptop will be retired soon and so I'll no longer have a serial port to use. I'm sure there are USB cables that will connect in. Does anyone have a link to one that they use that we know will work? Thanks for the help!
I have this one, works well for me: https://www.startech.com/eu/m/Cards-Adapters/Serial-Cards-Adapters/USB-to-Null-Modem-RS232-DB9-Serial-Adapter-Cable-DCE-FTDI~ICUSB232FTN
-
@ Veldkornet
Think this is the same thing?
https://www.amazon.com/USB-Serial-Adapter-Modem-9-pin/dp/B008634VJY/ref=sr_1_3?ie=UTF8&qid=1539289152&sr=8-3&keywords=startech+usb+null+modemEDIT: Found the model on the box in the image. It is indeed. Thanks for the rec!
-
@stewart said in PC Engines apu2 experiences:
@ Veldkornet
Think this is the same thing?
https://www.amazon.com/USB-Serial-Adapter-Modem-9-pin/dp/B008634VJY/ref=sr_1_3?ie=UTF8&qid=1539289152&sr=8-3&keywords=startech+usb+null+modemEDIT: Found the model on the box in the image. It is indeed. Thanks for the rec!
Yup, looks like the same one indeed! :)
