PC Engines apu2 experiences
-
@Veldkornet said in PC Engines apu2 experiences:
Is anyone using the CoDel / FQ_CoDel Traffic Shaping on the APU2?
Working well? Any problems?
I have an APU2 box at work to provide a separate network for personal devices. It is setup with the FQ_CoDel limiter / floating rules method described towards the end of the Playing with FQ-CoDel Thread. It has been rock solid and seems to provide equal bandwidth sharing for the 30 - 50 devices connected each day and 16 - 20 GB of traffic that is passed on our 150/150 FiOS link.
-
@Veldkornet said in PC Engines apu2 experiences:
@qinn SSH into it and install flashrom. No need to boot from USB etc.
pkg install flashrom
Upload the firmware to /tmp with scp and run:
flashrom -w /tmp/apu2_v4.9.0.2.rom -p internal:boardmismatch=forceShutdown pfSense, pull the power for 10 seconds, then boot up.
I still run the original (legacy) bios that came with my apu2c4 almost 2 years ago?! (maybe 1 year I cant remember). I also run the latest stable pfsense.
Anything I need to do (regarding settings or something else) before flushing from the pfsense itself??
thanks -
@daemonix Nope, just install the flashrom like above, then download the latest Mainline from here
https://pcengines.github.io/
then flash it and reboot, I have switched from Legacy to Mainline months ago and everything works still fine.
...and btw you don't need the force option, this is enough
flashrom -w /tmp/apu2_v4.9.0.7.rom -p internal
-
@Qinn said in PC Engines apu2 experiences:
@daemonix Nope, just install the flashrom like above, then download the latest Mainline from here
https://pcengines.github.io/
then flash it and reboot, I have switched from Legacy to Mainline months ago and everything works still fine.
...and btw you don't need the force option, this is enough
flashrom -w /tmp/apu2_v4.9.0.7.rom -p internal
Thanks a lot for the quick replay!
Im do it later in the evening and hopefully Ill have internet after the reboot heheheh -
This post is deleted! -
@Qinn said in PC Engines apu2 experiences:
@daemonix Nope, just install the flashrom like above, then download the latest Mainline from here
https://pcengines.github.io/
then flash it and reboot, I have switched from Legacy to Mainline months ago and everything works still fine.
...and btw you don't need the force option, this is enough
flashrom -w /tmp/apu2_v4.9.0.7.rom -p internal
Done without a problem!
I had a serial link to it so I did it from there so I can see the boot sequence.Now that I have time to experiment a bit.
What are the recommended combination of settings that favour performance on a openvpn server nowadays ?BSD crypto ON/OFF? CBC/GBC algo? etc..
I get 40mbit on the apu2 hosted server. -
@daemonix said in PC Engines apu2 experiences:
BSD crypto ON/OFF? CBC/GBC algo? etc..
I get 40mbit on the apu2 hosted server.From my knowledge for the APU2-Board the settings should be AES-NI (in CPU).
Regards,
fireodo -
I agree try AES-NI (in cpu) read this please, especially the reply from "jimp" https://forum.netgate.com/topic/114212/aes-ni-cryptodev-openvpn-help-a-n00b-understand/16
The setting is in :
System/Advanced/Miscellaneous
try it and see how it performs.
-
fast-io
sndbuf 524288
rcvbuf 524288added this, changed my PIA client to GCM (my server was already GCM) and I already had just the hardware acceleration only...
Gone from 45-sih mbit to 70-70mbit in both PIA and my server!!! -
@Qinn said in PC Engines apu2 experiences:
https://pcengines.github.io/
Does this mean that "AES-NI CPU-based acceleration" is better than the "AES-NI and BSD Crypto Device" option? I'm still confused what the difference between those two are.
-
@kevindd992002 said in PC Engines apu2 experiences:
Does this mean that "AES-NI CPU-based acceleration" is better than the "AES-NI and BSD Crypto Device" option? I'm still confused what the difference between those two are.
Yes! The Apu2 does not have a dedicated Crypto-Device, the Crypto-Functions are integrated in the CPU (much faster). IMHO
-
@fireodo said in PC Engines apu2 experiences:
@kevindd992002 said in PC Engines apu2 experiences:
Does this mean that "AES-NI CPU-based acceleration" is better than the "AES-NI and BSD Crypto Device" option? I'm still confused what the difference between those two are.
Yes! The Apu2 does not have a dedicated Crypto-Device, the Crypto-Functions are integrated in the CPU (much faster). IMHO
I see. But won't it use AES-NI anyway if the latter option is selected?
Also, in the OpenVPN settings you should chhose None in the Hardware Acceleration field, correct?
-
@kevindd992002 said in PC Engines apu2 experiences:
I see. But won't it use AES-NI anyway if the latter option is selected?
Freebsd will look for the Crypto-Device wich is not existent and will not fallback to AES-NI CPU based.
Also, in the OpenVPN settings you should chhose None in the Hardware Acceleration field, correct?
I admit I dont know. Sorry.
-
@fireodo said in PC Engines apu2 experiences:
@kevindd992002 said in PC Engines apu2 experiences:
I see. But won't it use AES-NI anyway if the latter option is selected?
Freebsd will look for the Crypto-Device wich is not existent and will not fallback to AES-NI CPU based.
Also, in the OpenVPN settings you should chhose None in the Hardware Acceleration field, correct?
I admit I dont know. Sorry.
Yes this is it. I did all the possible test combinations.
Indeed ONLY AES-NI should be selected -
Yes, the only thing to avoid here is enabling both aes-ni and bsd crypto. Doing that will cause the aes device to register for crypto acceleration via the framework which adds a load of additional steps. It's much faster to use the available CPU instructions directly. As long as it's enabled in the BIOS openssl, and hence openvpn, should use aes-ni.
Steve
-
@stephenw10 said in PC Engines apu2 experiences:
Yes, the only thing to avoid here is enabling both aes-ni and bsd crypto. Doing that will cause the aes device to register for crypto acceleration via the framework which adds a load of additional steps. It's much faster to use the available CPU instructions directly. As long as it's enabled in the BIOS openssl, and hence openvpn, should use aes-ni.
Steve
So you have to select AES-NI in pfSense and not in OpenVPN, then why is this option (Hardware crypto) present in OpenVPN config within pfSense? Could you please clarify this?
Cheers Qinn
-
I have personally never used that setting. But I have also never had a device with a specifically supported hardware crypto device which is where I would expect it to apply.
In testing I did when we went to OpenVPN 2.4 it was better to leave that set to None in every case.Steve
-
@stephenw10 kudos for clearing that one up!
-
@stephenw10 said in PC Engines apu2 experiences:
But I have also never had a device with a specifically supported hardware crypto device which ...
Don't want to crush this topic (and can't PM you) but lemme ask how far crypto in the SG-1100 has come? Last thing I know is that HW is present and waits for the software to follow. Anything changed in this regard?
-
That is still basically the status. I'm not sure how far along that work is, I did see some discussion of it a few days ago.
But that's a good point. On the SG-3100 where the crypto hardware is supported via the CESA driver I am currently running with BSD Crypto device set in both OpenVPN and as the system crypto device.
Steve
-
Although it is not downloadable at the moment, did anyone tried the new v4.10.0.0?
https://pcengines.github.io/
-
@Qinn said in PC Engines apu2 experiences:
Although it is not downloadable at the moment, did anyone tried the new v4.10.0.0?
https://pcengines.github.io/
There isnt any 4.10.0.0 version - look here:
https://3mdeb.com/open-source-firmware/pcengines/ -
Here it is: https://pcengines.github.io/#mr-25
v4.10.0.0 Release date: '2019-08-09' Fixed/added: - rebased with official coreboot repository commit 2a20d13 - enable basic ACPI support for GPIOs
-
@Veldkornet said in PC Engines apu2 experiences:
Here it is: https://pcengines.github.io/#mr-25
v4.10.0.0 Release date: '2019-08-09' Fixed/added: - rebased with official coreboot repository commit 2a20d13 - enable basic ACPI support for GPIOs
Have you download it?
-
Do you guys have any issues with the download links for v4.10.0.0? They're all "404 page not found" for me. Or were they removed intentionally?
-
@kevindd992002 Yes, I don't think it's build
https://github.com/pcengines/coreboot/compare/v4.9.0.7...v4.10.0.0
-
Just updated. Link is properly working. No issues so far.
-
@psp said in PC Engines apu2 experiences:
Just updated. Link is properly working. No issues so far.
Thanks!
-
@psp said in PC Engines apu2 experiences:
Just updated. Link is properly working. No issues so far.
Yup thanks
-
New APU2 user here. Recently upgraded from an EdgeRouter Lite to the APU2D4. So far, loving pfSense, it's much more flexible than the ERL.
The BIOS it shipped with was 20170228 and I was able to press F10 to access the boot menu and perform a memtest.
PCEngines apu2 coreboot build 20170228 4080 MB ECC DRAM SeaBIOS (version rel-1.10.0.1) Press F10 key now for boot menu Select boot device: 1. USB MSC Drive Kingston DataTraveler 3.0 PMAP 2. ata0-0: Samsung SSD 860 EVO mSATA 250GB ATA-11 Hard-Disk (2 3. Payload [memtest] 4. Payload [setup]
I upgraded the BIOS to 20190808 (v4.10.0.0) using flashrom and now when I press F10, I get the message "Booting from Hard Disk..." and it just starts to boot via the internal SSD. How can I access memtest again?
PC Engines apu2 coreboot build 20190808 BIOS version v4.10.0.0 4080 MB ECC DRAM SeaBIOS (version rel-1.12.1.3-0-g300e8b7) Press F10 key now for boot menu Booting from Hard Disk... /boot/config: -S115200 -h Consoles: serial port BIOS drive C: is disk0 BIOS 639kB/3405392kB available memory FreeBSD/x86 bootstrap loader, Revision 1.1 (Wed Nov 21 08:03:01 EST 2018 root@buildbot2.nyi.netgate.com) ... ... Boot continues here ... ...
EDIT: Also, how do I enter the BIOS to adjust settings? It seems that option is missing as well.
-
my first guess would be that the APU doesn't like what your terminal client is sending as F10. I'd try looking for options about what escape sequences are sent for F keys, or try a different client.
-
Hello everyone. After a year of learning this APU2 system, I would like to share some not-so-obvious things with anybody new to this topic:
-
ECC RAM is only available on APU2x4 edition (where x equals "a" or "b" or "c" or "d"), which means the motherboard has the total of 4GB RAM soldered. The 2GB RAM boards DO NOT CONTAIN the necessary ECC hardware. So that is hardware limitation. From the firmware side, you must upgrade to a recent Coreboot version, as the older versions have not enabled ECC, not even for the 4 GB boards! Reading pcengines.ch page (the official page of the vendor, who is selling these boarda via their distributors) this is totally not obvious from their datasheets. So you have been warned.
-
PPPoE: if your internet provider is using PPPoE protocol, and you have purchased a Fiber WAN over Gigabit, the APU2 cannot reach that 1 Gigabit in real life. More realistic is somewhere between 200 and 650 Mbit (the latter is the absolute maximum under real life condtiona). Reason: the PPPoE is single-threaded under *BSD, and the 1 Ghz cores in this SoC cannot handle that amount of traffic. PF and NAT can (and fornsure will!) decrease this value even further. There are no surprises, clock speed wins over core count in single threaded code. So this should set your expectations when your ISP is using PPPoE over that gigabit fiber connection!
-
Core Performance Boost, or in short CPB. That is the AMD equivalent of the Intel Turbo boost feature. The CPU in this APU2 AMD GX-412 SoC has base clock speed of 1 Ghz, and it has only single-core boost, which is 1.4 Ghz. If more than 1 core is busy, you can reach (not sure if that is even a valid CPB step) only 1.2 Ghz. Worst case if all 4 cores are busy, no turbo boost can happen at all. Extra fact: the CPB activation/deactivation is entirely automatic, triggered by the CPU internally. The firmware, or the OS has zero control over it. You cannot even see the clockspeed reaching over 1Ghz, even if CPB was active in that moment. The only control the firmware has, that you MUST have a recent Coreboot Firmware version to have CPB enabled at all. Older fw does not even have the CPB enablement implemented, so with older firmware you will never see single-core turbo boost.
-
after the firmware upgrade is conpleted, YOU MUST SWITCH THE APU OFF completely. E.g. power it off! It was earlier not written very clearly, but fortunately the Coreboot firmware page for APU (pcengines.github.io) has been updated to explain this better: at the end of the successful firnware update, you must turn the hardware off completely to clear some leftover registers in the system, that cannot be cleared via software-initiated restart. If you dont do this, the result is not guaranteed.
-
Coreboot mainline is preferred over legacy, since the 4.9.x version. The legacy is no longer necessarily "better" for *BSD, this statement was true only for older 4.8.x or 4.6.x.
-
-
@soder said in PC Engines apu2 experiences:
- after the firmware upgrade is completed, YOU MUST SWITCH THE APU OFF completely. E.g. power it off! It was earlier not written very clearly, but fortunately the Coreboot firmware page for APU (pcengines.github.io) has been updated to explain this better: at the end of the successful firmware update, you must turn the hardware off completely to clear some leftover registers in the system, that cannot be cleared via software-initiated restart. If you don't do this, the result is not guaranteed.
Thanks for the info and powering off after a firmware update I did not know ! Can you point out to me were this is written, on their site, as I can't seem to locate it.
Thanks and Cheers Qinn
-
Thanks @soder, your report is really appreciated. I've still have some of these APU2 running in HA for over 700 days in a datacenter without any issues. These devices are quite good cheap solutions for datalines up to 100 Mbps.
-
@psp said in PC Engines apu2 experiences:
Thanks @soder, your report is really appreciated. I've still have some of these APU2 running in HA for over 700 days in a datacenter without any issues. These devices are quite good cheap solutions for datalines up to 100 Mbps.
What kinda hardware would you recommend for higher speeds, than 100Mbps?
-
@Qinn
Any Intel Atom E3940 or (better) C3558 based boards for up to 1Gbps. -
@Qinn, check out this Github page.
https://github.com/pcengines/apu2-documentation/blob/master/docs/firmware_flashing.md#corebootrom-flashing
-
@Qinn said in PC Engines apu2 experiences:
Hardeware: APU2C4 16gb mSATA SSD - Bios: Coreboot Mainline Version 4.10.0.0 - Firmware: Latest-stable-pfSense (amd64)
May I ask you if you use the apuled.ko kernel module to drive the 3 Front-LEDs and if so is this still working after update to the mainline coreboot bios?
Thank you,
fireodo -
@Qinn said in PC Engines apu2 experiences:
@psp said in PC Engines apu2 experiences:
Thanks @soder, your report is really appreciated. I've still have some of these APU2 running in HA for over 700 days in a datacenter without any issues. These devices are quite good cheap solutions for datalines up to 100 Mbps.
What kinda hardware would you recommend for higher speeds, than 100Mbps?
Supposedly the APU2 is pretty decent out-of-box up to 600Mbps, but can be tweaked to run at 1Gbps.
https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/ -
@logan5247
So it's just addinghw.igb.rx_process_limit="-1" hw.igb.tx_process_limit="-1"
that does the trick? I wonder what those actually do. Don't limit what? Isn't it a cpu cap anyway? I'll have to try those out and see what I get. I usually tell people they are good for 500Mbps and most of the time they can with Suricata, Squid, and pfBlocker on. Not a lot of wiggle room, though. Not sure if these would help boost that number or not but it sure would be nice to be able to hit 1Gbps speeds with these boxes. I wonder if there is development on a more powerful model based on a Ryzen CPU but I don't know of any with a low enough TDP. These Geode's are only 6W TDP. The lowest Ryzen I've seen is 12W.