Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PC Engines apu2 experiences

    Scheduled Pinned Locked Moved Hardware
    711 Posts 73 Posters 793.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • QinnQ
      Qinn @Qinn
      last edited by

      ...and I would install Service_Watchdog, that way all you services are always up and running.

      Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
      Firmware: Latest-stable-pfSense CE (amd64)
      Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

      S 1 Reply Last reply Reply Quote 0
      • K
        kevindd992002
        last edited by

        For the apu2c4, would IPSec be faster than OpenVPN for site-to-site VPN? I have OpenVPN configured now but I'm not sure if it's wise to switch to IPSec as I read that it's almost always faster than OpenVPN.

        C 1 Reply Last reply Reply Quote 0
        • C
          cysiacom @kevindd992002
          last edited by

          @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
          On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

          PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

          iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
          After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
          Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

          OpenVPN: TLS Auth - 128-GCM- DH2
          Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

          Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
          I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
          Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

          K 1 Reply Last reply Reply Quote 0
          • K
            kevindd992002 @cysiacom
            last edited by

            @cysiacom said in PC Engines apu2 experiences:

            @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
            On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

            PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

            iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
            After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
            Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

            OpenVPN: TLS Auth - 128-GCM- DH2
            Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

            Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
            I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
            Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

            I don't understand though, why is there a "provider" involved between the IPSec tunnel that poses these crypto settings requirement? I thought it's just like OpenVPN where one end would be the VPN server and the other end the client?

            C 1 Reply Last reply Reply Quote 0
            • S
              stefanl @Qinn
              last edited by

              @Qinn Thanks for the tips. I don't think I'll reinstall pfSense to a single disk ZFS. My NAS haves it, but it haves 12x 4TB disks.

              Regarding watchdog, it's enabled and working, but somehow it isn't showing on the dashboard nor in status --> services.

              QinnQ 1 Reply Last reply Reply Quote 0
              • C
                cysiacom @kevindd992002
                last edited by

                @kevindd992002 I'm afraid I didn't explain myself clearly.
                Sorry for that.

                One customer needed to access to their provider servers.
                The access is granted via Site2Site IPsec VPN with very low crypto requirements and also quite unusual IPs (they use "fake" public IPs inside the tunnel).
                We had some trouble getting Phase2 working for that customer and provider so we planned some lab tests.
                There's no need for provider at all in general but just for this case, our customer, in particular.

                When doing local tests we did realize the speed change on IPSec tunnels compared to OpenVPN so we did some other test for our own purposes.
                Again.

                There's no need for external or any provider. It was just only on our specific customer needs.

                K 1 Reply Last reply Reply Quote 0
                • K
                  kevindd992002 @cysiacom
                  last edited by kevindd992002

                  @cysiacom said in PC Engines apu2 experiences:

                  @kevindd992002 I'm afraid I didn't explain myself clearly.
                  Sorry for that.

                  One customer needed to access to their provider servers.
                  The access is granted via Site2Site IPsec VPN with very low crypto requirements and also quite unusual IPs (they use "fake" public IPs inside the tunnel).
                  We had some trouble getting Phase2 working for that customer and provider so we planned some lab tests.
                  There's no need for provider at all in general but just for this case, our customer, in particular.

                  When doing local tests we did realize the speed change on IPSec tunnels compared to OpenVPN so we did some other test for our own purposes.
                  Again.

                  There's no need for external or any provider. It was just only on our specific customer needs.

                  No worries, I see what you mean now. Would you happen to have any best practice guide in setting up an IPSec tunnel in pfsense, at least for a home setup with "relaxed but secure" crypto requirements?

                  As for Remote Access VPN, is OpenVPN still the way to go? There's nothing stopping me of creating an IPSec s2s link and an OpenVPN remote access VPN gateway, right?

                  1 Reply Last reply Reply Quote 0
                  • QinnQ
                    Qinn @stefanl
                    last edited by Qinn

                    @stefanl said in PC Engines apu2 experiences:

                    @Qinn Thanks for the tips. I don't think I'll reinstall pfSense to a single disk ZFS. My NAS haves it, but it haves 12x 4TB disks.

                    Regarding watchdog, it's enabled and working, but somehow it isn't showing on the dashboard nor in status --> services.

                    I thought and have the same, this service isn't present in the services status, but if you stop 1 service, you will see it will come up again (if you have an mail address setup for notices, you should also receive a mail. Maybe read https://forum.netgate.com/topic/59761/new-package-service-watchdog/9

                    Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                    Firmware: Latest-stable-pfSense CE (amd64)
                    Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                    1 Reply Last reply Reply Quote 0
                    • fireodoF
                      fireodo
                      last edited by

                      Bios Update from 4.11.0.4 to 4.11.0.6 on APU2C0, until now without any issues :-)

                      Regards,
                      fireodo

                      Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                      SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                      pfsense 2.8.0 CE
                      Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                      1 Reply Last reply Reply Quote 1
                      • D
                        dugeem
                        last edited by

                        Yes the PCIe power management feature is now disabled by default (ie maximum performance) and ACPI change reverted (ie sysctl dev.cpu works).

                        Detailed APU* Coreboot v4.11.0.6 release notes:

                        1. Rebased with official coreboot repository commit d6f7ec5.
                        2. Updated sortbootorder to v4.6.18 bringing the PCI Express power management features runtime option. For details refer to sortbootorder documentation. When PCI Express power management features features are enabled, the network controllers (NICs and WIFi cards) may have reduced performance at the cost of reduced power consumption. By default this option will be disabled to not impact the network performance.
                        3. Reverted changes to ACPI CPU definitions causing BSD systems to not probe CPU frequency driver. The ACPI compliance of current BSD systems is not up to date, the situation should improve when the distribution will start to use FreeBSD 12.x, which works well with most recent rules of defining processors in ACPI.
                        4. Reverted changes with PCIe reset logic causing mPCIe2 slot connected modules to not appear in OS. The change did more harm than good. We are working to improve the PCIe modules detection in firmware, which is dependent on the AGESA.
                        5. Added IOMMU IVRS generation expanded with IVHD type 11h for newer Xen. This change should allow newer Xen images to utilize more IOMMU features.
                        6. Fixed memtest hang on apu1.
                        7. Fixed TPM2 detection on FreeBSD 12.1. Since FreeBSD 12.1 the TPM2 support is available along with FreeBSD ports offering TPM2 tools. We will provide documentation how to install and utilize those tools on FreeBSD systems soon.
                        8. Fixed a problem where SD 3.0 mode could not be disabled.

                        Applied to apu2c4 test system. All good so far.

                        K 1 Reply Last reply Reply Quote 1
                        • K
                          kevindd992002 @dugeem
                          last edited by

                          @dugeem said in PC Engines apu2 experiences:

                          Yes the PCIe power management feature is now disabled by default (ie maximum performance) and ACPI change reverted (ie sysctl dev.cpu works).

                          Detailed APU* Coreboot v4.11.0.6 release notes:

                          1. Rebased with official coreboot repository commit d6f7ec5.
                          2. Updated sortbootorder to v4.6.18 bringing the PCI Express power management features runtime option. For details refer to sortbootorder documentation. When PCI Express power management features features are enabled, the network controllers (NICs and WIFi cards) may have reduced performance at the cost of reduced power consumption. By default this option will be disabled to not impact the network performance.
                          3. Reverted changes to ACPI CPU definitions causing BSD systems to not probe CPU frequency driver. The ACPI compliance of current BSD systems is not up to date, the situation should improve when the distribution will start to use FreeBSD 12.x, which works well with most recent rules of defining processors in ACPI.
                          4. Reverted changes with PCIe reset logic causing mPCIe2 slot connected modules to not appear in OS. The change did more harm than good. We are working to improve the PCIe modules detection in firmware, which is dependent on the AGESA.
                          5. Added IOMMU IVRS generation expanded with IVHD type 11h for newer Xen. This change should allow newer Xen images to utilize more IOMMU features.
                          6. Fixed memtest hang on apu1.
                          7. Fixed TPM2 detection on FreeBSD 12.1. Since FreeBSD 12.1 the TPM2 support is available along with FreeBSD ports offering TPM2 tools. We will provide documentation how to install and utilize those tools on FreeBSD systems soon.
                          8. Fixed a problem where SD 3.0 mode could not be disabled.

                          Applied to apu2c4 test system. All good so far.

                          Do you happen to know the what the recommendations are for the APU2C4 v4.11.0.6 BIOS settings to achieve max performance?

                          D 1 Reply Last reply Reply Quote 1
                          • D
                            dugeem @kevindd992002
                            last edited by

                            @kevindd992002 said in PC Engines apu2 experiences:

                            Do you happen to know the what the recommendations are for the APU2C4 v4.11.0.6 BIOS settings to achieve max performance?

                            Defaults are fine. Only two that matter are:

                            1. Core Performance Boost (CPB) enabled (default)
                            2. PCIe Power Management disabled (now default)

                            Cheers

                            D K 2 Replies Last reply Reply Quote 1
                            • D
                              daemonix @dugeem
                              last edited by

                              @dugeem good to know. Im planning to update too.

                              1 Reply Last reply Reply Quote 0
                              • K
                                kevindd992002 @dugeem
                                last edited by

                                @dugeem said in PC Engines apu2 experiences:

                                @kevindd992002 said in PC Engines apu2 experiences:

                                Do you happen to know the what the recommendations are for the APU2C4 v4.11.0.6 BIOS settings to achieve max performance?

                                Defaults are fine. Only two that matter are:

                                1. Core Performance Boost (CPB) enabled (default)
                                2. PCIe Power Management disabled (now default)

                                Cheers

                                Got it. I know CPB enabled is the default but was disabled in the older BIOS'es where it was initially introduced. Just for the heck of it, is there a way to know if CPB is enabled aside from checking it during runtime? I currently don't have physical access to the pfsense boxes.

                                QinnQ 1 Reply Last reply Reply Quote 0
                                • QinnQ
                                  Qinn @kevindd992002
                                  last edited by

                                  @kevindd992002 I only know

                                  dmidecode
                                  

                                  on the command line, but sadly it doesn't show the CPB

                                  Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                                  Firmware: Latest-stable-pfSense CE (amd64)
                                  Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                                  1 Reply Last reply Reply Quote 0
                                  • K
                                    kevindd992002
                                    last edited by

                                    @Qinn said in PC Engines apu2 experiences:

                                    /boot/loader.conf.local

                                    @cysiacom and others

                                    Not sure why but my OpenVPN remote access server only 55-60Mbps speed when I'm trying to saturate the link. I'm not sure if my test is right but I connect my local laptop (same network as the OpenVPN remote access server) to the OpenVPN server and download a game in steam. My Internet connection is 300Mbps symmetrical speed, so I'm expecting much higher speed. As soon as I disconnect the VPN connection of the laptop, the download speed soars up to saturate the 300Mbps link.

                                    Here are my OpenVPN settings:

                                    7365ab28-4712-4b8a-bdc4-2256df7e1322-image.png
                                    1597bdc9-fb5e-4aa5-a717-0a1fb9108206-image.png
                                    901867ed-e370-4e3b-8772-ce7faa7b62c6-image.png
                                    922fac72-2235-4b67-9607-3625416a4258-image.png
                                    013095fd-bba4-4d66-bfeb-de74c951a75a-image.png
                                    d40459f8-a21e-40ef-ace1-d8d1cdf9a6b9-image.png
                                    160adaa9-7e41-4c27-906f-79f5685c8152-image.png

                                    When my laptop is NOT connected to the VPN server, here are the traffic graphs:

                                    51d2bf52-9f75-4ffe-89b2-75c539edd0cf-image.png
                                    4041d534-7f60-4af9-80b9-d7b1d5a42311-image.png

                                    As soon as I connect it to the VPN server:

                                    e57878e2-3758-44f7-9984-80d969a76331-image.png
                                    170bcf94-ee0d-4700-8fca-c5b85697e0bf-image.png

                                    See the sudden drop in speed? When I disconnect from the VPN server, the speed goes back up to the saturation point. Any ideas?

                                    1 Reply Last reply Reply Quote 0
                                    • DaddyGoD
                                      DaddyGo
                                      last edited by

                                      We use more than 20 APU4 boards in our system with OpenVPN.
                                      The value will not be higher! (cca. 55 Mbps)
                                      This is what the APU board can screw out of of itself, since OpenVPN uses only 1 CPU core and in this case is1 .0 GHz or 1.4 GHz (1.4 if you do the tuning, but that only applies to one core).
                                      We've been experimenting for a long time to achieve higher speeds, but low CPU clock can do just that with OpenVPN.

                                      Under the same conditions, a Supermicro M11SDV-4C-LN4F-based pfSense 310 to 340 Mbps can be accessed in the same location with the same installation.
                                      This is proof of the above.

                                      BTW, your setting is completely correct!
                                      Regarding OpenVPN, the CPU clock is your best friend.

                                      Cats bury it so they can't see it!
                                      (You know what I mean if you have a cat)

                                      1 Reply Last reply Reply Quote 0
                                      • K
                                        kevindd992002
                                        last edited by

                                        That's what I thought. Does that mean that the CPU usage in the pfsense dashboard means overall CPU usage and not just single core? I just don't see it peaking at 100% while running the test. I tried it again with speedtest and got it to be higher:

                                        0a4ee760-c610-40b5-b0ed-24c1e6688a77-image.png
                                        https://www.speedtest.net/result/9415734802

                                        I don't understand how it got higher though. The battle.net game download I had earlier was saturating the link without the VPN connection so I was expecting it to be more or less the same as the speedtest result.

                                        Does that mean that it's better to do IPsec road warrior server then?

                                        1 Reply Last reply Reply Quote 0
                                        • DaddyGoD
                                          DaddyGo
                                          last edited by

                                          @kevindd992002 said in PC Engines apu2 experiences:

                                          https://www.speedtest.net

                                          Exactly! if you want better values then IPsec. (multithreading)
                                          Since we use ExpressVPN and our own OpenVPN tunnels, we can't switch to IPsec, we just accepted the fact that, these motherboards can do just that.
                                          The measurements depend on a lot of things, the current load and what else is running on the APU under pfSense, such as Squid, pfBlocketNG, Snort / Suricata.

                                          The graphs show aggregate values and can be suggestive.
                                          Measurements can also be performed in several ways, such as https://fast.com/ and https://speedof.me/ and https://www.meter.net/ping-test/.
                                          I personally don't like the https://www.speedtest.net

                                          Cats bury it so they can't see it!
                                          (You know what I mean if you have a cat)

                                          K 1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Run top -aSH at the command line while testing to see how the CPU cores are loaded.

                                            You have NCP enabled and you have AES-CBC and AES-GCM set as NCP ciphers. Which is it actually connecting with?

                                            I would disable NCP there to force it to use AES-128-GCM which you have selected. I would expect that to be fastest.

                                            Steve

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.