• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

PC Engines apu2 experiences

Scheduled Pinned Locked Moved Hardware
711 Posts 73 Posters 791.4k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • Q
    Qinn @kevindd992002
    last edited by Qinn Apr 25, 2020, 1:08 PM Apr 25, 2020, 1:07 PM

    @kevindd992002 said in PC Engines apu2 experiences:

    @Qinn said in PC Engines apu2 experiences:

    @dugeem Kudos for the tweaks, could you explain what each of these tweak will do/accomplish on a APU2C4?

    Btw my speeds here are not high, as I am mandatory of using aDSL which has a max of 10mb down and as it is combined with 4G a total of 60 down and a total of 15 up

    Cheers Qinn

    I thought he already did explain what the tweaks do on an APU2C4? It's in his post above.

    I know what number on does, but the other 2 I don't and I don't seem to read what these to do

    2.Disable ICMP Redirects to enable tryforward routing path (under System / Advanced / System Tunables set net.inet.ip.redirect & net.inet6.ip6.redirect to 0)

    3.Add hw.igb.rx_process_limit=-1 to /boot/loader.conf.local

    Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
    Firmware: Latest-stable-pfSense CE (amd64)
    Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

    D 1 Reply Last reply Apr 26, 2020, 7:59 AM Reply Quote 0
    • D
      dugeem @fireodo
      last edited by Apr 26, 2020, 7:30 AM

      @fireodo

      Weird. Has this issue been around for a while or has it appeared with 2.4.5?

      Any hints from PPPoE logging? Also you could try reducing interface MTU (though normally PPPoE gets it right)

      F 1 Reply Last reply Apr 26, 2020, 7:42 AM Reply Quote 0
      • F
        fireodo @dugeem
        last edited by Apr 26, 2020, 7:42 AM

        @dugeem said in PC Engines apu2 experiences:

        @fireodo
        Weird. Has this issue been around for a while or has it appeared with 2.4.5?

        No. I had this issue only after adding hw.igb.rx_process_limit=-1 to /boot/loader.conf.local
        AND then disable TSO & LRO in the advanced settings.

        Any hints from PPPoE logging? Also you could try reducing interface MTU (though normally PPPoE gets it right)

        The only thing that I found was: "ppp: can't lock /var/run/pppoe_wan.pid after 30 attempts"
        As I said somewhere up in the thread without those "tunings" everything work rocksolid.

        Have a nice Sunday,
        fireodo

        Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
        SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
        pfsense 2.8.0 CE
        Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

        1 Reply Last reply Reply Quote 0
        • D
          dugeem @kevindd992002
          last edited by Apr 26, 2020, 7:43 AM

          @kevindd992002 said in PC Engines apu2 experiences:

          By the way, why can't ICMP redirect be enabled at the same time as tryforward path?

          This is a FreeBSD kernel restriction. I think it comes from concern that if you have to start generating ICMP packets from within the fast routing path then you can potentially overwhelm gateways with ICMP redirects.

          Also, if tryforward was used ever since does that mean ICMP redirect was disabled by default back then?

          Prior to FreeBSD 10.3 ip_tryforward() there was ip_fastforward() routing path - which had to be explicitly enabled by setting sysctl net.inet.ip.fastforwarding=1. pfSense has always had ICMP Redirects enabled - although they were not working from 2.3 to 2.4.4 due to upstream bug with FreeBSD. This is now fixed in pfSense 2.4.5

          K 1 Reply Last reply Apr 26, 2020, 4:14 PM Reply Quote 0
          • D
            dugeem @Qinn
            last edited by Apr 26, 2020, 7:59 AM

            @Qinn said in PC Engines apu2 experiences:

            2.Disable ICMP Redirects to enable tryforward routing path (under System / Advanced / System Tunables set net.inet.ip.redirect & net.inet6.ip6.redirect to 0)

            As hinted above FreeBSD has for years incorporated a fast routing path which speeds packet forwarding. Prior to pfSense 2.3 it was the ip_fastforward path - which then was changed to ip_tryforward with pfSense 2.3 (FreeBSD 10.3). Now that a bug in ICMP Redirect handling in the fast path has been fixed it is necessary to change the specified ICMP Redirects sysctls to enable the fast routing path.

            3.Add hw.igb.rx_process_limit =-1 to /boot/loader.conf.local

            Sysctl hw.igb.rx_process_limit sets the limit number of received packets which can consecutively handled by igb driver. Default is 100. On multicore systems this can usually be set to unlimited (value -1).

            Personally I've had no problems but as per recent posts from @fireodo this has caused issues with PPPoE.

            Q 1 Reply Last reply Apr 27, 2020, 8:48 AM Reply Quote 1
            • K
              kevindd992002 @dugeem
              last edited by Apr 26, 2020, 4:14 PM

              @dugeem said in PC Engines apu2 experiences:

              @kevindd992002 said in PC Engines apu2 experiences:

              By the way, why can't ICMP redirect be enabled at the same time as tryforward path?

              This is a FreeBSD kernel restriction. I think it comes from concern that if you have to start generating ICMP packets from within the fast routing path then you can potentially overwhelm gateways with ICMP redirects.

              Also, if tryforward was used ever since does that mean ICMP redirect was disabled by default back then?

              Prior to FreeBSD 10.3 ip_tryforward() there was ip_fastforward() routing path - which had to be explicitly enabled by setting sysctl net.inet.ip.fastforwarding=1. pfSense has always had ICMP Redirects enabled - although they were not working from 2.3 to 2.4.4 due to upstream bug with FreeBSD. This is now fixed in pfSense 2.4.5

              Ok, so if I understand that right, pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled, correct? You need to enable them explicitly in the system tunables if you want to use them.

              D 1 Reply Last reply Apr 26, 2020, 11:20 PM Reply Quote 0
              • D
                dugeem @kevindd992002
                last edited by Apr 26, 2020, 11:20 PM

                @kevindd992002 said in PC Engines apu2 experiences:

                pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled

                The upstream bug was actually that the FreeBSD kernel didn't check the ICMP Redirect sysctls - which meant that ip_tryforward was always used and that ICMP Redirects did not work on FreeBSD 10.3 (pfsense 2.3) thru FreeBSD 11.2 (pfSense 2.4.4). So the pfSense defaults for these sysctls had no bearing on this issue until upstream fix was implemented and pfSense switched to that release (FreeBSD 11-STABLE).

                Here is the actual fix applied to FreeBSD kernel: https://svnweb.freebsd.org/base/stable/11/sys/netinet/ip_input.c?r1=332513&r2=338343&pathrev=338343

                K 1 Reply Last reply Apr 27, 2020, 9:11 AM Reply Quote 0
                • Q
                  Qinn @dugeem
                  last edited by Qinn Apr 27, 2020, 8:50 AM Apr 27, 2020, 8:48 AM

                  @dugeem Kudos so tweak 2 only favors, if you use ipv6?

                  Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                  Firmware: Latest-stable-pfSense CE (amd64)
                  Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                  D 1 Reply Last reply Apr 27, 2020, 11:50 AM Reply Quote 0
                  • K
                    kevindd992002 @dugeem
                    last edited by Apr 27, 2020, 9:11 AM

                    @dugeem said in PC Engines apu2 experiences:

                    @kevindd992002 said in PC Engines apu2 experiences:

                    pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled

                    The upstream bug was actually that the FreeBSD kernel didn't check the ICMP Redirect sysctls - which meant that ip_tryforward was always used and that ICMP Redirects did not work on FreeBSD 10.3 (pfsense 2.3) thru FreeBSD 11.2 (pfSense 2.4.4). So the pfSense defaults for these sysctls had no bearing on this issue until upstream fix was implemented and pfSense switched to that release (FreeBSD 11-STABLE).

                    Here is the actual fix applied to FreeBSD kernel: https://svnweb.freebsd.org/base/stable/11/sys/netinet/ip_input.c?r1=332513&r2=338343&pathrev=338343

                    Ok and for home networks where only one router/firewall (pfsense) is used anyway, ICMP Redirects aren't really being used, correct?

                    D 1 Reply Last reply Apr 27, 2020, 11:51 AM Reply Quote 0
                    • D
                      dugeem @Qinn
                      last edited by Apr 27, 2020, 11:50 AM

                      @Qinn

                      If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                      If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                      Q 1 Reply Last reply Apr 27, 2020, 12:40 PM Reply Quote 1
                      • D
                        dugeem @kevindd992002
                        last edited by Apr 27, 2020, 11:51 AM

                        @kevindd992002 said in PC Engines apu2 experiences:

                        for home networks where only one router/firewall (pfsense) is used anyway, ICMP Redirects aren't really being used, correct

                        Yes. ICMP Redirects are generally only used where there are two or more routers accessible on an IP network.

                        1 Reply Last reply Reply Quote 0
                        • Q
                          Qinn @dugeem
                          last edited by Qinn Apr 27, 2020, 12:43 PM Apr 27, 2020, 12:40 PM

                          @dugeem said in PC Engines apu2 experiences:

                          @Qinn

                          If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                          If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                          Done it. I looked for boot.conf.local in /boot ,but only found boot.conf? Do I overlook something?

                          btw do these tweaks need a reboot?

                          Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                          Firmware: Latest-stable-pfSense CE (amd64)
                          Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                          K 1 Reply Last reply Apr 27, 2020, 12:48 PM Reply Quote 0
                          • K
                            kevindd992002 @Qinn
                            last edited by Apr 27, 2020, 12:48 PM

                            @Qinn said in PC Engines apu2 experiences:

                            @dugeem said in PC Engines apu2 experiences:

                            @Qinn

                            If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                            If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                            Done it. I looked for boot.conf.local in /boot ,but only found boot.conf? Do I overlook something?

                            btw do these tweaks need a reboot?

                            You need to create the file if it isn't there (by default it isn't).

                            Q 1 Reply Last reply Apr 27, 2020, 1:10 PM Reply Quote 1
                            • Q
                              Qinn @kevindd992002
                              last edited by Apr 27, 2020, 1:10 PM

                              @kevindd992002 Done it!

                              Cheers Qinn

                              Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                              Firmware: Latest-stable-pfSense CE (amd64)
                              Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                              1 Reply Last reply Reply Quote 0
                              • S
                                stefanl
                                last edited by Apr 27, 2020, 3:10 PM

                                I've changed my settings as @dugeem mentioned and I see a somewhat lower CPU load and my network stays usable even when my Tor relay is doing 300mbit in/out.

                                37cc4650-19c8-4582-a782-d6951d67a3bd-image.png

                                Next step is to see if I can hookup my fiber connection (PPPoE) directly to pfSense and maintain those speeds.

                                Q 1 Reply Last reply Apr 27, 2020, 3:28 PM Reply Quote 2
                                • Q
                                  Qinn @stefanl
                                  last edited by Qinn Apr 27, 2020, 3:52 PM Apr 27, 2020, 3:28 PM

                                  @stefanl as I see you use a UPS, may I give you some advise?

                                  you are using ufs as a filesystem, I would move over to ZFS, as It will give you much more security. ZFS requires a system with ECC memory, otherwise you're still not 100% safe and the APU2C4 has ECC memory. So this, using ZFS, will guard you from filesystems damage etc. when a power surge comes along.

                                  Cheers Qinn

                                  Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                                  Firmware: Latest-stable-pfSense CE (amd64)
                                  Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                                  Q 1 Reply Last reply Apr 28, 2020, 6:30 AM Reply Quote 1
                                  • Q
                                    Qinn @Qinn
                                    last edited by Apr 28, 2020, 6:30 AM

                                    ...and I would install Service_Watchdog, that way all you services are always up and running.

                                    Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                                    Firmware: Latest-stable-pfSense CE (amd64)
                                    Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                                    S 1 Reply Last reply Apr 28, 2020, 3:29 PM Reply Quote 0
                                    • K
                                      kevindd992002
                                      last edited by Apr 28, 2020, 7:08 AM

                                      For the apu2c4, would IPSec be faster than OpenVPN for site-to-site VPN? I have OpenVPN configured now but I'm not sure if it's wise to switch to IPSec as I read that it's almost always faster than OpenVPN.

                                      C 1 Reply Last reply Apr 28, 2020, 8:59 AM Reply Quote 0
                                      • C
                                        cysiacom @kevindd992002
                                        last edited by Apr 28, 2020, 8:59 AM

                                        @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
                                        On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

                                        PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

                                        iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
                                        After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
                                        Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

                                        OpenVPN: TLS Auth - 128-GCM- DH2
                                        Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

                                        Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
                                        I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
                                        Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

                                        K 1 Reply Last reply Apr 28, 2020, 2:18 PM Reply Quote 0
                                        • K
                                          kevindd992002 @cysiacom
                                          last edited by Apr 28, 2020, 2:18 PM

                                          @cysiacom said in PC Engines apu2 experiences:

                                          @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
                                          On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

                                          PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

                                          iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
                                          After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
                                          Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

                                          OpenVPN: TLS Auth - 128-GCM- DH2
                                          Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

                                          Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
                                          I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
                                          Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

                                          I don't understand though, why is there a "provider" involved between the IPSec tunnel that poses these crypto settings requirement? I thought it's just like OpenVPN where one end would be the VPN server and the other end the client?

                                          C 1 Reply Last reply Apr 28, 2020, 6:25 PM Reply Quote 0
                                          401 out of 711
                                          • First post
                                            401/711
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received